Thoughts on CS7642: Reinforcement Learning

I know, I know, I’m guilty of not writing over the last four months. Things have been super hectic with project Voyager at Lazada, switching over to the new platform in end March and then preparing for our birthday campaign in end Apr. Any free time I had outside of that was poured into the Georgia Tech Reinforcement Learning (CS7642), which is the subject of this post.

The course was very enriching and fun. Throughout the course, we learnt techniques that allow one to optimize outcomes while navigating the world, and were introduced to several seminal and cutting edge (at least back in 2018) papers. I highly recommend this course for anyone who’s part of the Georgia Tech OMSCS.

An especially fun project involved landing a rocket in OpenAI’s LunarLander environment. This was implemented via deep reinforcement learning approaches. Here’s how the agent did on its first try (in reinforcement learning, we refer to the “models” as agents; more here). As you can see, it crashed right onto the surface. =(

Continue reading


Image search is now live!

After finishing the image classification API, I wondered if I could go further. How about building a reverse image search engine? You can try it out here: Image Search API

What is reverse image search?

In simple terms, given an image, reverse image search finds other similar images—this would be helpful in searching for similar looking products.

How do I use it?

“My son has this plushie he really likes, but I don’t know what the name is… How can I find similar plushies?”


Continue reading

Image classification API is now live!

After toiling for a few months on this, product image classification is now live on! While the product classification API works with product titles, the image classification API works with product images, though only for fashion.

Some facts about the image classification API:

  • Works best with e-commerce like fashion images (as that’s what it was trained on)
  • Top-1 validation accuracy: 0.76; Top-5 validation accuracy: 0.974
  • Returns results under 300 milliseconds (will be faster in batch mode with GPU)
  • Built on Keras and Theano, and runs on a tiny AWS server without GPU.

Continue reading