Artificial intelligence touches every aspect of our lives these days. However, one of the areas where AI has the most upside potential is in ecommerce. AI can offer benefits to merchants in numerous ways – in predicting consumer behavior, in inventory management and decision-making, in product design and accelerating design cycles and in shortening the time to make purchase decisions for consumers. AI also allows new and innovative discovery options to consumers, including using smartphones to shop for similar items using both existing images (Shop Instagram posts, for example) or by combining AI with augmented reality (AR) to discover similar products simply by pointing the phone's camera at an item of interest. New discovery options also include shopping for matches to any color in the spectrum. nFlate offers a number of AI-based solutions for both ecommerce and non-ecommerce companies, ranging from similar items carousels and predictive recommendations to our full AI ecommerce engine for partners with large userbases that want to add an incremental monetization option.
Similar Items Search
When it comes to fashion, people are primarily influenced visually. What we see is usually the 1st and most important component in the decision process, above price, material and brand. So it makes sense for fashion ecommerce merchants to offer a way for users to discover their products based on visual search. Today, cutting edge visual search technologies rely on Deep Learning or Convolutional Neural Networks. A handful of companies, including Google and Facebook have released open source foundational codebases that sophisticated software developers can use to build complex visual search models. While the development process is complex and often time-consuming, the end result can be nothing short of jaw dropping.
With visual search, consumers can submit images from their phone’s cameras (or any static image from the web) to find and buy items in a merchant’s catalog that are visually similar. In order to make visual search useful to consumers, a merchant needs to have a minimum of 10s of 1000s (preferably 100s of 1000s) of products. In our own SiBi reference marketplace, we have about 750k products, each with an average of 3-4 product images. Hundreds or even thousands of product images just won’t provide enough to deliver compelling matches.
nFlate can build a visual search engine for any company that has a large image catalog that would benefit from visually similar discover. If you would like to see large scale visual search in action, try downloading and testing our SiBi – See It Buy It apps from either the App Store or Google Play.
A key component in the purchase decision for items like fashion and furniture is color. Historically, merchants have provided very simplistic color grids, often representing one shade of 8 to 12 different colors, as a way for consumers to filter color selections in a product catalog. However, the human eye can detect millions of shades or various colors and simply filtering on “red” or “blue” is not optimal. The reason for this predominant methodology is that searching on any color in the spectrum is hard and requires complex algorithms to perform. You can find a little more about the challenges involved in THIS ARTICLE.
nFlate offers catalog color indexing to ecommerce merchants. When implemented, we evaluated all of your images, eliminate dominant background colors and then identify the prominent colors present in the objects in the image. And when combined with object detection (see below), consumers will be able to perform much more specific searches (for example “chartreuse handbag” or “mint heels”).
One of the top advances in visual computing over the last handful of years is object detection. Think about what Facebook has done with it, using it to recognize people’s faces in complex images. Similarly, object detection also can add immense value to the ecommerce discovery process when combined with visual search. Without object detection, searching for matches on an image of a person with sunglasses, a hat, carrying a handbag, wearing jeans and a leather jacket would return people wearing sunglasses, a hat, carrying a handbag, wearing jeans and a leather jacket. When object detection is added to the mix, consumers can search for just matching handbags or jeans (rather than people!).
Likewise with object detection, shoppers can search for “looks”. For example, using the above image a consumer could request top matches to the sunglasses, handbag and hat. Or to the jeans and jacket. The results could be displayed in a mix and match carousel of products. Prior to an AI based solution, looks were either created manually or made based on collaborative filtering strategies (“frequently bought together” checkout offerings).
nFlate offers object detection and bounding box creation as another AI-based tool to our partners.
An important component of the object detection process is image labeling (or “object” labeling). Once object are detected and bounding box positions are identified, it is important to label the images. As you can see from the example above, the items in the image with the bounding boxes (showing in the example, they are usually hidden) have labels associated. If query links are then associated to the label area, users can tap on the label and get back product matches to the target object. nFlate offers object labeling as part of the object detection integration.
The final step in the object detection process is image classification. Like with bounding boxes, image or object classification requires a combination of human effort to define target classes and AI to find and identify objects in these classes in new images based on the original training set. This is one of the more time consuming pieces of the visual computing assembly line. Classes have to be clearly defined up front and hundreds to thousands of representative images in each class have to be identified by human resources. Then the training can start. In fashion, classes can include tops, jackets, jeans, trousers, bags, hats, sunglasses, swimwear and hundreds of additional potential classes. The more classes, the more complex the model. Accurate classification and labeling is the core to a successful visual computing ecommerce offering.
Another useful side-benefit of the object detection process is image tagging. Having accurate tags in a product database offers ecommerce merchants deeper options when setting up their stores. Once all of a merchants products have been classified and indexed by the AI process, tags can be exported back into the product database. Merchants can then use new classes to offer better navigation choices and, when properly integrated into the site’s text or voice search capability, shopper’s will benefit by receiving better and more accurate query results.
The word Taxon means a group of organisms, expanded: a need-based classification scheme that organizes controlled vocabulary into an hierarchical structure. Simplified: tree structure of terms users expect that reflects user needs. UX taxonomists (ecommerce site owners) face 4 key challenges:
Fashion, as an example, might have over 600 classes to apply a taxonomy to. Or it could be a simple as 10-20. It completely depends on the objective of the site owner. The key is that a good taxonomy is required as the foundation for effective discovery, either by site navigation or text/voice search. As part of our search engine development services, nFlate offers taxonomy buildout to sites that wish to optimize their discovery process.