Conventional product search engines barely require a detailed introduction. Basic site-search is available as a standard plugin for, or built into, all modern e-commerce platforms. Most of them are relatively simple and easy to use, featuring basic text-matching. They typically support indexing of product titles, descriptions and category structure, auto-correct, fuzzy-match (handle plural queries and stemming) and recognise synonyms (often through a configurable dictionary).
The classic search problem involves things such as language processing (tokenization, stemming etc), indexing, query expansion, product ranking and so on. Advanced search engines can focus on word similarities and word clustering.
The most basic search algorithms available do something like the following:
Break query strings into words
Compose the search query by targeting known data fields like title, description, features, word by word. The where statement will look like ‘where (description like ‘%red%’ or features like ‘%red%’ or title like ‘%red%’) and (description like ‘%bag’ … etc etc)‘
Display results in rank order depending on where the query term appeared in the product metadata (e.g. a match found in the product title has more weight than a match found in the description).
Unlike recommender systems that use behaviour data to draw conclusions about a single product or small set of products, search systems must find and rank all products against a text string (i.e. the search query), not against a product or set of products.
So although the difference between the query string 'blue hat ' and an actual Blue Hat product seem small at first, they are completely different input.
Unfortunately, more often then not, the only reason the search returned irrelevant or null results is because the visitor made a small typo in their query or they used a different taxonomy than what the retailer uses.
Related results and the future of on-site product search
Even with a moderate product range, countless hours can be spent on tweaking and redirecting search results, or adding related products on the individual product level. It often becomes time-consuming up to a point where the functionality is partly or entirely neglected, which in turn leads to lower conversion rates and a sub-par user experience.
Fortunately, opting for a quality third-party provider can mitigate all of these issues.
Product search engines typically focus on things like:
- Efficient indexing: used to quickly locate data without having to search every row in a database table every time a database table is accessed
- Attribute ranking rules: rank results by their expected relevance to a user's query using a combination of query-dependent and query-independent methods
- Fuzzy query matching: approximate string matches (e.g. “rmabo” vs “rambo”)
- Stemming: getting words to match each other even if they are not in the exact same form (e.g. run/runs/running/ran or cat/cats)
- Tokenization: the process of breaking a stream of text up into words, phrases, symbols, or other meaningful elements called tokens
In advent of machine learning, search algorithms have been able to make great strides when it comes to using behaviour data. Allowing click, add-to-cart and purchase behaviour to influence the sorting of results (i.e. no more sorting based only on where the text-match was found in the product metadata).
Machine learning has brought forward the ability for search engines to understand the relationships between products - before any query has been made or any behaviour data has been gathered. In this way, the search engine can locate similar or related products beyond a basic shared category or brand. It can identify complex patterns within all the metadata and build a deep architecture to the product catalogue.
From this foundation, a machine learning search engine can produce a list of related products that have no relation to the actual search query.
Providing a list of 'related products' that are similar to the product being looked at or searched for can - and will - lead to increased conversion rates and basket size.
This is how Loop54 works...
Machine learning product search
Machine Learning enables a search engine to map relationships between products that never existed before. In doing so, it can produce an intelligent, self-optimising list of 'related results' that are completely unlike those provided by recommender systems (aka. product recommendations).
Moreover, this type of engine requires basically zero behaviour data to function - nearly eliminating the cold start problem (note: behaviour data should be added to the engine to help refine the sorting of products for greater relevancy).
Loop54 does precisely this. Our related search results (named as such to avoid confusing our service with product recommendations), are a list of products that have nothing to do with actual search query. They are instead related to the products found through search.