How do Product Recommendation Engines Work?

You make trips to your local shop weekly, and after some time, the store clerk gets to know you, what you tend to buy, and what you like. Looking to be helpful (and make a buck), one day they pick out a suggestion they believe will be “right up your street”. And it is. 

By using the picture they have of you and making an accurate, intuitive leap, the clerk has achieved what all product recommendation engines set out to do.

And, along with dynamic pricing, loyalty programmes, email marketing, and much more, product recommendations make up part of a wider e-commerce strategy: personalisation.

While it’s been a key driving force within B2C e-commerce for some time, we're now seeing the B2B segment start to follow B2C's lead.

But how do product recommendation engines operate in this quest for personalisation?

How do Traditional Product Recommendations Engines Work?

A product recommendation engine, also known as a recommender system, is a type of information filtering system that attempts to predict the preference or rating a user would give to an item. There are four main approaches:

1.      Collaborative Filtering

Collaborative Filtering is a technique used to create user “profiles” for personalised experiences. By looking at a user’s data (usually ratings or purchase data), the user can be matched with similar users, and thus new knowledge can be gained about products that the user has not yet interacted with.

Collaborative Filtering is used in e-commerce, however it is not used in e-commerce site-search. Instead it is used in other areas such as email marketing and personalised product recommendations.

Since most e-commerce shoppers do not have a long history of interactions with the site, it is impossible to use Collaborative Filtering in situations where the intent is narrow or virtually imperceptible, such as site-search or in-session personalisation. The basic steps are:

1. A website visitor is matched with other customers based on their previous purchases of the same or similar items.
2. These items are grouped together.
3. Items the user has bought previously are disregarded.
4. The remaining items are then recommended to the visitor.

2.      Content-Based

This technique is built upon a keyword-based description of the item and a profile of the sorts of items the visitor likes (their preferences). Algorithms look at the items a user has liked in the past and then recommend similar items.

3.      Demographic

Unlike collaborative and content-based filtering methods, a demographic based recommender system doesn’t need a history of user ratings to operate. Instead, visitors are grouped based on attributes, and recommendations are tailored to the various demographic profiles.

4.      Hybrid

As a combination of any of the above methods, hybrid recommender systems aim to provide the best of both worlds by exploiting the strengths of each technique and minimising the weaknesses.

Netflix uses a hybrid approach, basing suggestions upon a user's ratings for previous content (content-based filtering), as well as comparing the habits of similar users and then forming recommendations (collaborative filtering).

The unifying component of all four approaches is their connection with user behaviour. While this is a cornerstone of the recommender system, for search engines it’s an added dimension. If you'd like to learn more, we've explained further the differences between product search engines and product recommendation engines in an earlier post.

A key issue with traditional recommender systems is the cold start problem – the fact that the initial recommendations are relatively poor, as the system doesn't have enough data at both the aggregate and individual level to make recommendations.

The problem is exacerbated by the fact that when catalogues change, most of that big data is lost. This often results in manual workarounds and over-reliance on simplistic business rules to decide what products should be recommended to the user – for example, sale items, new-in items, items from another category, highest margin items, etc.

A Modern Approach to Product Recommendations

Machine learning search engines are revolutionising the world of product recommendations. While previously search results were only sorted according to where and how frequently a text-match was found in the product metadata, search algorithms can now use add-to-cart, click, and purchase behaviour in deciding which products to return as results and in what order they should be presented.

By using past behaviour to determine which products best match the intended meaning of the query, recommendations become highly personalised and relevant to the visitor.

These search driven recommendations are exclusive to Loop54 and were released after years of R&D. What makes them unique is that they require much less behavioural data to learn shopper intent than traditional recommender systems. That's because Loop54 maps the relationship between products, which allows each user behaviour data point to be connected to a network (or context) of products, not just a single stock keeping unit (SKU). 

The relevance of search results and product recommendations should depend on a visitor’s intent, not the accuracy of their words. Therefore, it shouldn't matter whether a visitor uses the words cookies or biscuits to describe what they are looking for – the products displayed to them in search and recommendations should be sweet baked treats regardless. It's by understanding how products relate to one another that a web page can effectively show visitors the full range of relevant products that match their buying intent. This in turn substantially increases conversion rates and basket size.

Machine Learning has brought forward the ability for search engines to understand the relationships between products – before any query has been made or any behaviour data has been gathered. In this way, the search engine can locate similar or related products beyond a basic shared category or brand. It can identify complex patterns within all the metadata and build a deep architecture to the product catalogue.

 
From this foundation, a Machine Learning search engine can produce a list of recommended products that have no relation to the actual search query. Although these recommended products are generated differently than those generated by classic recommender systems, they can have the same affect on the customer experience.

 
Providing a list of 'recommended products' that are similar to the product being looked at or searched for can – and will – lead to increased conversion rates and basket size.

Relevant, personalised search and navigation increases revenue via very accurately targeted experiences; showing the right products, at the right time, and in the right order, to each individual visitor based on what we know about them (persona) and their real-time shopping intent.

Frictionless Shopping Experiences 

Search and recommendation engines should always help you optimise your search and navigation experience. They should provide the most relevant search results, category listings, and product recommendations, but you should also be able to decide how those results and lists are presented to the user and how it fits together with the rest of the website experience.

In order to lend a hand, we produced a guide of best practices for website design, in the six key areas: homepage, category taxonomy, main navigation, cross-navigation and cross-selling, product list layout, and site-wide layout.

The guide, formed from our own and other's research, demystifies the website design practice and provides useful, accessible, and up-to-date tips on the process.

If you'd like to learn more and see for yourself, download a free copy today.