After activating a product recommendation engine, the last step is using an A/B test to measure its efficiency. While this step is not mandatory, it is highly recommended.
Expected duration: 30 minutes ⏳
Technical complexity: medium
One option is to compare the results of your shop (in terms of conversion rate and AOV) with the results in the previous period or the same period of the past year. However, this will be subject to biases, as it assumes that the externalities remain the same, which almost never happens (think of the COVID pandemic as an example). Moreover, this would only give you the big-picture metrics without giving you any indication of what part of your recommendations work better (e.g. similar items, cross-sell items - see how to set them up here).
An A/B test presents two (or more) versions of your website to random visitors during a set period of time and can be configured to evaluate more granular metrics (e.g., the number of clicks on the recommendations widget). For this reason, we recommend an A/B test to find out if your website’s number and value of transactions increase, and what exactly drives those results. The simplest method, which does not require advanced programming and can be implemented by digital specialists and marketing professionals, is using the Google Optimize platform.
The three needed tools are Google Analytics, Google Tag Manager, and Google Optimize. They are all free and have great support documentation.
<aside> 💡 You will need admin access to your website, as snippets of code need to be added for all three tools!
</aside>
Here are some valuable resources that take you through the process for each tool installation:
When installing the recommendation widgets, make sure you are NOT REPLACING anything in your previous website, and only ADDING the new widgets to what you already had. This might mean that there are now two recommendation widgets where there was only one before, but this will all make sense in a minute.
Google Optimize works by HIDING certain parts of your webpage to a fraction of the visitors. When setting up the experiment, you will hide the new recommendations for group A (we will call them the control group), and hide the old recommendations for group B (we will call them the test group).
<aside> 💡 Please note that once the A/B test is completed, you should only keep one recommendations widget – the best performing one.
</aside>