How to A/B test checkout upsells

‍‍Overview

A/B testing allows you to compare different upsell offers in checkout to determine which performs better. This helps optimize conversions and maximize revenue.

Step-by-Step Guide

To enable A/B testing, go to the UpsellPlus app. Create an offer, choose ‘Checkout’‍

Log in to Shopify Admin

Open the UpsellPlus App
Go to Apps in the lefthand menu
Click UpsellPlus from the list of installed apps

Create an Upsell Offer
Click Create Offer and configure the first upsell (Version A).
Choose the product and customize the offer details.

Enable A/B Testing
After adding your product and customizations, click Add an A/B Version
This will duplicate the offer as Version B



The app will copy your current offer and create a B version. You can toggle between the two versions with the tabs at the bottom. ‍

Here’s an example of a simple A/B test setup that will test the callout banner that introduces the upsell offers in checkout: ‘Frequently bought together’ vs ‘73% of customers also bought’‍‍



Once you publish the offer to your checkout, 50% of shoppers will see version A and 50% of shoppers will see version B. ‍You can also see the impressions, conversion and revenue for each version.

Note: The smart rules are not part of the A/B version testing as that would invalidate the 50/50% visibility for shoppers.‍

Some tips to run an effective A/B test:‍

1. Set a goal and hypothesis‍

To get a useful result out of an A/B test, define your hypothesis. In simple words, say what you want to test:‍

“Changing the discount from 20% to 30% of my socks checkout upsells will increase the conversion rate from 3% to 5%”‍
“The CTA button text ‘ADD TO CART’ will convert 1% higher than ‘add to cart’”‍
“A stack layout for my multiproduct checkout upsell will earn 20% more revenue than a carousel layout”‍

This will help you understand what the goal was of the test when you come back to it. You can add this hypothesis as the internal name of the upsell offer at the very top of the page.

2. Only test one variable at a time‍

Choose 1 variable to change between the two versions and let the test run until you hit at least 30 conversions.‍

Ideas:
Discount %: 20% vs 30%
Callout banner text
Design
Button CTA text
Button color
Upsell product
Multiproduct layout: stack vs carousel
Upsell description
Upsell marketing message

3. Get a statistically significant result‍

The UpsellPlus app will declare a winner after a statistically significant result has been reached for your store. It will indicate a winner has been found with the ‘trending’ badge next to the winning version.‍‍



This badge will also show in your offer overview to notify you that a winner has been determined.‍‍



Note that finding a statistically meaningful result might take a while if you’re having a low conversion rate for the upsells. Give the A/B test enough time to produce useful data.‍

4. Create a 'control' and a 'challenger'‍

When you had an upsell run without an A/B test, keep that unchanged (‘control’) and add a ‘challenger’ with one variable that’s updated to test a hypothesis.‍

Note: When you choose to run an A/B test, you can no longer switch the offer from a manually chosen product to an automagically (AI) chosen product. This would invalidate the A/B test as too many variables would change.‍

Key Tips
Test One Variable at a Time: This ensures clear results (e.g., product choice, discount, or copy).
Run the Test Long Enough: Aim for at a minimum number of conversions per version to get meaningful insights.

Related Articles
How to Enable Checkout Upsells Using the Checkout Editor
How to Show an Upsell Based on Cart Value

Updated on: 11/02/2025