> ## Knowledge Base Index
> Fetch the complete knowledge base index at: https://help.upsellplus.com/sitemap.xml
> Use this file to discover available pages before exploring further.
> Pure-Markdown content can be obtained by appending a '.md' suffix to the content URLs listed in the sitemap (without the trailing slash).

# How to run an A/B test on an upsell offer

**An A/B test shows different versions of the same offer to different shoppers and tells you which one earns more.** Pick an offer, choose what to change (for example the button text, layout or discount), split the traffic, and UpsellPlus tracks each version until there's a clear winner. Add more variants to run a multivariate test.

- **Where:** **Experiments** → **Create Experiment**
- **Up to 5 variants**, including the original (control)
- **To measure whether offers add revenue at all:** use a holdout experiment instead


![A/B test setup testing the call to action, with results showing the winning variant](https://storage.crisp.chat/users/helpdesk/website/-/a/b/f/a/abfa0cd47f75e80/ab-test-variants-and-results_167yrqv.png)

## What can I test?

For each variant, select the fields to change:

- **Banner Text** and **Banner Subtext**
- **Call to Action**
- **Layout**
- **CTA Styles** and **Card Styles**
- **Learn More Link**
- **Discount**
- **Products**

## Create an A/B test

### Step 1: Setup
1. Go to **Experiments** and click **Create Experiment**.
2. Enter an **Experiment Name** and a **Hypothesis**, for example *Changing the CTA to "Buy Now" will increase conversion rate by 15%*.
3. Choose the **Base Offer**.
4. Choose a **Primary Metric**: **Conversion Rate**, **Revenue Per User**, **Add to Cart Rate** or **Average Order Value**. Optionally add **Secondary Metrics** to monitor.

### Step 2: Variants
1. **Control (Original)** is the offer as it is.
2. For each variant, give it a name, select the fields to test and enter the new values. The preview shows each version on desktop and mobile.
3. Click **+ Add Variant** for a multivariate test (up to 5 variants in total).

### Step 3: Traffic & Duration
1. Choose **Equal Split** or **Custom** traffic allocation. Each variant needs at least 5% of traffic, and the total must be 100%.
2. Check the estimate: enter your **Baseline Conversion Rate**, **Minimum Detectable Effect** and **Estimated Daily Traffic** to see the **Required Sample Size** and **Estimated Duration**.
3. Choose **Start immediately** or **Schedule for later**, and optionally **Auto-stop after** a number of days.
4. Click **Create & Start** (or **Create as Draft**).

## Auto-end and guardrails

Turn on **Auto-end** to stop the test when the result is statistically significant:

- **Confidence threshold:** 90%, 95% or 99%
- **Minimum runtime:** the test won't end before this many days

The winning variant is declared, but not applied automatically: you review the results and apply it. Guardrail metrics set to **Stop test** always apply, even with auto-end off.

## Read and conclude the test

A test ends with one of these outcomes: **Winner found**, **No clear winner / Inconclusive**, or **Manual stop**. Tests can also end because a guardrail was violated, there was no meaningful difference after twice the required sample size, or the scheduled end date was reached.

While a test is running, you can't change the offer's placements.

## Tips

- Test **one big idea at a time** (for example discount vs no discount) before small styling changes.
- **More variants need more traffic.** If the estimate is over 60 days, test fewer variants or a bigger change.
- Home suggests offers that are ready for a test (verdict **Test**).

## FAQ

**What's the difference between an A/B test and a holdout?**
An A/B test compares versions of an offer. A holdout compares showing offers with not showing them. See [How to measure the real revenue impact of your upsells with holdout experiments](https://help.upsellplus.com/en/article/how-to-measure-upsell-revenue-impact-with-holdout-experiments-t3on4r/).

**Can I test showing an offer vs not showing it?**
Yes, with a holdout experiment on **Selected offers**.

**Which plan includes experiments?**
A/B experiments and holdouts are included on the Pro plan.

## Related articles

- [How to measure the real revenue impact of your upsells with holdout experiments](https://help.upsellplus.com/en/article/how-to-measure-upsell-revenue-impact-with-holdout-experiments-t3on4r/)
- [Understanding the UpsellPlus Home page](https://help.upsellplus.com/en/article/understanding-the-upsellplus-home-page-w4owa/)
- [Understanding UpsellPlus analytics](https://help.upsellplus.com/en/article/understanding-upsellplus-analytics-metrics-and-reports-1bc53n7/)