# What is A/B testing? Canonical URL: https://support.optimonk.com/en/articles/what-is-a-b-testing A/B testing is a method of comparing two or more versions of a campaign to determine which one performs better. Instead of guessing what works, you let real visitor data decide. In OptiMonk, you run A/B tests on your popups to improve conversion rates, grow your email list faster, and increase revenue. --- ## How does it work? When you run an A/B test, the traffic of one campaign is split between its **variants**. Each variant is shown to a separate group of visitors, and OptiMonk tracks the key metrics to identify the winner. Traffic is shared **equally** across all active variants — two variants, half each. Once enough data is collected, the winning version takes over. --- ## A/B testing in OptiMonk: variants of one campaign OptiMonk runs one kind of A/B test: **variants inside a single campaign**. You create several versions of the same popup — a different headline, image, offer or CTA button — and OptiMonk splits traffic between them. You add a variant on the campaign's **Overview** tab, under the variants table, with **+ Add A/B test variant**. There are three ways to get one: - **Duplicate a variant** — *Clone an existing design to tweak.* The usual choice: an identical copy where you change one thing. - **Create with AI** — *Generate a new design in the designer.* A genuinely different design for the same campaign, not a copy. - **Control (no popup)** — *A holdout bucket that sees nothing.* **Best for:** testing changes within a single campaign — copy tweaks, design changes, different discount offers. ### The control (holdout) arm The **Control (no popup)** variant is worth its own paragraph. It is a slice of your traffic that sees **no** campaign at all, and it answers a different question from the others: not *which popup is better*, but *is this popup worth running at all*. Because the control is measured with exactly the same metrics as your designs, the uplift you read against it is the incremental effect of the campaign itself — no attribution modelling in between. One control per campaign. --- ## The two numbers to read While a test runs, the campaign's **Overview** tab shows a green **A/B test running** banner naming the leader, and the variants table gives you both numbers per row: - **Uplift** — how much better (or worse) a variant converts than the baseline. The baseline row itself shows a dash. - **Chance to win** — how confident the result is, shown as *{pct}% to win*. This is the number that tells you whether you are looking at a real difference or at noise. Before there is enough data, the banner says *Data collection in progress.* and the metrics band says *Not enough conversions yet to calculate uplift.* That is not a fault — it means "keep waiting". --- ## Ending a test: automatic winners You don't have to watch the numbers every day. Click the value in the **Traffic** column to open **A/B test settings**, where **Auto-declare winner** is on by default: *Automatically end the test when a clear winner emerges.* It fires only when all three conditions are met at once: | Setting | Default | | --- | --- | | Declare at … chance to win | 95% | | Min conversions | 200 | | Min days running | 3 | All three are editable, and they belong to that one campaign — changing them here does not touch your other campaigns. Together they are a guard against the most common A/B mistake: calling a winner from a handful of conversions on day one. You can still end a test by hand with **Declare winner at 95%** in the banner. If the leader is below your threshold, OptiMonk warns you first: *Ending the test now may pick a winner that isn't statistically reliable yet.* — then it's **Keep testing** or **Declare anyway**. --- ## Not sure what to test? Ask. In the chat panel beside a campaign you can ask **"Suggest an A/B test I could run"**. OptiMonk first asks what kind of test you have in mind: - *Test the copy only (headline, CTA…)* - *Change the offer / incentive* - *Try a whole new design variant (takes longer)* - *Surprise me — mixed ideas* Then it proposes concrete ideas with previews. Each one opens a **Variant comparison** — your current version next to the suggestion, with **Why this works** and **What changes** — and **Add variant** turns the idea into a real arm of the test. --- ## Why should you A/B test? - **Make data-driven decisions** — stop guessing and let your visitors tell you what works. - **Improve conversion rates** — even small improvements have a big impact over time. - **Reduce risk** — test changes on a portion of your traffic before rolling them out to everyone. - **Learn about your audience** — every test teaches you something about what resonates with your visitors. --- ## Tips for effective A/B testing - **Test one thing at a time.** If you change too many elements at once, you won't know which change made the difference. - **Let your test run long enough.** Don't call a winner too early — the auto-declare defaults (95%, 200 conversions, 3 days) are a good yardstick even when you decide by hand. - **Start with high-impact elements.** Headlines, offers and CTAs typically have the biggest influence on conversions. - **Two arms beat five.** With three or more variants a reliable result takes longer to build up than with two. - **Add a control when the question is "is this worth it".** Comparing two headlines tells you which headline wins; only a holdout tells you whether the popup earns its place.