A/B testing is a method of comparing two or more versions of a campaign to determine which one performs better. Instead of guessing what works, you let real visitor data decide.
In OptiMonk, you run A/B tests on your popups to improve conversion rates, grow your email list faster, and increase revenue.
How does it work?
When you run an A/B test, the traffic of one campaign is split between its variants. Each variant is shown to a separate group of visitors, and OptiMonk tracks the key metrics to identify the winner.
Traffic is shared equally across all active variants — two variants, half each. Once enough data is collected, the winning version takes over.
A/B testing in OptiMonk: variants of one campaign
OptiMonk runs one kind of A/B test: variants inside a single campaign. You create several versions of the same popup — a different headline, image, offer or CTA button — and OptiMonk splits traffic between them.
You add a variant on the campaign's Overview tab, under the variants table, with + Add A/B test variant. There are three ways to get one:
Duplicate a variant — Clone an existing design to tweak. The usual choice: an identical copy where you change one thing.
Create with AI — Generate a new design in the designer. A genuinely different design for the same campaign, not a copy.
Control (no popup) — A holdout bucket that sees nothing.
Best for: testing changes within a single campaign — copy tweaks, design changes, different discount offers.
The control (holdout) arm
The Control (no popup) variant is worth its own paragraph. It is a slice of your traffic that sees no campaign at all, and it answers a different question from the others: not which popup is better, but is this popup worth running at all.
Because the control is measured with exactly the same metrics as your designs, the uplift you read against it is the incremental effect of the campaign itself — no attribution modelling in between. One control per campaign.
The two numbers to read
While a test runs, the campaign's Overview tab shows a green A/B test running banner naming the leader, and the variants table gives you both numbers per row:
Uplift — how much better (or worse) a variant converts than the baseline. The baseline row itself shows a dash.
Chance to win — how confident the result is, shown as {pct}% to win. This is the number that tells you whether you are looking at a real difference or at noise.
Before there is enough data, the banner says Data collection in progress. and the metrics band says Not enough conversions yet to calculate uplift. That is not a fault — it means "keep waiting".
Ending a test: automatic winners
You don't have to watch the numbers every day. Click the value in the Traffic column to open A/B test settings, where Auto-declare winner is on by default: Automatically end the test when a clear winner emerges.
It fires only when all three conditions are met at once:
| Setting | Default |
|---|---|
| Declare at … chance to win | 95% |
| Min conversions | 200 |
| Min days running | 3 |
All three are editable, and they belong to that one campaign — changing them here does not touch your other campaigns. Together they are a guard against the most common A/B mistake: calling a winner from a handful of conversions on day one.
You can still end a test by hand with Declare winner at 95% in the banner. If the leader is below your threshold, OptiMonk warns you first: Ending the test now may pick a winner that isn't statistically reliable yet. — then it's Keep testing or Declare anyway.
Not sure what to test? Ask.
In the chat panel beside a campaign you can ask "Suggest an A/B test I could run". OptiMonk first asks what kind of test you have in mind:
Test the copy only (headline, CTA…)
Change the offer / incentive
Try a whole new design variant (takes longer)
Surprise me — mixed ideas
Then it proposes concrete ideas with previews. Each one opens a Variant comparison — your current version next to the suggestion, with Why this works and What changes — and Add variant turns the idea into a real arm of the test.
Why should you A/B test?
Make data-driven decisions — stop guessing and let your visitors tell you what works.
Improve conversion rates — even small improvements have a big impact over time.
Reduce risk — test changes on a portion of your traffic before rolling them out to everyone.
Learn about your audience — every test teaches you something about what resonates with your visitors.
Tips for effective A/B testing
Test one thing at a time. If you change too many elements at once, you won't know which change made the difference.
Let your test run long enough. Don't call a winner too early — the auto-declare defaults (95%, 200 conversions, 3 days) are a good yardstick even when you decide by hand.
Start with high-impact elements. Headlines, offers and CTAs typically have the biggest influence on conversions.
Two arms beat five. With three or more variants a reliable result takes longer to build up than with two.
Add a control when the question is "is this worth it". Comparing two headlines tells you which headline wins; only a holdout tells you whether the popup earns its place.