Test two or more versions of an Experience side by side to find out which performs better. Use the Audience Randomizer to split users into groups, then compare Goal conversion or completion rates across variations.

Prerequisites

• Two or more Experiences (your variations)
• A target audience large enough for meaningful results — aim for at least 500 users per group
• A Goal attached to each Experience (optional but recommended)

Set up the test

1. Create your Experience variations

1. Build each variation as a separate Experience. To save time, create the first Experience, then clone it from the Experience list and edit the clone in the Builder.
2. Set the frequency of each Experience to Show once so each user only sees one version.

2. Split your audience

You split traffic between variations using the Audience Randomizer — a number from 1–100 that Appcues assigns to every user at random on their first login.

How the split works

The Audience Randomizer is spread evenly across 1–100, so the width of the range you target is roughly the percentage of users who'll qualify. You don't enter a percentage directly — you choose a number range, and the size of that range decides the split:

• 1–50 → ~50% of users
• 1–25 → ~25% of users
• 1–33 / 34–66 / 67–100 → ~three even groups

Note: Because assignment is random, your groups won't be exactly even — especially with smaller audiences. The larger your audience, the closer the split gets to your target percentages. Aiming for at least 500 users per group (see Prerequisites) keeps the difference small.

Create the Segments

1. Go to Segments and click Create a Segment.
2. Add a condition using the Audience Randomizer property. For Group A, set Audience Randomizer is less than 51 — this captures users 1–50 (~50% of your audience).

3. Clone the Segment and update the clone for Group B: set Audience Randomizer is greater than 50 — this captures users 51–100 (the other ~50%).

4. Keep all other audience criteria identical across both Segments.

Tip: The Audience Randomizer is a static property — a user keeps the same number once assigned. If you plan to run multiple A/B tests over time, shift the ranges so different users become eligible each time. For example, on your next test, target Group A to greater than 25 and less than 75, and Group B to the remaining users.

3. Apply Segments and Goals

1. Open each Experience's Settings and assign the matching segment.
2. Confirm all other targeting criteria are identical across Experiences — page targeting, frequency, and priority.
3. Add the same Goal to each Experience. For Goal setup details, see Set up Goals.

Test more than two variations

Create one Segment per variation and divide the Audience Randomizer range evenly. For an A/B/C/D test with four Flows, use ranges like 1–25, 26–50, 51–75, and 76–100.

Measure results

Open each Experience's Analytics page and compare Goal conversion rates (or completion rates if you aren't using Goals). Use a significance calculator to confirm your results are statistically meaningful before making changes.