Experiments (A/B Testing)

Experiments let you run A/B or A/B/n tests and compare how different variants affect a goal. Use them to test product changes, pricing pages, onboarding flows, copy, layouts, or any other change where you want to measure the effect before rolling it out.

Overview

With Swetrix Experiments, you can:

• Create experiments with 2 to 20 variants, one of which must be the control.
• Store a hypothesis so the decision is tied to what you expected to happen.
• Use rollout percentages to control how traffic is split across variants.
• Link the experiment to a goal so conversions can be attributed to exposed visitors.
• Back the experiment with a new feature flag or link it to an existing flag.
• Choose whether exposure is recorded from feature flag evaluation or from a custom event.
• Review launch guardrails before starting an experiment.
• Estimate required sample size from baseline conversion, minimum detectable effect (MDE), and daily exposures.
• Analyze segmented, time-bucketed results with Bayesian probability of winning.

Experiment lifecycle

Creating an experiment

Open the Experiments tab in your project dashboard and click Create Experiment.

Launch guardrails

Launch guardrails call out configuration problems before the experiment starts.

Sample size planning

The sample size estimate helps you plan whether the experiment has enough traffic to answer the question.

Feature flag source

Exposure tracking

Exposure means that a visitor has entered the experiment and is eligible to be counted in the results.

import { track } from "swetrix";

await track({
  ev: "checkout_experiment_viewed",
  profileId: "user-123",
});

Multi-variant exposure

If the same visitor is exposed to more than one variant, Swetrix can handle that in two ways:

Client-side implementation

To use experiments in your application, use the Swetrix tracking script.

import { getExperiments } from "swetrix";

const experiments = await getExperiments({
  profileId: "user-123",
});

// Returns: { 'experiment-id-1': 'variant-a', 'experiment-id-2': 'control' }

import { getExperiment } from "swetrix";

const variant = await getExperiment("checkout-redesign-experiment-id", { profileId: "user-123" });

if (variant === "new-checkout") {
  // Show new checkout flow
} else {
  // Show control (original) checkout
}

You can pass an optional third argument when you want a fallback variant if the experiment is not running or cannot be found:

const variant = await getExperiment(
  "checkout-redesign-experiment-id",
  { profileId: "user-123" },
  "control",
);

Experiment and feature flag evaluations share a 5-minute cache. You can clear it before the next evaluation if you need a fresh assignment:

import { clearExperimentsCache } from "swetrix";

clearExperimentsCache();

Attribution

Swetrix attributes conversions when the same profile:

• Is exposed to an experiment variant.
• Triggers the selected goal after that exposure.
• Falls inside the effective result window.

For pageview goals, normal pageview tracking is enough. For custom event goals, track the goal event after exposure using the same profile.

Results

Click an experiment row or the results button to open the results view.

Completing an experiment

Help us improve Swetrix

Was this page helpful to you?

YesNo

Learn how to contribute•Report a problem with this content