← All analytics projects
03
Project 03 · Experimentation
A/B Test Decision Calculator
Decide whether an observed conversion lift is statistically reliable, commercially useful, and supported by an adequately planned sample.
ExperimentationConfidence intervalsPowerPractical significance
Control rate11.52%
Treatment rate12.80%
Relative lift+11.1%
Two-sided p-value2.01e-18z = 8.757
Experiment workspace
Test the observed result
Uses a two-sided pooled two-proportion z-test and an unpooled confidence interval for treatment minus control.
Cumulative experiment counts
Edit all four values; results update immediately.
Effect and uncertainty
Absolute lift+1.28%
95% CI low+0.99%
Incremental conversions1,280
95% confidence interval: [+0.99%, +1.57%]
Observed conversion rates
Pre-registration
Plan the next sample before seeing the result
Equal-sized groups, two-sided α = 0.05, normal approximation to the two-proportion test.
Sample-size assumptions
Method
What the calculator assumes
Core assumptions
- Independent randomized users
- Concurrent experiment variants
- Pre-specified conversion event and window
- Adequate expected successes and failures
- No unplanned repeated stopping
Controlled decision brief
This Vercel edition generates no live LLM prose. The decision label is a deterministic rule applied to the displayed statistics and your practical threshold.
Confirm guardrail metrics, randomization quality, and experiment duration before rollout.