A/B Test Planner

What it is

This skill helps product managers and growth teams plan rigorous A/B tests for features, pricing, UI changes, and onboarding flows. It calculates sample sizes, estimates test duration, defines success metrics, and provides a decision framework for interpreting results. You get a complete test plan that prevents common pitfalls like peeking early or shipping inconclusive results.

What it does

• Creates structured test plans with hypothesis, variants, metrics, and decision criteria
• Calculates required sample size and estimates how long your test needs to run
• Defines primary success metrics and guardrail metrics to protect key business outcomes
• Provides a results interpretation guide so you know when to ship, iterate, or stop
• Flags risks like low traffic, multiple comparisons, and premature result-checking

Use it when

• You're planning to test a new feature, pricing change, or UI redesign
• You need to know how many users and how long a test requires
• You want a clear decision framework before launching an experiment
• You're unsure whether your test setup will produce trustworthy results