A/B Test Design and Reading
Designs a test that can actually detect the effect you care about, and states the decision rule before the data arrives.
Free — no card needed
Create a free account
to open A/B Test Design and Reading — and the other 364 prompts across 21 categories.
Already have an account?
Running it, start to finish
- Name one primary metric and the smallest effect worth acting on.
- Run the power calculation before launching.
- Check sample ratio before reading the result.
What you get back
The output this produces, every time.
- Computes whether the test can detect the effect you care about, and says so if it cannot.
- Makes inconclusive an available outcome, without which a test always produces a decision.
- Puts sample-ratio mismatch first, since a broken randomisation invalidates everything after it.
Getting better results
Where this usually goes wrong, and how to avoid it.
- Believe the power calculation. If it says fourteen weeks and you have three, the test will not answer the question. Change the metric, the effect size, or do something else.
- Write the decision rule down first. Its whole value is in existing before the data. A rule written afterwards is a rationalisation.
- Check sample ratio before anything else. An imbalance means randomisation is broken, and every downstream number is meaningless however good it looks.
More Data Analysis prompts
All Data Analysis →- Data Analysis
Segmentation Analysis
Produces segments that are actionable and stable, since statistically clean but unusable is the standard failure.
ClaudeChatGPT002Open → - Data Analysis
Chart That Makes the Point
Chooses the chart from the claim you are making rather than from the data's shape, and names the misreadings to design against.
ClaudeChatGPTGemini000Open → - Data Analysis
Metric Definition Review
Treats two teams reporting one metric differently as a governance problem rather than a query bug.
ClaudeChatGPT000Open →
Written for The AI University. Every prompt in this library is original work — authored, tested and revised here, not collected from elsewhere. 365 of them, free with an account.