The Complete Guide to AB Testing

We'll introduce the concepts and basic core principles you need to know before starting A/B testing, and share examples of excellent companies that are actually using A/B testing.

• What is A/B Testing?
• A/B Testing Corporate Case Studies
• A/B Testing Basic Core Principles

We've collected only the essential statistical concepts needed for A/B testing.

• Sample and Population, Sampling Method, P-Value, Confidence Interval
• Chi-Square Test, Hypothesis Test, Type 1 & 2 Error

We'll explain in detail how to write an experiment plan for actually running an A/B test.

• How to Write an A/B Test Experiment Plan
• Determining group size (minimum sample size)

To demonstrate a mock A/B Test case, we'll introduce a randomly created example app.

• Introducing A/B Test Example Domains

We will introduce two mock A/B test cases and explain everything from writing the experiment plan to deriving analysis results.

• Create a 1-Pager
• Interpretation of results

I'll explain the things you must keep in mind when conducting A/B testing experiments.

• Peeping problem
• Confounding Factor
• Primacy effect and novelty effect
• Follow-up monitoring of completed A/B tests