Stat-Talk #3

Stat-Talk #3

3 Ways to Tune Sample Sizes

3 Ways to Tune Sample Sizes

3 Ways to Tune Sample Sizes

Set Accuracy and Visitors

Set Accuracy and Visitors

Set Accuracy and Visitors

Select the MDE. Cap the False Positives. Pick Low-Variance Metrics.

Select the MDE. Cap the False Positives. Pick Low-Variance Metrics.

45 mins + 15 mins QnA

45 mins + 15 mins QnA

Topics

checkTips to set the right MDE for your test
checkPicking low-variance metrics that save visitors.
checkSetting the accuracy by selecting the right alpha (false positive rate).
checkUnderstand one sided and two-sided tests.

Audience

This talk is for you if:

checkYou do not have enough traffic to reach statistical significance.
checkMany of your winners revert back to mean after they have deployed.
checkYou have been hearing about MDE and are unsure how to set it.

After this talk, you will have a practical understanding of setting the sample size for your tests.

Meet Your Mentor

Meet Your Mentor

Ishan Goel

Ishan Goel

Built the Stats Engine at VWO

Built the Stats Engine at VWO

Founder & Educator

Former Head of Data Science

Author

Trained 60 Leaders

Founder & Educator

Former Head of Data Science

Author

Trained 60 Leaders

Hi! I am a statistician, a teacher and the founder of Thinking Bell.

I am known for simplifying experimentation with analogies, doodles, and games that make decisions sharper and science practical.

I have led data science research at VWO and I have authored the VWO Stats Blog. In 2025, I taught a statistics bootcamp for a team of 60 marketers, product managers and data scientists.

Join me to master experimentation with clarity, depth, and fun.

Stat-Talk #3

3 Ways to Tune Sample Sizes

Set Accuracy and Visitors

Watch Previous Stat-Talks

Our most liked sessions available right away!

Dealing with High Variance Metrics
Dealing with High Variance Metrics
Boosting the Power of Your Tests
Simplifying Stats of A/B Testing
Featured
Simplifying Stats of A/B Testing
Making Sense of Data
3 Ways to Tune Your Sample Sizes
Featured
3 Ways to Tune Your Sample Sizes
Set Accuracy and Visitors
Dealing with High Variance Metrics
Dealing with High Variance Metrics
Boosting the Power of Your Tests
Simplifying Stats of A/B Testing
Featured
Simplifying Stats of A/B Testing
Making Sense of Data
3 Ways to Tune Your Sample Sizes
Featured
3 Ways to Tune Your Sample Sizes
Set Accuracy and Visitors

Explore recorded StatTalks on Thinking Bell's Youtube Channel

Curious to Dive Deeper?

See Our Entire Course Curriculum

Join the 30-student cohort and master A/B testing stats through live classes, games, and community.