a/b testing

Communicating A/B Test Results for Conversion Rates with Ratios and Uncertainty Intervals

Aug 20, 2022 · 8 min read · a/b testing data science statistics ·

A/B testing is a tool for supporting decision-making in business, and so in addition to getting the statistics right, it’s really important to communicate well with the non-statisticians who will have the final say on the go/no-go decision. Most A/B tests in practice are testing ratios, conversion rates of various …

A/B Testing and Product Ratings, Part 2: Multiple Ratings

May 23, 2022 · 3 min read · data science analytics business statistics a/b testing ·

Share on:

Image not found

Web path: https://www.harlan.harris.name/images/path/thumbnail.png

Disk path: static/images/path/thumbnail.png

Using Page Bundles: false

Second, your A/B test split users, but you’re measuring ratings, which are per-item. Some customers will buy multiple pairs of shoes, and most likely will either rate all of them or none of them. Does this fact affect how confident you can be at the end of the experiment? Randomization Unit Mis-Match A/B tests are …

A/B Testing and Product Ratings, Part 1: Delays and Bias

May 22, 2022 · 7 min read · data science analytics business statistics a/b testing ·

Share on:

Suppose you’re a data scientist at an e-commerce web site that sells shoes, responsible for supporting A/B tests. Many A/B tests are easy, and there are a number of companies that sell tools that make the easy cases as simple as clicking a few buttons and looking at pretty graphs. But A/B tests can get statistically …