High-quality A/B testing infrastructure creates unreasonable amounts of value.

· Bits and Bobs 10/2/23

Experimentation is one of the most reliable ways to create value in an automatically-cohering way, by climbing hills.

But experimentation requires a lot of overhead to administer and extract insights from.

The cost for a team to create the right tools to sense what's happening in a given experiment is way larger than the amount of value any given experiment might create.

That means A/B testing is a perfect fit for making infrastructure: pay the cost once, and every user now benefits.

On Chrome, one of the most game-changing pieces of internal infrastructure was Finch, the system to administer and measure A/B tests in the wild. It was trajectory changing for us.

When you have mature A/B testing tools and infrastructure, the more spaghetti you throw at the wall, the faster you find valuable ideas. This can allow a self-hoisting feedback loop.

More on this topic

From other episodes