If savvy users can fix a product failure case, that gives the potential for a self-bootstrapping quality system.

· Bits and Bobs 4/8/24

Your product must have quality good enough most of the time for most of your users.

Some products, when they don't work, don't work at all: a slammed door.

But some have the characteristic where savvy users can easily tweak or fix the broken result to make it work.

If you can aggregate these tweaks, you get the ability to automatically improve the product's quality by the wisdom of the crowds.

Here's a concrete example from a search context.

Imagine a user searches for [foo], a new kind of thing that he wants to see images of.

No images show up in the search results; perhaps the search engine hasn't yet noticed that foos are very image-y.

The user fixes the issue with a new query: [images of foo].

This is an unambiguous signal to the search engine of what the user wants, and it can confidently show images.

This was one savvy, motivated user. But we can aggregate small amounts of savvy user fixes into a global quality improvement.

When deciding to show images in the search result, a simple (stylized!) decision procedure, given the query [foo]:

Check how many times in the past 90 days the queries [foo] and [images of foo] were issued.

Divide the latter by the former, and if it's above some tuned threshold, trigger the images.

This simple procedure is self-healing; the system will automatically notice new image-y concepts just from the actions of a small number of savvy users fixing their results manually.

This basic pattern is captured in an old essay of mine, [ITW] Self-hoisting Feedback Loops

A viral ecosystem with these kinds of loops can get huge amounts of momentum.

More on this topic

From other episodes