A pattern to automatically improve quality: make educated guesses of what a user might want to do based on the aggregate behavior of similar users.

· Bits and Bobs 5/13/24

The more that the suggestion is accepted (or at least not rejected) by users in other contexts, the more confidence you get in that quality and generalizability of that suggestion.

This approach doesn't work for high-downside scenarios.

Especially scenarios where unless a user takes the time to audit it, they wouldn't notice a mistake in the suggestion.

In that case, just because the vast majority of users have not rejected the suggestion does not mean that it's high quality.

But you can fix that by creating an asymmetry: if any user rejects the suggestion, take that as a much stronger signal than many, many users accepting the suggestion.

A user making a proactive "that's not right!" is way more powerful than a user passively going "shrug, looks OK to me I guess".

But in each case what matters the most is: which situation had the most proactive and informed user intent? And how bad is the downside if the suggestion is wrong?

More on this topic

From other episodes