An ideal system would benefit from the insights of a swarm of users.

If there is a lot of overlap of queries from users, then even small bits of engagement and signal can be used to improve the quality for large classes of users.

The first time an automatically-generated answer is shown to a user, it's not known to be very good.

LLMs return answers that society cached in oft-repeated utterances.

But until a human says "yeah, that looks OK" or even better "yup, this is good" then you don't know if it's appropriate to cache and return this answer automatically, quickly, to future questions from other users.

But the more users who see that result and don't barf, the more likely it is to be good.

Even a small number of highly motivated users interacting with the result in a high-intent way (e.g. pinning it) is a great sign it's useful.

How can you have humans in the loop implicitly with small bits of signal that stand out from the background noise

Humans are a form of ground truth, pinning down what's valuable / useful.

If you have humans in the loop and the signal can pop out of the general noise, you get a very powerful, self-ratcheting quality system.

Even if only a small proportion of users lightly interact with only a small number of things, at scale that's still enormous amounts of signal--so long as the other interactions from users that don't line up with quality are uncorrelated.

A pattern where to any specific individual it looks like noise, but if it's consistent, at the level of the population it pops out clearly when you average.

A thing that if you look at the swarm as a whole it looks just like noise, but the consistency of alignment creates coherence.

That signal, at large enough scale, could help ratchet up quality significantly.