The quality of a cherry picked thing doesn't reveal anything about the underlying distribution of quality.
- The quality of a cherry picked thing doesn't reveal anything about the underlying distribution of quality.
- It only reveals something about the quality of the curation function: the quality of the cherry picking process.
- If you don't know how much effort went into the cherrypick, it could be like the magic trick where the magician just puts orders of magnitude more effort into it than you thought would be reasonable.
- "Wow, the demos are so good, this system must be great!"
- "But how much effort did it take to find a thing to demo that was that good?"
- People judge the quality of the magic trick based on the impressiveness of the result, but it's really more about "impressiveness per unit effort" that matters.