The Anomaly Almanac: A meta-pattern for systematizing a real-world domain into a default-converging model.
Describe the model as it currently exists, as well as a list of "surprising" examples that don't yet fit the model.
At the beginning, your model is empty, so every observation is surprising.
Then, fold the surprising examples into your model.
Now keep going, looking for more observations and keeping any surprising ones.
You can discard the ones that aren't surprising.
Every so often, fold the new surprising ones into your model.
The list of at-some-point-surprising observations provides an easy test to check your model against as you build it.
How good your model is is how rarely you find surprising examples.
Of course, this presumes you are actively looking for disconfirming evidence.
That exploration will help you detect when the conditions have changed and you need to start smoothly adapting your model, instead of getting caught off guard and having your model shatter in a new context.
Once you have a few new surprising examples, you have a model that is "dialed in".
This meta approach naturally samples from how common use cases are in the wild.
This is the exact opposite of how surprises are treated in everyday execution in modern business.
Every surprise is inconvenient, possibly distracting, something to sand down and get rid of.
Instead, we should treat surprises as precious.
One watch out: be careful to get diverse-enough input.
We tend to only get input in domains where applying our model gives us good-enough results.
That can lead to pulling you at an accelerating rate into a small niche.
That's why surprising examples are extraordinarily valuable.