Use big models in proportion to surprise.

· Bits and Bobs 4/27/26
  • Use big models in proportion to surprise.
    • Big models are very expensive, but also very good at handling surprise resiliently.
    • Use the big model to start without a harness.
    • Then, as you get a better handle on the scope of inputs and can make a harness to dial in the unsurprising bits.
    • Once the harness handles more of the common cases, you can use a smaller cheaper model.
    • As you keep on going on this path, ultimately the harness is 100% and you're left with just mechanistic code.