Any time a formal-ish system meets the messiness of the real world you get a combinatorial blow-up.
- Any time a formal-ish system meets the messiness of the real world you get a combinatorial blow-up.
- The real world is fractally complicated.
- If you have only black-and-white formal descriptors, then they need infinite resolution to capture that fractal complexity.
- Unless you can flip it not as a top-down description, but a bottom-up description it emerges from.
- The former can always exist, but might blow up in space.
- The latter can efficiently encode complex real-world phenomena… but only a specific subset.
- For example, DNA can describe all of the living things on earth… but there are presumably living things that could be imagined and described that couldn't be encoded in this DNA codec.
- Even small changes from what exists might look simple to describe but be impossible to generate efficiently.
- This is one reason that legal text is inscrutable and overlong to a lay person.
- LLMs can absorb the real world complexity much more efficiently than a black and white formal grammar because they are fundamentally able to represent shades of gray.