The LLM is just an input.
- The LLM is just an input.
- Intelligence requires the structure around that input.
- That structure needs to be stateful.
- The LLM is the only non-trivial component in the system.
- Everything else is trivial
- If a provider has a small lead and wants to compound it, they'd add features that store state for the users.
- Then they hope that like a monkey trap the users do the convenient thing and then get stuck.
- That would then allow their temporary lead to become a permanent one.
- Don't store state with your LLM provider.
- They have the expensive component and can lock you in.