LLMs do magic at non-trivial expense… and current systems just throw that out.
- LLMs do magic at non-trivial expense… and current systems just throw that out.
- Often we don't cache or keep around the result.
- For example, distilling a bit of mechanistic software that can be run without the LLM.
- How can you make it so that the number of overlaps is increased?
- That is, where a previous answer can be used without having to re-calculate with an LLM.
- What if you could do overlaps across users, strangers even?
- You'd need to make it safe to run a stranger's code.
- You'd also need to make sure that the bits were split up into smaller components, so the likelihood of an overlap of a sub-component is higher.