LLMs are so noisy that it's hard to figure out the quality of underlying components built on top, because the noise dominates the signal.

2025-05-05 · Bits and Bobs 5/5/25

LLMs are so noisy that it's hard to figure out the quality of underlying components built on top, because the noise dominates the signal.
- Did it break because the surrounding system broke or because the LLM's behavior changed with an additional period that you added to the prompt?

More on this topic

From other episodes

The United States frontier was partitioned according to the Public Land Survey System.2026-06-15
LLMs make intellectual technicians less necessary.2026-06-15
LLMs are great at making prototypes more real.2026-06-15
A tweet making an interesting observation: "No one really claims that alphafold is conscious, or that sora or midjourney or dall-e are conscious."2026-06-08
LLMs can make a superficially compelling argument even on absurd premises.2026-06-08
A new piece from Ben Hunt: Contact: AI and the Semantic Dimension.2026-06-08
LLMs have made it so much more of the potential of programming is possible.2026-06-01
The level of difficulty that people can tackle goes way up with LLMs.2026-06-01