1,749 mentions · 790 chunks · 131 episodes
... the Semantic Dimension. Makes the case that there is a semantic dimension that LLMs are able to surf through. A dimension that is in a separate plane from the spatial dimensions. Before, we could sense it, but only indirectly. Now we...
LLMs have made it so much more of the potential of programming is possible. Programming has infinite, open-ended possibility. But before, it was too expen...
The level of difficulty that people can tackle goes way up with LLMs. The stuff that was hard is now easy. The stuff that was impossible is now hard.
A sweet spot for LLMs: drawing on expertise across many different domains. Humans can only become experts in a handful of domains. There are tons of combinations of domain...
.... A warehouse full of books deliberately destroyed by the ingestion process for LLMs. I'm glad they did it, I think LLMs are useful for society and I'd rather have them include that data than not. But I feel visceral disgust when I se...
.... Before, your differential in quality on that task gave you an edge. Now, when LLMs are better than you (and most everyone else), everyone just outsources to the AI. The rising quality of LLMs forces you to commoditize your different...
...hem on the ladder, where they could iteratively pull themselves up. But now the LLMs mean that for what used to be the earliest rungs, the LLM can do a better job. You might as well use the LLM to do it… but without understanding it, ...
... from my friend Kasey: English isn't a programming language (yet) It feels like LLMs allow a new kind of boundary object between domains in software development.
...s get more and more confused over the course of a session.[h] Humans learn, but LLMs don't. It's a fixed underlying model, and the context, which approximates learning in a session, gets increasingly polluted and corrupted.
LLMs are like sea level dropping by 10 meters.[j] A previously expensive thing (writing code) drops its cost by multiple orders of magnitude. The most obv...
It's unclear how LLMs will affect open source. It's possible that we've been in a golden age of open source that is coming to a close. Before the difficulty of making the ...
...h and productionization has always felt wildly slower than the first phase, but LLMs 10x it. The first phase sets the expectations unreasonably high!
We grow software now. That was always true, even before LLMs. But now it's obviously, inescapably true.
... long to train and adapt. They are also a closed set. Another approach: use the LLMs as dumb muscle, not the brains. The brains would still come from real users with situated judgement. If you could have the human judgment at a faster...
...ue with distilled ranking signals in Search. But it's also true in the world of LLMs.
...t than we can. We can only do it with compression. There are certain tasks that LLMs can do easily that humans can't. Needle-in-a-haystack style tasks.
Hierarchical Task Network Planning might finally be viable in the world of LLMs. This was a technique I learned in my AI class in college in 2007. You take a high-level task and break it down into smaller tasks, recursively, unti...
... better spent on explore than exploit. Exploring is about surprise, which means LLMs are great for it. Exploiting starts off being about surprise, but as it gets dialed in on a given hill, becomes about efficiency. Mechanistic softwar...
Giving the right context to LLMs is such a powerful unlock that it feels like cheating. They're a steam engine, just fill them with the right fuel and they can move mountains.
...ntic Engineering was just robust file editing. As recently as a few months ago, LLMs editing files was error-prone and slow. Streaming out an edited file, token by token, and hoping the LLM doesn't get confused or try to 'improve' som...