59 chunks · 44 episodes
In every paradigm, evolution pops up a pace layer and then takes over that layer. it pokes through and then spreads out and changes everything. That's the history described in my old essay The Runawa...
... that happens complexity isn't possible to emerge. Once it does it can pop up a pace layer.
.... Things roll down hill. Evolution is gradient descent within faster and faster pace layers. Every so often a new paradigm creates a new even faster pace layer on top. Gradient descent without a goal, without a north star of meaning, optimi...
Lower pace layers must go slower. They are higher leverage but must move slower. If you think you're at a higher pace layer but are at a lower one, you'll have a bad ...
...eed gives you something beige. Curation and caching needs to happen in a higher pace layer, outside of the model weights, and it needs to involve deeper reflection from real humans.
...uild on a foundation of jello. You need an infinite viscosity layer as the base pace layer, the bedrock.
The force that separates things into pace layers is the rate of change. A layer that is supposed to be a lower layer but has a high rate of change is impossible to build on top of.
...o? Does the model set the ceiling for what you can do… or the floor? You need a pace layer outside the model to accumulate intermediate insights. Those intermediate insights can be fed back into the model in future iterations as context to ...
Three pace layers of prototyping with LLMs: 1) The LLM does everything. Expensive, loosey-goosey, flexible 2) The LLM behavior is sublimated into a mechanistic harnes...
...t. It's a snapshot of when it was trained. The model can't shift but the memory pace layer can.
Biology also has pace layers. All animal cells are remarkably similar. But at the macro scale the animals that are made up of cells are wildly diverse. A boring lower level give...
A fast pace layer can be sublimated into a lower pace layer once the best practices are conclusively discovered. At that point, no one bothers trying much different at...
Maybe we'll see a new explosion of innovation at a higher pace layer because of the great LLM freeze. The "Javascript Industrial Complex" has led to an extraordinary amount of "innovation" in the client layers for the ...
Two pace layers intermixed will be chaotic and slow. If you have two pace layers intermixed, they fight each other in an eddy current and neither can run at their f...
...orce-fed AI-generated and auto-ground-truthed examples. This codex is at a slow pace layer; it takes months for the LLMs to be trained and then deployed. It's also indirect. In a world of infinite software you'll want a tiered system, with ...
...current app to the current model. We're used to the lower layers being a slower pace layer. But now they can go at a faster pace layer, and they lap your app and swamp you.
...of what the platform creators made. It's easier to change, and thus is a higher pace layer. Subsuming functionality into the platform is expensive and should only be done when there's some benefit to doing it in the platform. An overly-larg...
Innovation moves up pace layers over time. As a given pace layer gets to the late-stage, where the momentum has coalesced around the obvious winners, competition moves up the stack...
...to use with the latest Tailwind version. RL improvements in models are at a low pace layer; it might take a year or more for the model to catch up to how Tailwind is now written. There was always some hysteresis in developer ecosystems. It ...
Just because you can heal it doesn't mean you should. Healing at the wrong pace layer is harmful because it hides the brokenness below. If it's broken below in a fundamental way making it look harmonious on top is bad.