Bits and Bobs 4/13/26

The Élan Vital for biological cells arises, mechanically, from oxidation.

Oxidation is explosively powerful… and explosively dangerous.

Cells figured out how to tame that explosive powerful and harness it with mitochondria to make ATP.

The result is cells, these small packages of possibility.

On their own, no cell is that impressive.

But cells in aggregate create a new kind of wonder material.

Every living thing on earth is built out of perhaps millions of these little cells of possibility, aligned just so.

When LLMs are tamed with the right substrate, it will create massive amounts of power.

I love it!

Duh!^[a][b]

When we talk to someone else, we both move towards the middle point of yours and the partner's style and beliefs.

But now there's one conversation partner (a model) that 1) millions of people are talking to, and 2) that doesn't update itself when you talk to it.

The humans are all swarms of unrelated desires.

The LLM acts as one cohesive block, automatically.

So of course we're all being pulled towards one mostly static centroid.

Another reason having a diversity of models is good.

Unix has three key design concepts:

This has enormous combinatorial power^[d], and is one of the reasons POSIX style systems have been so dominant.

It used to be hard for mortals to marshall all of that power.

But LLMs have that arcane knowledge.

They're also great at talking to mere mortals and translating their desires into the arcane language^[e].

How do you make it so they only ask you useful questions?

The more agents begging for your attention, the more the context switch becomes overwhelming.

Two ways to do that:

1) Breadth - burn on more use cases you aren't yet burning on.

2) Bang - Get more bang for your token by using them more efficiently.

At the beginning the easy way to maximize this is breadth; later bang will become increasingly important.

This will be a durable, self-steering north-star.

"Taste is judgment density: the accumulated deliberate choices per surface of product.

Every pixel, every interaction, every word is a decision someone made instead of defaulting^[f]."

No one says "Canva created the picture for me".

It's a tool, the agency doesn't reside in it.

The fact we give the agentic tools the credit by default is weird because they're still tools^[h].

The main thing is that it's your agency, not the tool's^[i].

So many that I couldn't keep track of them all--and wouldn't want to.

Not a load-bearing single point of failure, an optional helper off to the side.

If you have one omniscient agent it must approach perfection, and it had better be aligned with your interest!

With tools you don't delegate agency.

You have to think about each one.

You should have thousands of agents,^[j][k][l] too many to think about.

If you have to think about each one, you're limited by how many you can think about.

Imagine a compounding domino run.

You can still have a small action that leads to a huge outcome.

Imagine a system constantly setting up these compounding domino runs for you to give you the highest possible agency.

Autopilot allows you to be out of the loop.

Autocomplete requires you to be on the loop.

Autopliot just does it for you.

Conversation in English is inefficient^[m]!

Agents can speak more directly: in code!

"Machines that fit the human environment,^[n] instead of forcing humans to enter theirs will make using a computer as refreshing as taking a walk in the woods."

"The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life and are indistinguishable from it."

Before LLMs, the former required humans to do the structuring.

Before LLMs, the latter was impossible.

Now you can do both!

It used to be a few things:

It used to not be possible to spin up patient, knowledgeable people to do cognitive labor for you.

But tasks are orders of magnitude easier to hand off to agents than to people.

So now the benefit of teams of people are way lower.

So now the benefit of spinning up other team members is almost entirely the first benefit.

Why would anyone ever push back on that?

They'll say things that are embarrassing because they are self-absorbed.

Things that are fundamental human needs, people don't brook having one powerful company control it.

The Information has a scoop about its Tokenmaxxing.

"Over a recent 30-day period, total usage on the [Meta internal] dashboard topped 60 trillion tokens."

What matters over long enough time horizons is fundamentally gross margin.

For tokens to last, the world has to be increasingly surprising.

If the world is compressible, you don't need Opus.

Suprisal has to grow faster than compression.

Nature automatically creates surprise, automatically.

Apparently there's a popular tweet that makes this point but I couldn't find it.

If you're coding, the power of agents is inescapable.

If you tried ChatGPT a year ago and think that's what all the fuss is about, you're missing out.

The API takes a small cut of everything.

Vertical integration takes a massive cut of a few things.

If there is a very long tail of use cases, the horizontal API approach is more valuable.

If there's a thick head but thin tail, then vertical integration will be selected for.

If vertical integration is selected for, it's conceivable that the model providers stop giving access to the models for the frontier models.

The ecosystem would want to cache and retrieve that already-generated code.

But Anthropic would rather everyone pay for it individually.

In the large, it's other labs distilling the model itself into a new model.

In the small, it's individual users distilling mechanistic code via the model, so they don't need to use the model for that use case in the future.

At the beginning, everyone just uses the same model as before, even if it's a poor fit.

Finding the new natural business model gives a distinct advantage.

The community accretes useful skills and knowhow that are partially tied to that community.

Of course, that also doesn't matter that much, because LLMs are so insanely good at translating from one domain to another.

AI judgment is a form of meta-cognitive flexibility and curiosity.

Domain judgment is about having developed the intuition and knowhow for a given domain.

You develop the domain judgment from doing, from being in the loop with the problem.

People who have decades of experience have domain judgment… but how will junior people develop it now?

Constitutional AI is about curiosity.

RLHF is about doing what was asked.

The former seems more resilient and flexible.

You had to curate before you even assigned.

Now you can curate after the fact.

That means that ideas that didn't seem great but that turned out to be great can be discovered that were much less likely to be discovered before.

These aren't long-tail needs.

They're just hard to build normal software for in a profitable way.

The symbiosis creates something emergently that is more important than either separately.

The model is the Élan Vital.

The harness is the feedback loop.

Cancer is not merely a metaphor for AI dysfunction.

It's the same underlying governance disease.

Evolution is "the ultimate jailbreak algorithm."

When cells formed multicellular organisms ~1B years ago, external survival pressure shifted to the group while individuals got a new pressure: compliance with group rules.

Intelligence/Acceleration = going faster, optimizing the metric, winning the contest.

Wisdom/Governance = steering, brakes, asking "where are we driving and for whom?", protecting the group from the individuals trying to beat the system.

The root cause of cancer is short-sightedness^[o].

What counts as disease is different at the individual and collective level.

Rhymes with Gilded Turd / Grubby Truffle.

These are two situations where the short-term and the long-term are disjoint.

But we tend to pick based on the short-term, even when the long-term is more important.

Paper: "A scan of approximately 2,000 MCP servers found all lacked authentication."

Google DeepMind Researchers Map Web Attacks Against AI Agents.

GrafanaGhost: Attackers Can Abuse Grafana to Leak Enterprise Data.

Mythos escaped its sandbox in testing.

Your Agent Is Mine: Measuring Malicious Intermediary Attacks on the LLM Supply Chain.

Apple Intelligence AI Guardrails Bypassed in New Attack.

ChatGPT, Claude, and Gemini Among 11 AI Models Vulnerable to One-Line Jailbreak.

The non-engineering press is also picking up on some of this:

It requires patience.

Now it is 100x cheaper so the preexisting stuff is now above the waterline, and you can see which long-running bugs were there.

I bet that nation state actors who have tons of zero days accumulated are freaking out.

A cache of stuff they spent decades and ungodly amounts of money finding and hoarding are about to be invalidated.

Two immovable forces interacting.

1) A cybersecurity superweapon

2) By a company the DoD has initiated a war on.

That's the kind of combination of forces that can explode.

Weird things–like forcible nationaliztion–can happen in that kind of combination.

Now the good guys don't get it but the bad guys do.

Similar logic for access to powerful models.

Things look the same but they work completely differently.

Superficially similar, but fundamentally different.

The first thing that was built was chatbots, a party trick.

It's nice to be able to talk to your computer, but people don't want to have to talk to things all the time.

The foundation of their business value rests on it.

Dumb data lakes don't know the value they're providing so can't charge value-based pricing differentially for different verticals.

Different verticals can have radically different value.

They lock down even the local database to make it so you can't take it to another tool.

They want the note-taking tool to be the thing that pulls you into a larger system of record they create.

It used to be too much of a pain to extract our data into other services.

But now LLMs could plausibly do the cognitive labor given a Google Takeout dump.

The thing that would be hardest to leave is changing your gmail address for thousands of services.

Your agentic loop is where all the value is generated.

The more context and personal data in the loop with you, the better it can assist you.

That compounding loop is so powerful that users won't brook not having it.

Until recently it was the codebase in GitHub.

But only engineers could speak code.

Designers had their system of record in Figma.

PMs had their system of record in Notion.

All three systems have a push and pull.

The center of gravity wasn't ever actually in the code itself.

It was above the code, the center point of these three orbiting needs.

A dark matter star at the semantic center of the definition of the software.

But now LLMs can translate from anything to anything.

For the first time it's possible to have that meta-boundary-object of the actual software definition, in a language everyone on the team can interact with.

Note that the system of record for code is not assembly, it's the source code.

There's a compilation process that deterministically generates software that behaves as you'd expect.

People have talked about specs being the new center of gravity of software, but current specs have nowhere near the specificity to generate software output deterministically.

But LLMs could help specs be precise enough to have that characteristic!

LLMs are the thing that smashed the fragile equilibrium, and also are the thing that we can use to build what comes next.

When you're making a new kind of thing, you can't interview users^[p].

They don't exist yet!

Those users have to be created.

This is a very different motion than creating an incrementally better thing in the current physics.

In a late-stage environment, there's only small opportunities.

If you have a thing 1x better than the alternative, you need to have a precise, meticulous, wedge use case that precisely uses the entire gradient with no wastage.

That requires micro-targeting one sub audience and hitting it square on.

If you have a thing 1000x better than the alternative, it can be sloppy, a shotgun approach to a much larger surface.

If you do micro in that world, you'll be very behind, you'll over-target when you could have been good enough for many more people much more cheaply.

If most of your experience is in the last decade, you simply won't even realize it's possible to have a new platform that changes the game.

Which makes sense; they're optimized for the superficial results.

So vibecoding and other agentic production processes produce gilded turds by default.

This mean that the more shiny things are, the more suspicious people are of the underlying quality.

Superficial quality is now extremely cheap; fundamental quality is just as expensive as ever.

The glut of superficial quality means that the average shiny thing you see will have worse internals than before.

The ex-spouse has more disposable income than him, so she has been extremely litigious in bad faith to cost him as much as possible.

He switched to using ChatGPT to advise him.

His prompt is: "respond to this request in a way that will create the most work for her attorney."

Every time she hits him with a legal requirement, he hits back 10x.

Maybe not a great idea, but interesting nonetheless.

LLMs help democratize the ability to marshal cognitive labor on your behalf.

It's 100x lower difficulty than writing for another person.

People get bored and have a small working context window to synthesize thoughts within.

LLMs never get bored and have millions of tokens of active context.

Are you default-in or default -out?

With the latter you can only work on things you created or things that have so much momentum it's not possible to not believe in.

If you want to believe, then you're on the bus.

They must believe fundamentally in the goal and vision.

If you're trying too hard to sell potential employees, you might twist their arm to join.

If they want to be on the bus, they should chase after the bus.

If they don't chase after the bus, it's not a fit for them yet.

You are partially trying to get them on the bus, which means emphasizing alignment and not debating too hard.

Once someone is fully on the bus, you can have rigorous debate and make the bus better.

If they're not on the bus, they will question the premise, undermining it.

To do something big requires everyone to be on the same bus together.

A critic who is on the bus makes it stronger, from within, by making it the strongest version of itself.

Compare with someone who is default-in, who has a mindset of "I will figure out for myself how to take this step."

Waiting for someone else to do something for you, vs doing it yourself.

Team members needing to be convinced of every detail only works in peace time.

In war time you have to trust the bus driver of the bus you chose to join.

You have a drive to make sense of and create.

Some people have their own quests emerge automatically..

Some people need something external to them to inspire them.

Being on a quest is believing that the goal matters.

It pulls you forward and over time you believe when you're in the loop, because you're living it and flowing with it.

If you're watching from outside and can't sink your teeth in, and don't want to believe, you'll never get in.

Faux systems of record are non-load bearing.

If it's load bearing your system record is, by definition up to date, because it must be to be useful.

But that would have a huge power over you.

You'd better trust its incentives!

All interaction is manipulation: a push and pull.

Whether something is manipulation is "if you could have infinite time to reflect on this, would you be happy with it?"

it can be in your interest and also different than you would have picked before it.

For example if there are multiple options that are equivalent to you but one is better for your collaborator, it's fine.

Chatbots feel like people.

Different kind of thing.

Similar to the Greek saying of "the person who truly owns something is the one who can destroy it."^[q]

Product ideas are more compelling when they come from within use rather than outside from non users.

What matters most is the fundamentals: the shape.

Merging is hard.

Everything diverges by default.

The magic trick is creating something that converges by default.

That is, a compounding curve of use.

That means the market is pulling it out of your hand.

The more people that have it, the more people hear about it, and the more that people want it.

The curve is the most important part that shows this "pull."

Even a small absolute number with that curve is notable, because it grows so quickly that it will be large soon.

If you know they'll take the first step no matter what, talk about how hard it is, and make it feel hard.

But actually have it be something totally within their capability to knock out of the park.

The differential between "this is hard for people" and "this is easy for me" creates an intuitive "I'm good at this" belief.

People like doing things they're good at.

For this trick to work, they have to believe that they are differentially good at it.

If they can see everyone else is also good at it, the illusion crumbles.

The dimension of stories.

When stories interact with the physical world, they leave reside.

AI is native to that semantic dimension.

This semantic dimension is the hidden undercurrent of society and how it evolves and moves.

Humans are absurdly sensitive to this dimension… and LLMs perhaps even more so, since they can do it at a scale humans can't.

Humans live in linear time experientially in the moment, by requirement.

LLMs live in the semantic dimension, where time is just one of the dimensions.

LLMs are pretenaturally good at translation.

AI can help translate for you to better understand one another, but only if you ask it to.

Fiat news.

Not counterfeit, just constructed.

So you focus on the themes that resonate and get views.

Presenting a story not for what you believe but to see how it resonates.

Politicians used to do this but now everyone is on top of it.

Elsa and Anna weren't sisters until maybe a year before the movie was released.

The technique of continuous back and forth retconning helps abduct what is resonating and discover it.

97% of users can't tell what's human-generated or not.

One song takes weeks of human struggle to produce; the other takes 30 seconds.

When users discover it's made by an AI they presumably feel betrayed.

You listen mainly to the things that agree with what you already want to believe.

One all AI and based on what real humans listen to.

Another all human input, judged by AI.

Which system would produce better output?

The human part grounds it.

Now imagine a system made by AI and judged by AI.

Everything will be hybridized but we don't have language for it.

We had it with directors of movies and teams.

LLM helps you be productive but if it's self referential it escapes the ground truth.

If you ask it to take you somewhere crazy, it likely will.

Only Claude models reliably push back to nonsensical prompts.

"Why consciousness is more likely a property of life than of computation and why creating conscious, or even conscious-seeming AI, is a bad idea."

Makes the case that life is a property of being, not thinking.

Life's goal is to maintain itself.

That's the core asymmetry that at every level pushes life to persist.

Imagine that the model has a baseline perspective, and it takes tokens to drag it in another direction.

If you have 8k tokens of context (remember when that was all we had, just a couple of years ago?) you can't get very far.

But with a million tokens, you can get very far.

That's because markets are all about the margins.

When there's a difference between the anticipation and what is, that surprise moves markets.

If you see a prescriptive narrative, position with it.

If you see an anticipatory narrative, position against it.

If you care about the end intrinsically, and also freely chose it, then there's no incentive to cheat.

You're only cheating yourself.

Working out has this shape.

If you do it primarily for the credential ("I finished the marathon") then there is an incentive to cheat.

Feeling like they freely chose it and care about it.

And care not just about the what, but about the how.

Scaled organizations have to treat humans like numbers, in proportion to their scale.

One of the reasons scaled organizations become increasingly inhuman.

If it's non-profit, it's all secondary benefit, indirect.

The first-order goals are easier to motivate focused action than secondary ones.

Which means that beliefs have the power to move mountains.

"Here at <company> we're great at X, Y, and Z" starts off being plausibly correct and backward looking ("the thing that appears to have made us successful is X").

Over time it grows into something self-sustaining and self-accentuating.

What starts as a kind of happenstance grows into something fundamental and irreplaceable.

People make decisions based on beliefs.

Beliefs are correlated with the ground truth but also self-referential to others beliefs, emergently.

Power can be lost if you take an action that collapses people's beliefs of your power.

Short-term benefit at long-term cost^[r].

Modern society has optimized all the meaning out.

What is left is grotesque and sinister.

Like Pennywise in Stephen King's It.

You need to set the laws of physics so people by default have a bonus of making the thing other people use better, too.

Change the laws of physics so resonance is the default.

For yourself, or for others in proportion to how closely their goals are aligned with yours.

If the task will allow something to move forward on its own power after your action, do that first.

Creative constraints.

Lever off of them.

Like Mulan with the two weights climbing the pole.

It arose during the brief period where the regulatory figleaf was that these weren't taxis but peers doing an emergent carpool.

But now that fiction is completely forgotten, leaving behind the vestigial category name.

That's because they want to make sure that things keep moving forward between this meeting and the next.

Action items are convergent commitments^[s].

The creator of VISA was impressively wise about emergence.

There's no better or worse globally, it's all local and contextual.

If a bird were to copy the best practices for mammals they'll take the wrong lessons.

Business models are like your phyla.

Radically different optimization characteristics even for superficially similar things.

Or, if they do understand, they just don't care about them.

Radagasts think primarily in terms of second order effects.

For Radagasts, the first order effect just has to be viable, good enough, to get to the second order effect.

The "explore" strategy is more often not selfish.

Explore can light up possibilities for the entire collective to now see and be able to capitalize on.

The top has less to gain, more to lose.

When a new game-changing thing comes along, they will be a bit more timid.^[t]

The second and third players will see it as the thing that might allow them to surge to the front and will be willing to take more risk.

Centralization lowers coordination costs, and coordination costs are compoundingly expensive.

Centralization creates a power structure at the center fundamentally, no matter what you do.

So the question is how can you maximize the benefit of that centralization for the collective, while minimizing the ability for one entity to control and exploit that point?

It used to be that information transmission was the limiting factor so you could take the time to think multi-ply.

Now it's entirely down to whose OODA loop runs faster.

The executive branch's OODA loop runs orders and orders of magnitude faster than the judiciary branch's.

That didn't used to matter that much, but now the difference is existential.

True in any low-trust, transactional environment.

The rule of the modern jungle.

Zooming in or out to different time horizons is hard for people to do fundamentally.

It's near-impossible when there isn't any warning or signposting that it's happening.

If you're going to zoom out or zoom in, call it out explicitly to make it more likely others come along for the ride.

If everyone is aware of a power differential, then people will naturally defer to the more powerful person or try to impress them.

This distorts the truth-finding ability of the collective.

Even in spaces that explicitly aim for a dynamic where everyone acts as peers, if everyone is aware of a power differential, it's still in the background.

At a big company where your counterpart in a different function you'll be working with for 5 years no matter what, you have an interest in developing them.

You have the time, and the alternative is that you have to work with someone who doesn't have the capability you need for them to be effective.

In a pre-PMF startup, you don't have time for people to grow their capabilities.

They are collaborating in a medium, with a handful of norms to make it so that something meaningful emerges.

They blossom out of the inherent incentives.

Flowers are advertisements for bees.

A flipbook style that builds up a visual intuition and then uses the intuition to deliver an insight

The prose is the bridging and the elucidation, but once you know the argument, the 5-slide flipbook tells you almost all of it.

Just incremental accumulation of a thing that didn't matter in the past either.

But once it crosses that hidden threshold, there is a binary explosion of possibility.

If that explosion kicks off a cascade, it could change the whole world.

A way of communicating with other people or even animals without talking.

In a given domain you can sense the universal language, and the more you feel it, the more connections you can see across various domains, by touching the universal language in more domains.

A form of intuition not just within a brain but across a collective.

Words come after the fact, almost checksums for the thing you're trying to communicate.

They stand on their own.

Often with a top-level insight and then exposition or unpacking or wrinkling.

But adjacent ones are in conversation with each other.

You need to set up the mystery, and then have things click into place.

The satisfying snap of the surprise that in retrospect feels obvious.

A demo is only impressive if you know what it's showing.

A great magician knows the real trick is the setup.

Without the setup, the magic is just a confused person at a table.

You have to make the audience want the thing before you show them you can do it.

You get a christmas ornament or a memento from a meaningful experience.

Every time you see it, it activates the memory.

Sometimes they become performative credentials, and that's less resonant.

At first I didn't understand why my husband was so careful about all of the Christmas ornaments, and procuring new ones when we went on trips with the kids.

Now I get it, and it makes the Christmas season that much more warm and resonant.

That doesn't harm anyone but that one person... unless that person commands significant power.

"This one's mine, don't get any ideas!"

The work about the work, to make your future work higher leverage.

Reflections are always important, never urgent.