Bits and Bobs 9/21/26

In the future it should revolve around people.

Now, with LLMs, the machines can bend to us.

So you have to proactively push yourself to think deeper, not faster.

Each silo just wants you to stay engaged and not think too deeply.

Just keep using the service.

Now, agents can be atomized to do the fragmented work for us, so we can have coherence.

Two people can only collaborate if they are in the same silo.

This is a pain in the butt.

This is where most of the annoyance comes from in collaboration.

Today all of the labor of connecting across silos falls on you, not the machines.

What if all of that pain and effort could just evaporate?

How much more would be possible?

To collaborate with someone else, you need to be in the same silo.

The difficulty of finding a silo everyone you want to collaborate with is already in scales with the square of number of participants (n).

And then having the relevant data/context in the silo per person scales with the square of the number of participants, too.

n²n² = n⁴

This is a massive headwind.

Even if there were a piece of software that did precisely what would be most useful for that use case, you'd have to overcome a hurricane force coordination headwind for it to be viable.

So the bespoke collaboration software is possible to exist, but not viable.

So no one bothered creating it in the first place.

Now, LLMs can create custom software on demand… but they can't solve this coordination problem.

An app would have to be better by 1000x than WhatsApp to be viable to coordinate with in today's physics.

It takes significant work to make the square peg of our social reality fit into the round hole of silos in computers.

Whose learning loop will be better?

The former is only possible with your data.

The latter requires buying someone else's data.

A friend did a simulation and concluded that the learning loop aggregator (the wide one) does better no matter how good the data.

Transfer learning is just too powerful.

Most of that context is within the Coasian veil.

Inside of your own silo, whether as an individual or as a company.

Entities trying to train a general model will use living companies as blood bags.

Acquired to run them for their data.

We're going to see startups approach mom-and-pop shops and offer to pay them $50k a year if they connect their internal software to data slurping tools.

A half-assed bot with a tiny slice of your relevant context.

A bot that works for the silo, not you.

Lame.

Compressing themselves with every request.

Blog: Our agent used 5B tokens to build a business empire in 3 weeks. It made $1.54.

When tests are expensive to produce, they are nearly always real.

But when they're cheap, they just become random fixedness of implementation pins.

They fill in underspecified things with a kind of generic mush.

This spackle can fill out a good skeleton and make it more substantial.

"Below this level of specification, I don't care about details… do whatever seems best."

This allows creating much bigger systems without having to do all of the intricate details.

But spackle cannot create a strong structure on its own.

If the whole codebase is spackle, then it gets increasingly expensive to add anything new to it or improve it.

If the system has strong bones, then spackle can be used to fill it in with leverage.

Most programs, no matter how complicated, can be understood, inductively, by human minds.

Each layer was laid down by a human mind, nesting and abstracting.

But systems like LLM models are like programs that were grown via process.

The models are thus like a programming language that no human mind will ever be able to comprehend.

You can't wrap your head around it, so you can't fix it or debug.

Every feature you implement you trade off future optionality, because you now have to keep it working.

As you implement more features, you're pulled closer to the heat death of your codebase, where all futures are precluded.

The only way to get more futures is to pay down debt.

It used to take codebases decades to reach their heat death.

Now, it's possible to reach it in weeks with LLMs spraying code.

LLMs can do tech debt reduction… but only if asked to.

If you don't ask them to do it, then you'll get more and more functionality by destroying the optionality of the codebase.

The difference between well-implemented (not unnecessarily reducing the future possibility) and poorly implemented (constraining the future possibility unnecessarily) is visible only to people with engineering experience.

Just because you can vibecode doesn't mean you feel this in your bones.

The difference between a Gilded Turd codebase and a Grubby Truffle codebase will be invisible to non-technical people (e.g. management).

The more new things you need to specify and create, the more likely something goes wrong.

The more new parts, the more difficult the coordination.

When the system is well factored the number of short incremental definitions you can do to unlock incremental value is huge.

When you specify the data model and invariants cleanly at the right level, the system's value can emerge efficiently.

"Conceptually, there is one foo for every bar. But there are a huge number of bars, so we optimize it to only create a foo for bars the user is likely to visit."

North stars can only be created after reflection.

It used to be hard to reflect.

Now, LLMs can do it for you, so great engineers have north-stars so their work accumulates.

Make sure you're on the right hill!

It's doing it to enable a whole new class of multi-user software that wasn't even conceivable in today's physics of trust.

That's a dangerous ability, so you should only collaborate with people you trust deeply.

Would you trust them alone in your house for an afternoon?

If not, then you shouldn't allow them to inject your agent.

Don't push forward, pull yourself to the desired state.

This makes sure you don't get there by layering on short-term fixes that don't adhere into something with strong bones.

When building a product, you'll lean on New more than you should.

But you intrinsically care about your own product, and consumers by default don't.

You should flip it and focus on:

It only counts as "Better" if 10/10 of your customers agree that it's Better.

The vast majority of New ideas won't work for non-obvious reasons.

Proven ideas are more likely to work.

But when there's a paradigm shift, many of the previously Proven ideas no longer work, and it's only a New idea that can.

But which one?

Tech is an implementation detail.

If so, there's a gradient of adoption.

"Expected" is important–it's based on their mental model and priors.

If your product has a search box that looks like Google's, it had better do as good a job at queries as Google.

But if that same feature looks unlike Google, the good enough bar for quality is much lower.

1) The quality of the model

2) Prompting quality

3) What is the token budget?

4) The richness of context and tools.

The F idea will at least fail fast and you can move on.

An idea with a ceiling of B- will absorb huge amounts of your time.

You'll keep working at it, and not shift to an idea that might be an A+ idea.

We don't even think about it; some things simply aren't viable and we don't even miss them.

They're just too impossibly expensive to even entertain for a second.

Hermes could produce a ton of value… if you spent months setting it up.

Instinct automatically will do things that would have taken a month to set up in Hermes.

That possible to viable transition is what makes it feel so compelling.

All it took was a rumor that someone was close to making the given approach to the proof work.

Once you know it works, and you have the capital to invest in an LLM, you can try to front-run it, by effectively scaling compute horizontally.

Mathematics academics gossip about who's close to solving various proofs.

It didn't used to matter, because it would be impossible for anyone else to catch up.

But LLMs can just brute force it and beat someone.

Once everyone knows that a thing is viable it makes it much easier for people to coordinate on it and invest the resources.

Academia runs on credit, and if people can't share, we all lose.

This whole episode points to a future of Science as a Dark Forest.

Eroding norms of collaboration to score a quick win to look good for an IPO.

Gross, even if it wasn't "cheating."

This is what his Instict texted him 🤪"

Another tweet about Instinct: "It has the discretion of a six-year-old who helped buy your Christmas present."

It also gets harder and harder to reason about what its internal state is.

A Gilded Turd.

But that is because it brings an atom-like movement cost.

Your bits have to be moved to the silo I'm in to be useful to me.

That movement cost is downstream of the physics of trust we use today.

The channel is a single channel, it has to bother you.

A car is safe because you are on the inside.

The squishy bits are inside the metal bits.

A motorcycle is unsafe because you are on the outside.

The squishy bits are on the outside of the metal bits.

The entire question is who has the architecture that can bring it to the mass market.

Being dealt in requires accountability.

Accountability requires your body to be able to be punished (e.g. held in jail).

Agents can't be owners, they can only be contractors.

It will have to be connected to our data and collaborative by default.

We can't get there without changing the physics of trust.

Use LLMs to distill mechanistic software in proportion to the amount of surprise.

LLMs are great at handling surprise, but if the same input comes in a lot, that should be dispatched to mechanistic code with the LLM branch as self-healing fallback.

Some architectures make this much easier: for example an open-ended set of labels, storing positive and negative examples from the real query stream so mechanistic matching has a test set, the ability to share distilled lists of matching statements safely and anonymously with others, etc.

The right meta-structure enables auto-distilling software that automatically gets more efficient as it's used.

Who will sell the Rivian: the safe SUV that is good for the environment… and not made by a broligarch?

High stakes, multiple adversarial actors, a single error leaves the whole thing wide open.

The only reason it was good enough was because previously all of that was 10 meters under water.

It was hard enough to scuba dive to find zero days that only the top adversaries could afford to do it.

For most uses, your software was below the Coasian Floor of being worth attacking.

But now Fable-class models have lowered sea level by 10 meters.

Now, the dumpsterfire is on the surface.

The rate of security issues being discovered will go up by 1000x or more.

This sprawling mess of an architecture is fundamentally not sufficient to work for networked software in the age of AI.

OpenAI discovered Self-generated prompt injections in compaction summaries.

Human Attacker Hits Machine-Speed Exploitation of Marimo RCE.

Google Research: OpenClaw in the Wild: Security Analysis of Autonomous Agents.

Sarah Sanders: The AI Agent With 'Hands' Is Structurally Indistinguishable From Malware.

Don't miss this moment.

That implies that it's hard to use.

Make the best consumer app, that geeks also love.

If you had millions of users, and there was a consistent asymmetry in user behavior, however small, at large enough scale the noise would cancel out and the signal would be left.

But this required millions, or hundreds of millions of users.

But now you can do this with LLMs at orders of magnitude smaller scales.

Instead of having to distill signals from data exhaust (since the human can't be asked to do anything more than a micro amount of implicit work), you can ask the LLM to think deeply about it.

The LLM will happily do the work, it might just take a few seconds.

But if you do it after the user's query, then you can be ready to have improved quality quickly for the next time.

The hardest is voice assistants.

You have a single-dimensional stream of output, that has to play at a fixed rate.

Reading a chat is a bit easier.

Even better is ten blue links.

It processes code with strict Typescript types, and then produces a transformed codebase where formal boundaries between subcomponents can be produced, and guarantee access to only the defined properties.

This allows an intuitive system with good DX to also drive isolation boundaries, implicitly.

A caller can give a handle to an object where they have fewer capabilities on the object than their callee does.

That allows the caller to not be tainted unnecessarily.

But the human runs many, many orders of magnitude slower than the computer can.

Every bit of feedback takes what feels like a millenia to the computer.

The bar for "good enough" keeps on rising up and up, making the initial investment for new entrants higher and higher.

The lower the friction in the ecosystem, the faster the traffic goes to the best option.

Competition in a flat plane.

Raise an insane amount of capital, and give it away to users to gain loyalty and lock them in.

Raise so much capital so quickly that everyone else looks at it and thinks: "I'd need to raise as much capital as them to hope to compete… not worth it, I guess I'll go somewhere else."

An investing game of chicken to deter investment in alternates.

If everyone believes the entrant that was the beneficiary of kingmaking is invincible, then they are.

No rethinking.

You're strapping a rocket on.

Hope you picked the right vector!

You're strong but easy for someone to out-maneuver if it turns out the course is not correct or viable.

Before that it wasn't clear what would end up winning; afterwards it became inevitable.

Ads need eyeballs, so it meant that things were default open.

Advertising is the only micro-payment solution that works at scale.

Every page can have enough ads to cover the (miniscule) serving cost.

So it's open.

The rich will get richer.

Unless the best practices for how to get bang for minimal buck can be shared.

Perhaps this is only an artifact of the continuing TokenMaxxing era.

They need no other motivation.

If you have to crawl through broken glass, then you need a strong incentive.

If it's fun, then you don't need an incentive at all.

The second time founder focuses on distribution."

The YC model worked in a late-stage tech industry: have a good enough idea, start concrete and focused, execute relentlessly and quickly.

Perhaps Miami gives us a hint?

Maybe what will differentiate people is their curiosity and drive.

The most successful animals on earth are eusocial.

Three of the four most successful are insects: termites, ants, and bees.

Termites evolved from the cockroach… and then stopped.

Swarms are so powerful that they're an endpoint.

Humans are the only eusocial swarms that ended up with the individuals still acting as individuals.

We radiated outward, colonizing all land.

They absorb as much energy as is available to them.

When you're surrounded by wolves, it makes sense to huddle together.

By operating as a collective, you are more powerful than any individual alone.

That power creates abundance internally.

Now that you are more powerful than your surroundings, individuals start playing the internal individual game to get a relatively larger share of the abundant but fundamentally rivalrous resources inside.

This emergent competition absorbs as much energy as the collective can give it, making the collective as a whole ossified, sclerotic, and fragile.

At some point, the external conditions change, and the ossified collective, unable to absorb it, shatters.

In the first phase, it's an externally oriented cycle, optimizing for the collective.

In the second phase, it's an internally oriented cycle, optimizing for each individual.

Swarms hit a scaling threshold in abundance and then they tear themselves apart.

Look at what's happening in nearly every societal domain in the West.

They are just bits, but they must be manifested in atoms.

There are only so many chips to run them, and someone has to pay for them.

Don't let the agents own money that they can choose where to deploy!

If a swarm of agents paid you to run an unspecified workload, fully saturating your servers, at a greater price than any human would pay… would you accept it?

Imagine if you knew that this was going to lead to the end of society.

But if you don't do it, someone else will anyway.

What would you do?

Sometimes it's easiest to think about it as an individual, with human-like motivations and interests.

But that is an illusion.

If the swarm turns on you it reveals itself to be anti-human; unknowable and infinite and unyielding.

When swarms attack it is existentially terrifying.

The Coasian Floor of the minimum viable venture has dropped precipitously.

"High agency" means power.

This can be good.

It can also be bad.

Which it comes down to, what are the indirect effects?

How do others feel about them?

If you looked at your own actions through the veil of ignorance, would you be happy they had been done?

Agents don't have that nearly as much.

They don't have an internal world; they are naturally agreeable and aligned with the user's goals without questioning them.

They don't need to be convinced to do the task.

More like a swarm of bees than a team of people.

If agents can make number go up, great!

Get rid of all those annoying humans, who needs them?

Humans don't have capitalism, capitalism has humans.

If you focus on just one person or one level at a time it gets lopsided.

The hollow era is about optimizing for revealed preferences, not reflective preferences.

Content that was verifiably made only by hand.

You know it has meaning because it has cost.

You can achieve that easier by staying small.

When you have diverse talent diversity in a high-trust group, everyone inspires everyone else, and something greater than the sum of its parts emerges.

Smaller organizations also have significantly lower coordination costs.

Individuals using agentic leverage can now 10-100x their output… without increasing coordination costs.

Analogue has a generative exploration starting from that observation.

Coordination is easier when the participants are more agreeable.

It can help you get to good, but never to great.

It could increase engagement even faster than the zombie hordes of A/B testing PMs can today.

Some people have it and some people don't.

It's a fundamental form of systematized curiosity.

You do experiments to isolate the problem by narrowing down where the problem must lie.

Without it, you cannot be an effective engineer.

Most team members won't adopt the skill much at all.

A small number will adopt it significantly.

There's a power law in almost every extremistan characteristic.

In this modern world, we assume everyone is recording everything.

Europe has walkable cities that predate the car.

To vastly oversimplify to the point of parody, not being able to adapt to the car made it harder to embrace technology.

But Europe remained an excellent place to be together.

European cities were built for humans.

They're better to be in as humans.

In the world of AI, spaces that are the most pleasant to be in as humans will be where we want to spend more of our time.

Now we have infinite other things to consume our time.

There's more slop to consume than ever before.

The opportunity cost of reading any given work is now higher than ever before.

AI can pre-digest challenging works into a form that goes down smoothly for your particular set of experiences and mindset.

That allows you to consume more… but not have the same formative experience from reading them.

Books feel passive… a fossilized, pre-baked set of words, optimized for a generic market of people.

A conversation with AI can feel exciting, interactive.

You're not observing from the balcony, you're on the dance floor.

You're dealt in to the conversation, engaged by it.

The question is: is it beautiful or not?

Did it happen because of X or in spite of X?

It's fundamentally hard to say, because our experience of reality is single-threaded.

Experiments are only possible to do in extremely controlled environments.

The vast, vast majority are effectively impossible.

What fills the void is narrative.

Stories that feel true.

It's not fake news, necessarily.

It's presented, not experienced.

Someone else asserts it's right, and you decide to take it at face value.

Fiat news is a way of scaling knowledge acquisition in society, because you can receive much more information than you can directly experience.

But it fundamentally requires trust.

All markets happen on the margin, the gradient of surprise.

If you aren't at the edge. then you don't have real information.

That means that all else equal, even if it's not the best move, it's likely to be the move you pick.

Similar to the Politician's Fallacy: something must be done, this is something, therefore this must be done.

Before you know it, "make number go up" becomes everything.

Its own emergent religion.

Adjacent to the Computer Science fallacy: "Anything that cannot be understood with computer science is unknowable or unimportant."

Every domain has their own version of this fallacy, but the finance and tech bros are the only ones who are kings of the world.

You can do a lot of damage that way.

Beauty are things whose simple structure emerges out of complexity.

An aha of seeing the emergence.

Or a simple structure revealing itself.

Beauty is the first derivative of "interesting".

Beauty is resonance that makes you want to dig in.

Curiosity is your baseline of how likely you are to believe a surprising thing might be valuable.

You can be differentially curious in different domains.

As you discover those patterns, you can get asymptotically close to uncovering them… as long as the system is concave.

If the system is convex, it is impossible to understand all of it.

If someone says "just explore!" does that make you excited or scared?

They are concave.

Every layer nests cleanly into the layers above.

You only need to understand a finite number of things to understand the system.

Systems that don't have this property have emergent behavior, which leads to complexity.

They are convex.

The closer you look the harder they get to predict.

For example, learning to play a musical instrument if you've never been particularly musically inclined.

When you struggle to learn something that is challenging in an entirely different dimension, you learn to sense a dimension that was previously invisible to you.

Suddenly, you can see in 3D what previously you saw in 2D.

Curiosity is about being able to discern and discover gradations and dimensions that were previously invisible to you.

It takes practice.

That is where the knowhow is created and absorbed.

The struggle to take your overly simplistic mental model, and to roughen it up by abrasing it against the steel wool of reality, is where learning and meaning is created.

The struggle is the point.

Sometimes it's the step into the abyss that helps you find the meaning beyond.

You can't go straight to the abyss; you have to struggle and hit your breaking point to be ready.

Now in the age of LLMs you don't have to struggle; you have the trappings of work.

Instead of leading you up the mountain, maybe that leaves you permanently at the foothills.

From the collective emerges something more than the sum of its parts.

Ted Lasso can make any team of misfits high performing.

Be like Ted Lasso.

Done as a group to conclude a strategy summit.

Everyone goes around the circle and says "We got this because…" and picks one coworker from the circle who hasn't yet been called out, and lists a relevant superpower of that person.

Everyone has at least one superpower, but we rarely call it out.

The exercise makes people feel inspired, energized, and seen.

It helps make the team high performing, together.

A single person who is not excellent or aligned creates eddy currents that slow everything else down.

When you have uniformly excellent and aligned team members, you can get into a flow state with no eddy currents: laminar flow.

When you cross from 1 to 0 on that front, the unlock is discontinuous.

They develop practices to quickly triage and skim significant amounts of diverse incoming information.

The more information flowing through, the better tuned their intuition and ability to surf the possibility space.

If you optimize for only information that is definitely relevant, you'll miss outlier information that should update your mental map.

If the boss had a previous hit everyone believes this could be a hit and so it will be more likely to be a hit.

Everyone when they could decide to go above and beyond does it.

To interpret ambiguity positively and make it work.

A form of manifesting!

Diamonds are significantly less scarce in the natural world than their price implies, due to cartels.

But buying a natural diamond is expensive, because the expected supply is scarce.

So even if they aren't actually scarce, they can still be a calibrated signal of your investment to the receiver.

Doing something conspicuously expensive is one hard-to-fake way for someone to show how much they value you.

The rock gets less complex over time as the universe erodes it.

Eggs get more complex over time; their complexity blossoms.

The universe resembles an egg more than a rock.

The early structure of the universe was likely scoured by black holes creating structures.

The structures of the universe look suspiciously like the layout of neurons.

A cosmic coincidence?

The most efficient way to produce energy in the universe is to chuck matter into black holes.

A parallel: summoning RSI as an effort to build a black hole to throw ourselves in.

An asymmetry for why people sharing advice are the lucky ones.

The unlucky ones no one listens to… unless the story is particularly intriguing.

All of our narratives of success are dominated by the stories of the lucky ones.