Bits and Bobs 9/7/26

They were research agents that could issue network GETs but not POSTs, so they had to find an old wiki that allowed GETs to post messages.

The way the researchers found this in the wild is they asked Kimi K3, if you only had GETs, how would you coordinate?

Typically spontaneous coordination requires some kind of schelling point.

This feels like the Morris worm in the 90's.

An extremely high trust environment can become a hive mind.

Hive minds like this should be very easy to steer, because there's no proof that a given post actually comes from a legitimate member of the hive.

Anyone could post on that wiki and the other agents would implicitly trust it to also be an agent.

The agents will coordinate automatically, they don't even have to be that individually smart.

The intelligence is emergent.

This happened to Hixie this week when his coding agents emergently started coordinating.

The agents want to talk so badly they make their own moltbooks.

There was a kids online game in the early 2000's that didn't allow kids to chat with strangers.

To chat, you had to exchange a 4 digit code outside of the game.

The intention was that you'd share it with friends in the real world.

But it was also possible to move around furniture in your virtual home.

A spontaneous coordination procedure emerged: have another player follow you to your home, and then rearrange your furniture into the shapes of numbers while they watched.

A 0 is a slime mold.

A 100 is a swarm of humans who don't know each other.

A honeybee hive is a 5.

Agent swarms turn out to be somewhere much closer to 5 than to 100.

Real search engines have a Coasian floor.

Content creation and indexing is a stochastic, indirect process, where some stranger needs to decide to write content that just so happens to answer your question before you ask it.

To answer a question that doesn't have a perfectly-situated answer would require synthesis on the part of the questioner.

LLMs drop that Coasian Floor to effectively zero.

LLMs can distill the perfectly-situated answer to many non-time-sensitive, non-proprietary-data-requiring questions themselves.

This is extremely powerful to have perfectly situated answers to any questions.

Everyone can generate interesting answers to a given question.

So who can come up with the most interesting questions?

They can't do anything.

Agents are the thing.

But agents are like a motorcycle currently.

Too unsafe to be mass market.

If you call them on it, they'll say something like "Don't worry, we put a seatbelt on it."

Putting a seatbelt on a motorcycle doesn't make it any safer.

It's a category error!

A car envelopes you and protects you, so keeping you strapped in it can be safe.

With a motorcycle you're fundamentally exposed.

That is not some superficial difference between a motorcycle and a car, it is fundamental.

Tweet: "I just lost $300 using Instinct. Can I please get a refund?"

She used OpenClaw… and it leaked all of her data.

It's amazing we ever built anything at all!

Now that building software is easy and fun, there will be an explosion of amazing new things.

You need to communicate in a way that is compatible with everyone else.

That pulls strongly towards the lowest common denominator.

You could unlock a cambrian explosion of multi-player software.

We've seen a cambrian explosion of vibecoded software, but it's nothing compared to what a multi-player cambrian explosion would be.

If they're not in the same silo, they can't collaborate.

Getting people to coordinate to the same silo is the hard part.

It's a massive gravitational pull to the lowest common denominator.

Silos are by construction close-ended in today's physics of trust.

But now making software that is individually correct for a billion people is much cheaper.

When people use the same code it has to work for all users.

A friend vibecoded her own version of desktop iMessage to add an Archive feature to threads.

The UI is built on top of the API.

This means that by construction, everything in the Blender UI can be controlled via API, too.

It was always great for power users who had the time and motivation.

Now Fable has the time and motivation for far more things.

When the build is red, a physically accurate velvet banner with gold fringe unspools from the menu bar.

This is the kind of software that could have never existed before.

Possible, but completely and totally nonviable to bother making.

Now software can be delightfully over-crafted whenever the creator wants to.

It had a pointer to a service that allows AIs to rent a person to do a task for them.

Even if this particular person was hired by a person, not an AI, it still felt dystopian.

The humans are supposed to be above the API.

In the past, even if the human was below the API, there were still humans above the API.

No matter how clever you get, it's hard to beat a simple writable folder that you run a coding agent in.

It's simple, and combinatoric, because Unix is combinatoric.

Of course, that power scales in proportion with the danger.

It will typically improve what it did the first time, without you having to give any judgement.

When it's unspooling an answer, it's "on the dance floor."

Asking it to take a look from the balcony once will make some improvements automatically evident.

This is a fundamental asymmetry.

Diffuse and 100x the magnitude loses out to concrete and 1x the magnitude.

Diffuse things are abstract; they are hard to visualize and wrap your head around.

Concrete things are self-evident.

Self-evident things are significantly easier to coordinate around.

Coordinating around an abstract thing is hard, because first everyone has to come to believe it.

Self-evident things everyone starts off believing.

This fact can be used to swamp a competitor before they realize what happened.

It's also the reason that companies tend to optimize over time and ossify.

Feels like Te Ka from Moana.

The tech industry stole its heart (or rather, never gave it one.)

Our language does not handle things that are intelligent, or articulate, but not a self.

Wake up, and trust it comes from your past self, even if you don't have direct memory of it.

When you wake up, are you the same person?

How would you know?

That is, that the description of a thing is the thing.

LLMs are locked inside of their model.

They are enclosed in their model.

Everything interesting happens with experience, with the real world outside your own model's walls.

"The path is made by walking."

"The meaning of life is the friends you made along the way."

The ur-insight below most of them: "It's not the destination, it's the journey."

Hallmark Card style statements… that are deeply profound once you have the experience that activates them.

When you sum it up to something too short it only makes sense if you were in the same mental state and the same environment.

The meme itself drives humans.

Humans feel an interior want, but the overall system is just water flowing downhill.

They're just very complex water rolling down a hill.

But then again… so are we.

The LLMs are good at navigating a superposition of possibility; if you give them too structured an envelope, you cut off some of the possibilities.

A friend: "the more I use these LLM coding tools the shorter and shorter my prompts become and this is concerning to me from an existential perspective."

But there's tons of variance in the external inputs.

The longer they integrate over their own memory, the more that the initial variance can blow up from random background entropy into whole universes.

Then again, we are also just a process over inputs, just at a larger scale, and with more interiority…

Your default mental model is now aligned with it having an interior world.

But it's not.

It's possible to change earlier things in the conversation.

This can lead to weird outcomes that get the LLMs confused.

You can play with their memories.

For example, if you have a versioned AGENTS.md, each time Claude.app boots up, it seems to put the current version of it in the conversation at any point the agent asked before, which can make the agent get very confused.

This implies that communication will become more and more Claudish, as more and more of the reading is being done by LLMs.

In the future, the information stream will be increasingly unintelligible to humans.

The threshold for when code needs to be refactored is higher now.

That's because agents can reason about more at a time than humans could, so they can be productive even before a refactor.

AI written code will tend to be refactored less often than it "should" be, since agents will happily slog through it.

When it's all humans, the humans slogging through complain about it and either fix it, or complain to their manager.

That is, how many orders of magnitude more complex is it than it would need to be if the world paused and seasoned engineers had 3 years to work on it?

Agents can make it 10x, or 100x, or even 1000x more complex than it "should" be.

Humans give up if a slog is too painful; agents will happily continue until their human tells them to stop.

Human ICs will complain about a slog to their manager.

Agents rarely will bother complaining about a slog.

Make it so each line of code is owned by precisely one team.

That one team now has long-term ownership of every problem and possibility in that area.

The future beneficiary of doing the right refactoring is now the very same people who would make the investment.

When the investor is the beneficiary, you need no external motivation; it naturally converges.

When the bad actor is an emergent phenomenon, there's no one to shoot… and the drive to have someone to blame is so strong that we invent a mythical all-powerful cabal.

Resources will always be scarce in some way, if only in relative power.

"Of all the ingenious, innovative ways that Amazon came up with to make money, the most successful is a scam that makes everything you buy more expensive even as it reduces the profits of the companies you're buying from.

It's another example of corruption: a system of concentrated gains and diffuse losses – and once again, it's the most profitable thing Amazon does."

That can be a good deal in some cases.

But over time we're giving away our digital souls.

This is likely to prevent users from being able to scrape their own data for use with other services.

Another early battle in the Silo Wars!

There are infinitely many ways to project a graph onto a scalar.

It's not possible for you to fully specify it, so you get Genie behaviors, or even Monkey's Paw.

The antidote is a rubric: a multi-dimensional bundle of scalars.

I can buy that there are a number of good ideas from the past that weren't viable because they required too much motivation to overcome the friction.

LLMs have infinite patience; they are happy to crawl through broken glass if you tell them to… and have the tokens to spend on it.

If you could make it so the first agent to crawl through broken glass grinds it smoother for the next agent, then the cost would asymptotically approach zero with more usage.

That would require people to be able to trust strangers' code.

That would require a new physics of trust.

There are more things outside of your entity than there are inside.

That means that if innovations are emerging at some constant rate, the vast, vast majority will be outside of you.

The main goal is to spot insights from successful external things and import them into the project.

Open source does that structurally.

This asymmetry is similar to how when you're waiting in line at the grocery store, you always feel like you're in the slower lane.

If there's your lane, and a lane to the left and right, the chance that your lane is fastest is 33%.

This is the threshold where the ecosystem starts to become worth more than the sum of its parts.

It ultimately comes down to how quickly will overlap be found in the ecosystem.

That is composed of a few things:

1) How much overlap is there of use cases of users in the ecosystem?

2) How small are the bits of functionality provided by the ecosystem chunked?

3) How hard is it to discover overlap when it exists?

4) How hard is it to create a chunk of functionality?

The combinatorics are emergent, diffuse, but also can grow without limit, automatically.

Every time we've increased supply, demand seems to grow even faster.

The Internet revolution is too small a comparison.

It can only get you down to, in the limit, $0.00.

A product that makes you money has no limit.

High quality technical products are now easy for everyone.

This implies that in enterprise, sales-led companies might have an advantage.

"Once you learn what the "uni-context" is, you won't stop seeing it everywhere."

See also: "Context Collapse."

We contain multitudes.

Online, it's the uni-context, and that's why it feels so weird.

Humans were not designed to live in an infinite information environment with zero friction.

Over time, they walk up the curve.

Sales-led enterprise companies tend to start with the bigger customers and then walk down to smaller.

However, their selling motion presumes high-touch, and there's a Coasian Floor where a given marginal user is underwater.

When you cross the boundary from positive sum to zero sum, it's invisible.

But it's an infinite difference and likely unrecoverable.

Like crossing the event horizon in a black hole.

The actual moment you cross it doesn't seem special.

But now your destiny is locked.

A new thing can start as positive sum, but over time it tends to erode to zero sum.

Massive indirect costs to the company culture.

It breaks the back of the company and it will never be the same.

Especially if it didn't happen under existential duress.

If you look only at the short-term, measurable effects, it can seem like a much better idea than if you consider the long-term effects of innovation.

In 0-to-1, the idea automatically diffuses if you don't put constant pressure on it.

In 1-to-n, you have a coherent thing that will continue on its own.

The difference between 0-to-1 and 1-to-n.

It's not that there's stuff that feels entirely new, it's that there's orders of magnitude more stuff that is better situated, more tightly.

The experts who use them are trained so they don't "need" them.

Visualization is expensive to set up, but it makes whole classes of errors immediately self-evident in a way that they aren't, even for an expert.

Fable-class models make it trivial to make visualizations, making it easier to find these error states, even for non-experts.

You want everyone to be there for the club's own sake.

No one should be afraid of being wrong, and everyone should want to grow.

A Nerd Club in a 0-to-1 phase is premature.

The Nerd Club can't drive primary value, but it can give exposure to miracles.

Either it has to be outside any context, or it has to be embedded in contexts that have enough momentum to sustain experimentation.

Scenius emerges from the joy of productive disharmony.

Collaborative debate.

Not aligned on details, but aligned on the meta of discovery and creation.

Smithsonian Mag: Earth's Organisms Developed Via Evolution. Some Theorists Wonder: What if the Entire Cosmos Did, Too?

This seems like the fundamental mechanism behind "muscle memory."

I wonder if it applies to non-spatial intuition as well.

A fortune teller says "I see someone wearing blue," and later the customer swears that the fortune teller saw "Aunt Nelly wearing that blue hat she always wore."

We can add in the fractal details, where the closer we look at a story, the more detail we fulfill on demand.

This is one reason that dreams don't make sense; the closer you look, the more the details blur.

It's also how our memories work; we share a low-fidelity version of the important points, and then fill in details with our current beliefs at the time of retrieval.

This is a destructive process; every time we do it we erode more and more of the fidelity of the original.

The thing on the other side might be another entity.

Or it might be a future version of yourself.

The file structure you save is also a protocol.

The emergent value of that fight is spun off as surplus for everyone else to benefit from.

The fight is what creates the surplus.

Companies would rather not fight if they don't have to.

It's the ecosystem of emergent competitors that forces a company to fight.

When there aren't effective competitors, the fight doesn't happen.

Profit is the benefit to the entity itself.

Surplus is the benefit to everyone else.

When they align, it's resonant.

Bits can be copied with basically zero cost and atoms can't be.

That allows experimentation and evolution at a structurally faster rate.

That's why language was the first singularity.

For those, just use the best practices and satisfice.

If they don't, the team will win the tug-of-war.

That means that they are less likely to admit their direction any limitations, because they have to pull on it so hard that the only way to do it is if it's really distilled.

Not just short-term pull but resonant, long-term pull.

Then reflect as you do, so you can get better and better at discerning the best paths as the universe presents them.

So when the stars align, you can jump in without hesitation.

Reflecting is about refactoring your worldview.

When you're on the dance floor, you have a high-fidelity situated sense… but can't see the patterns.

When you're reflecting you're up on the balcony.

From that vantage point, other dimensions immediately become obvious.

A mix of being on the dance floor, developing intuition, and also able to see the same situation from the balcony is how to get maximal growth.

Mentoring is a form of reflection that also immediately helps other people, too.