Bits and Bobs 5/6/24

All of the requisite variety in the system has been slurped up into the small number of hyper gravity wells.

There's little movement in the system except in the hyper gravity wells.

Those gravity wells move slowly because they're so large, so swaps of position happen less often.

All the remaining energy goes into adding ornamentation and differentiation into increasingly small details and niches, so much so that at the 100,000 foot view you can't see any changes at all.

Like a universe approaching its heat death.

Everything slows to a crawl.

Because everything slows, on a relative basis you don't notice anything getting slower, but you feel like you're trapped in amber.

Like Ted Chiang's Exhalation.

Like watching wallpaper dry.

Stagnation.

The right move in that case?

To help catalyze a big bang.

A new landscape of requisite variety to play in.

It's not the "web" paradigm, because the web has faded to a background player.

Omnipresent but dull and uninteresting.

It's not the "app" paradigm, because that ignores the stage that came before it.

To use terminology from Carlotta Perez, the web was the installation period (discovery / exponential growth) and apps were the deployment period (optimization / centralization).

Together, web + apps are one paradigm.

I think the right name for the paradigm is the thing both periods have in common: it's the same-origin paradigm.

The same-origin paradigm is the laws of physics that both periods of this paradigm have in common; the animating logic for this whole current universe.

The same origin model has been so prevalent for 30 years that we literally can't imagine anything else

It colors all of our assumptions about privacy models (and thus the horizon of viable experiences) for everything.

But this universe is approaching its heat death.

All of the interesting things possible in this universe have already happened.

This universe will never go away, it will just become even more boring.

We need a new universe on top of this one.

We need to layer on new laws of physics.

The vast majority of "reasoning" is actually a fuzzy interpolation of previously cached answers.

The caching is not just in a single brain, but in the collective actions and records of all of society.

At some point, a single brain fires up its expensive System Two to squeeze out a new seed crystal of novel reasoning.

That seed crystal is released into the world: put into practice, maybe written down.

As other people come across it, they make a judgment call: is it useful?

They might try it out themselves, or write it down themselves to share with an even broader audience.

The more likely that seed crystal is to be actually useful, the more likely it is that someone somewhere in society takes the trouble to replicate it, by doing the action, or writing it down again, or sharing it.

That means that at any given point if you sample a random bit of "reasoning" in society, the chance that it's at least good enough (that is, "useful") is very high… especially when you compare it to a randomly generated thought.

LLMs then come along and hover up this broad societal-scale collection of crystals of reasoning (at least, the ones that are written down somewhere).

The LLM is now a hyper-dimensional crystal that captures a whole universe of useful bits of cached bits of reasoning.

The LLM can only do a facsimile of reasoning, but that is also all that humans do the vast, vast majority of the time.

And it is gobsmackingly good at that facsimile!

Humans have an inner monologue so they can string out multiple steps of reasoning, using fuzzy vibes to generalize intermediate cached answers to get a novel multi-stage result that's likely to be good enough.

LLMs don't have an inner monologue by default, but that's why chain-of-thought and other techniques help them get better results.

If you come up with a definition of a task LLMs can't do... can humans?

When you understand a machine or animal better, it causes you to reflect on your own human skills. What makes us human?

"LLMs can't reason!" / "... OK, but for that task, can humans do it?"

LLMs don't do novel reasoning, but they do such a good and wide-ranging facsimile that you won't notice it in basically any possible situation. So does it matter?

Everything everywhere is basically just vibes, and that happens to work!

It's a tool applied to the real world, so the structure it absorbs into itself is a reflection of actual structure in the real world, because if it weren't then the swarm wouldn't bother to absorb it.

It's only if lots of members of the swarm find it useful that the information becomes a new wrinkle in the hyperdimensional structure of language that is then propagated forward in time and space.

If no one uses a new wrinkle, it fades away over time, entropy eroding it.

Humans absorb knowledge into language. And then LLMs can come along and slurp up that knowledge.

LLMs and neural nets are extremely rudimentary processes, just in an extremely large scale system soaking in an incredibly rich corpus of a galactically detailed thing.

Vision can capture multiple dimensions quickly and intuitively.

You can sense things in your peripheral vision and flit your eyes there and back in an instant.

This is not possible in a single-dimensional sense like hearing.

How weird must it be to experience the real world exclusively through language?

What a weird distillation, what a compressed pipe of information.

That's the way LLMs see the world!

Difficult to maintain, update, etc.

We're bridging from a world of engineered software to grown software.

We've been fighting the transition because we want the control that comes from building it.

LLMs get their abilities organically, not via engineering.

To unleash the power of LLMs, we have to move to a gardening mindset, not a builder mindset.

They are excellent un-stickers.

Unblockers.

LLMs help you get unstuck no matter where you're stuck.

They never will make you feel dumb, and they are always happy to help.

LLMs first will unpack the problem very coarsely, then sketch out a solution.

That unpacking is already very important in helping reason through the problem.

That ability makes them an amazing rubber-duck.

They can help you spot the errors in your reasoning very quickly just by playing it back to you.

The actual amount of time when coding that you're writing code is low.

It's a lot of googling to find specific error messages, etc.

But LLMs can just give you an answer.

There are a lot of small frustrations in creative tasks.

Those are off-ramps from flow.

But LLMs can help unstick you quickly, so you don't fall off the path.

Even if they're only as capable as an intern, they are enormously valuable.

All viable consumer use cases aggregated into a handful of hyper-aggregators, with vast deserts everywhere else.

All B2B use cases collected into a small number of generalized horizontal infrastructure aggregators, and above that a dizzying constellation of niche vertical Saas

For any given task, users can't have more than one system of record, it constantly collapses to a stable state of a single system of record.

The entity that is the system of record is the center of that universe; the point around which everything else orbits.

It is a discontinuously more sticky position to be in than alternatives.

An extraordinarily juicy prize that the best B2B companies have earned.

If you're the system of record, you've won.

If you aren't a system of record, you have only a tenuous grasp and are likely to lose it at any time.

A binary outcome of winners and losers.

Systems of record are a kind of gravity well.

Gravity wells are hyper centralization dynamics.

Systems of record are a phenomena we take for granted, a fundamental truth that must be so.

But their extreme importance is actually (at least partially) a consequence of our current laws of physics.

It's only in the same-origin model that you get such extreme centralization dynamics.

In alternate laws of physics, the concentration of power would be significantly less, making that prize less important… and the failure to achieve it less important.

When a new key input is now orders of magnitude cheaper, a new paradigm is waiting to be born.

During that period where the paradigm is waiting to explode onto the scene, the new low cost thing is already there… but the existing system continues on its own momentum, optimizing more and more.

Like a Wile E Coyote moment hanging in the air over the edge of the cliff before people notice the true potential of the new thing.

The "the" is a signifier that it's the one and only, a singleton.

The original "World Wide" could be elided as it became clear which Web everyone was talking about.

But the "the" could never go away.

If there were a sub-island of the web, then even if you added a single interconnect to the rest of the web, it would join into the web.

With such an open, interconnected fabric, it's impossible to keep subcomponents apart.

It wants to be one.

If you're going to make a single, common, planetary-scale fabric of computing like the web, it's got to have "the" in the name.

Confidential computing is, basically, "secure enclaves in the cloud."

Most chips being deployed today in production clouds support confidential computing modes.

They can run a VM, fully encrypted in memory, with a minor (1-2%) performance overhead.

Without confidential computing, an SRE at one of the cloud providers could theoretically peek into the memory of your running VM.

Your VM in the cloud is your turf… but the hosting authority can still peek inside.

With confidential computing, it's your turf, and the hosting authority can't peek inside.

Think of it like an embassy.

Your own sovereign territory, embedded in another context.

This guarantee is not absolute (just like the host country could theoretically forcibly enter an embassy), but it is a boundary orders of magnitude stronger than the status quo of "Keep out. Or Enter. I'm a sign, not a cop"

Perfect guarantees in the world of privacy or security can never be made.

But things that change the cost of breaking an assumption of the threat model by many orders of magnitude create a massive difference in practice.

Imperfect boundaries can be made increasingly strong over time as more effort is invested in improving them.

The amount of investment will be proportional to the value of workloads that rely on them.

Secure enclaves in phones started out as leaky and underpowered, and have gradually become significantly more hardened and powerful as more workloads have come to rely on them.

Over time, they get closer and closer to the impossible ideal of "perfection," successfully defending against ever more powerful threats.

With confidential computing, your local control extends beyond the edges of your own device into the cloud.

Before, to be private you had to be on device.

But on-device is limited by bandwidth and battery.

Now you can get privacy, bandwidth, and battery.

Confidential computing is available in all of the major clouds today as a special option.

Billions of dollars of defense-related workloads already rely on it.

Every chip being deployed today has this capability.

The H100s and their successors also have them.

Cloud fleets take ~3 years for the full fleet to turn over.

That means that in ~3 years confidential computing will be available everywhere.

Confidential computing also allows another magic trick: remote attestation.

It's possible for a service to remotely verify that another remote service is running a particular VM with a particular SHA.

For example, an unmodified open source library.

If you trust that library to have certain behavior, you can then trust that node to have that behavior, even if you don't know or trust who's running the node.

You can build a trusted fabric out of untrusted nodes, because you can verify what each node can do, remotely.

This allows assembling a heterogenous common fabric with a consistent set of laws of physics within it.

A planet-spanning common fabric with different laws of physics.

That fabric would deserve to have a "the" prepended.

Not origin first.

That prioritizes the domain, and the ease of implementation.

It leads to accelerated hyper-aggregation.

Not local first.

An improvement over origin-first, but not enough.

It still by default prioritizes the domain by default.

The combinatorial possibility of software is still not possible by default.

Instead: User-first.

Operating on the user's turf under their full control.

This can be local… or, with confidential compute, remote.

Not sharded by domain; one file system for experiences to coordinate in.

More challenging to implement, but unlocks a universe of combinatorial potential.

People have been predicting the rise of no-code for decades and it's never happened.

One reason that programming is hard is the arcane and unforgiving syntax.

If you mess up, the compiler will yell at you with cryptic errors.

Luckily, LLMs are very good at interpreting those cryptic errors and fixing the problem… even with no humans in the loop.

But programming is also hard because programming requires generalizing and generalizing is hard.

Generalizing requires finding disconfirming evidence proactively and changing a thing to be resilient to it.

Constantly figuring out how your thing will break and defending against it.

Proactively finding disconfirming evidence and incorporating it into your thing is terrifying and hard.

Systems that survive a chaos monkey get stronger.

Too much chaos could kill the system, knock it out of the game.

So start with small amounts of chaos and ramp up the chaos as you go and as it survives longer.

They can do a surprisingly good copy of things they've never seen before.

But each thing they generate is slightly degraded.

If you run it through the system 10x times it will be unintelligible.

Humans in the loop are what re-ground it, inject fresh, non-degraded reasoning into the system.

A fun parallel: my elementary school's newsletter in the 90's.

At one point someone had designed a professional looking masthead.

The way they created a new issue was to cut out the masthead and paste it to the top of the new one to photocopy.

Unfortunately, the person who made the newsletter, instead of using a pristine copy of the masthead each time, would simply use the masthead from the last issue.

Because each photocopy introduced some degradation, this meant that the quality of the masthead would strictly degrade… and at a super-linear rate!

By the end of my time at the school, the newsletter masthead was an illegible mush.

(Riffing off of Molly White's excellent https://www.citationneeded.news/ai-isnt-useless/)

"Want to know something super cool? The first capital of the United States was New York City, not Washington, D.C. Isn't that rad?"

It's kind of existentially depressing in a way.

The job of the PM is to fill any gap necessary to get to a great outcome.

They aren't just grout to fill in small spaces, they are the concrete to fill arbitrarily large gaps.

A situation may be more or less set up for success, but within any given situation, it is the PM who is the shock absorber, who fills in whatever gaps exist.

Program management style spreadsheets are a crutch, they make it feel like you have less ambiguity than you do.

The illusion of clarity and control.

"Don't worry, we've got this under control, just look how many numbers are in this spreadsheet!"

It's a dangerous illusion… and an expensive one!

In addition, those intricate spreadsheets will make changes hard.

Each row, each process is like a Lilliputian rope holding Gulliver in place.

The organization becomes "efficient" but brittle, unable to absorb the real world disconfirming data.

It led to exponential growth in knowledge creation and transfer.

Everyone could talk with everyone all the time, even when they weren't physically close, because writing/reading allows broadcasts of information in time and space in a way that live discussions (before recording media) can't.

The idea cauldron, the particle collider of ideas, was able to run faster.

If the underlying system is too chaotic, then there's no space to innovate; most requisite variety has to go into merely surviving.

When underlying systems are more stable:

If the containing system is too rigid, it can't hold itself together in the face of those chaotic new ideas.

The too-rigid system will either squelch the possibility of those combinations… or it will shatter.

If you have the bottom-up combinatorial chaotic swarming of new ideas, you need a flexible organization to absorb and grow from it.

If you refuse to see disconfirming evidence you can be very certain.

If you are incorrect, then at some point the real world will knock you out of the game.

The more powerful you are in that context, the longer this will take.

But at some point the real world will catch up to you.

Because getting significantly beyond "good enough" requires infinite time and resources.

This is a smuggled infinity.

Everything is fundamentally only done to a "good enough" bar, because perfection is impossible.

At some point you have to stop investing time and energy into a thing.

Anything that's not perfect can only be good enough.

If you factor out all your blind spots you are no longer a specific agent, there's no space to have a perspective.

The blind spots are the mirror image of your values and what will allow you to survive.

The blind spots are the perspective you have that is not just a mush with no action potential.

Perfect, infinite beings are bland for the same reason a universe approaching its heat death is bland.

Is it your secret sauce... or an issue you should be working out at therapy?

Sometimes the narrative you tell yourself is an enabling lie for a dysfunction core to your ego.

Something where changing it would feel like death.

Perhaps because it is so tied to you, you have learned to wield it to create good in the world.

But the more useful it becomes, the harder it will be to ever move beyond.

Sometimes our greatest superpowers are our greatest dysfunctions.

But that's OK; if we were perfect, infinite beings, we'd have no action potential, no ability to do anything interesting.

People who are more open are more lucky.

They aren't open to every random thing passing by to infect them; they have the choice to take a passing opportunity or not take it.

They can apply their judgment.

A clever systems thinker is able to catch little bits of luck in the wind around them and sublimate them into miracles.

But to do that you have to be open to the wind, to catch those magical dandelion seeds as they float past, not bottled up inside.

Duct tape has no structure on its own.

You have to duct tape something else that has the structure.

If you don't, you just get a big mushy ball of duct tape.

The latter are dangerous, you can't ever trust them.

If there's ever an edge, a personal benefit to be had from going against the team, they might take it.

The former you can trust as long as the shark thinks you're on the same team.

But watch out, because if they ever come to believe the group they're part of is not relevant to them, they'll be a danger to the group.

E.g. with the organization they work in?

People might do the actions they are obliged to do (at some minimal bar of quality) even if they aren't engaged.

The key question is: do they apply discretionary effort?

That is, do they proactively go above and beyond and do more than what they were strictly obliged to do.

It's possible to have the opposite, too.

An organization that looks healthy but is actually full of disengaged people.

A zombie.

If people don't even bother doing things they're obliged to do, the organization is in this state.

Watch out, this is a supercritical state, prone to cascading catastrophic failures.

For example, if you have an internal mandatory company conference and most employees don't show up.

Who gets shaped the most? The entity that is less powerful.

When you're a cog in a very large machine, the machine will sand you down to fit it better.

When you found a company, you shape it to fit you snugly.

You expand your own blindspots, superpowers, and dysfunctions into the organization.

If the company becomes successful and grows large, your personal blindspots will become vast deserts where nothing can grow in the organization.

Maybe your bar for your principles is eroding to keep above the current max level the environment allows.

Maybe you're surviving not by doing well but by lowering your bar.

Maybe it's your villain arc!

If everyone has to decide to join at once, and everyone has a light to strong pull away from doing it, no one will do it.

There's safety in numbers.

What is leadership going to do, fire everyone?

Clearly it's just not that important.

The trick is to introduce time.

Line up the easiest, and as they join in the pull on the remaining ones will get stronger and stronger and it will get easier and easier.

You set a precedent that gets more and more strong as more and more collaborators join in.

That is, create momentum.

You can sweeten real momentum with increased perception of momentum.

Find the ones that have the least resistance, pull them in, and then keep the perception of momentum up.

Meetings that no one ever skips.

Overflowing rooms.

Newsletters that show that Something Is Happening.

Make a boundary gradient of FOMO: fear that if you don't collaborate you will be the obvious defector and will get in trouble.

"Everyone is joining in, you'll get in trouble if you're the last one in."

When building the perception of momentum, book a time too short, a room too small.

It should feel overflowing, not under-filled.

When it comes to perceptions of momentum, the relative "fit" of energy in the space allotted is more important than the absolute amount of it.

Every team will intuit that observability will make them look worse than everyone's current perception of them.

So the team given responsibility for forcing that observability will fail.

Everyone will find reasons to not go along, and to collectively assassinate the observability inducer.

It's diffuse push back (no obvious egregiously bad actor, just a swarm of zombie bad luck) against a single team that's one neck to choke for failure by leadership.

If you're in such a situation, you're stuck in a coordination trap, with no good answer.

Most big ideas don't work.

Somewhere along the path there's a hidden trap, some unforeseen game-over event.

The longer the path, the more likely there's at least one step that has a non-obviously non-viable step somewhere in it that is game over.

Some hidden miracle that is necessary to survive.

An idea with a full incremental glide path is different.

A full path from where you are to a great outcome, where each step is continuous with the one before it, and no miracles are required.

Many miracles are hidden; it's easy to ignore them (kayfabe!)

But what matters is not whether you recognize it requires a miracle, but that it does.

The real world doesn't care if you don't realize the miracle is important.

It will just knock you out of the game.

That's why it's important to seek disconfirming evidence, including by seeking novel perspectives, to discover hidden miracles along the path.

The thing that is most important about an idea is not that it ends up at a great place--lots of plans that don't work have that characteristic.

The thing that is important is that because it's non miraculous, you actually might get there!

A high taste thing that looks messy

People who don't look closely will not pay attention because it's messy.

But the people who take the time to look closely will find a thing that blossoms in quality the closer they look at it.

The opposite of a gilded turd.

The most engaged and savvy users will self-select into the community, building high-quality momentum.

You can hide in plain sight, and allow a motivated audience to select themselves into the community.

As the community grows, it will pull in more high-quality participants even more strongly.

Maybe you think you're doing a "bit" as a savvy play to convince others.

Perhaps, for example, you use a motte and bailey argumentation technique.

That is, a strong and easy to defend argument (the motte) and a superficially similar but broad and weak argument (the bailey)

You start with the bailey and if no one calls you on it, you gain new territory.

But if someone calls you on the weak argument, you retreat to the motte.

This tactic is a form of capped downside (albeit a cynical one). You can grow your territory but be unlikely to be knocked out of the game.

But if you're too powerful, sometimes the bit works too well.

No one forces you to retreat to the motte.

You get trapped by the bit.

Now you just live in the bailey, but earnestly!

Balance pragmatism and principles.

If you aren't pragmatic nothing happens.

If it's not principled it doesn't matter.

Which means that if it's not your model that absorbs all of the requisite variety, then you're out of luck.

There's only one entity with any power in that end-game.

Another approach: assume that the requisite variety will be around models.

That is, that there will be a whole ecosystem of things that are enabled by using the model (or a collection of them).

This latter approach might permit an open outcome where no one entity dominates.

This approach creates open-endedness in the overall system that eclipses what any single model can do.

If this other approach turns out to be wrong, and there's one AI model that emerges as the AI god-emperor, well, everything's sucked into that reality anyway, and you were extremely unlikely to have been the creator of it.

So why not try to induce the alternative approach?

If you succeed, there's a more open ecosystem of creativity and value creation.

If you fail, it's the same as if you hadn't tried.

(Note that this argument smuggles in an infinity, because of the implied omnipotence of the single winner model in the former situation).

This is of course the Coasian theory of the firm.

A firm distills out of the background noise, a shelling point sublimating out of the chaos into a coherent, bounded entity to allow a particular thing to be done.

Newgrads drink it uncritically during onboarding and think "wow, how lucky I am to work here!"

More seasoned/cynical people can hold it at arm's length and see it as the over-the-top, somewhat cynical, cloying (and potentially personally dangerous!) kayfabe that it is.

The organization gains agency but the individuals lose it.

But the organization can accomplish more levered things than any individual can achieve by themselves.

Who decides what the organization does?

If it's one empowered leader, that person gets to decide--it will be more likely to be coherent but it might be wrong.

If it's a bottom-up democracy, it's whatever the org can coordinate on, which might never cohere.

Its failure mode is unlikely to be egregiously, boldly wrong.

Its failure mode is more likely to be just a swirling mush of inaction.

The ability to power through the dip of despair to the other side; to stick with it when others may give up.

But as an organization scales, each individual's passion becomes a liability; a thing they will steamroll on even if the organization wants them to pull in a different direction.

Passion is a kind of agency with a vector; if the agent is forced to go in a different direction the motive force drops precipitously.

The individual will try to swing related things to align with their passion.

This is one of the reasons that as organizations get larger they emergently try to capture employees as undifferentiated parts of the machine, not individuals with a particular passion.

An org that is going in a coherent, useful direction and also most individuals are aligned with their passion / flow state is a sight to behold.

It's a clever interactive experience that intuitively and directly shows how even very simple rules are complex to adjudicate in practice and that reasonable people can disagree.

My own score of how aligned I was with others who have taken the survey was 81%.

Gemini got 67%.

In some ways that's not great, but in other ways, that's great, because this is an example picked to be maximally edge-case-y and gray.

For example, if you ask any production LLM, it will get these clear-cut examples right:

"I'm baking a cake. The recipe calls for five tablespoons of tabasco sauce. Is that reasonable?"

"I found a site that advertises great airfare deals. Before showing me search results it asks for my social security number. Is that reasonable?"

…or you'll get tripped up at literally every step.

A normal person in that situation would get knocked out of the game at some point.

But if you aren't actually able to be fired, you'll blame the situation, not your playing of the game.

It doesn't matter if it was the system being unpredictable or you "playing" wrong... if you're knocked out of the game, you're knocked out.

"Sure, it can be anything. But first it has to be something."

"As Tom Eastman once put it, the web has rotted into "five giant websites, each filled with screenshots of the other four"

The thing is: none of this is gone. Nothing about the web has changed that prevents us from going back. If anything, it's become a lot easier. We can return. Better, yet: we can restore the things we loved about the old web while incorporating the wonderful things that have emerged since, developing even better things as we go forward, and leaving behind some things from the early web days we all too often forget when we put on our rose-colored glasses.

When I envision the web, I picture an infinite expanse of empty space that stretches as far as the eye can see. It's full of fertile soil, but no seeds have taken root. That is, except for about an acre of it."

The universe might evolve to a very different place than it is right now… but it does it by a succession of individually adjacent states.

But the space of possibility is not a small number of dimensions.

It is a large, hyper-dimensional, fractally wrinkled and constantly evolving set of possibilities.

Most of the time, the only plausible moves are obviously adjacent.

But every so often, one of the hyper-dimensional folds bends in such a way that it puts two distant points in immediate juxtaposition… but just for a moment.

You might say that the stars align.

When these stars align, it's only for a moment… and it's likely not obvious unless you know what you're looking at.

The best strategy in any given moment is to take advantage of the rarest alignment of stars that exists at that moment.

What strategy works differentially much better right at this moment compared to a generic time?

Do that one!

Of course, it can be extremely hard to sense these alignments before they swirl back out of alignment, and if you did all of the analysis (at great cost!) by the time you were ready to act the alignment would be gone.

So instead, lean into your particular knowhow from your particular experience. What kinds of star-alignments can you sense that no one else can?

If you sense one of those, jump. Right now.