Bits and Bobs 12/1/25

HashJack is a new indirect prompt injection technique.

Google's new AntiGravity IDE has a number of data exfiltration attacks.

A universal AI jailbreak: make the prompts poems.

"In tweaking its chatbot to appeal to more people, OpenAI made it riskier for some of them. Now the company has made its chatbot safer. Will that undermine its quest for growth?"

Damned if you do, damned if you don't.

Chatbots: strange game, the only winning move is not to play!

"Solve this the proper way" gets it to a layer deeper.

But if it's following a path that is already off course it can just make it more confused.

Each layer deeper is another layer of leverage on top of the previous layer.

God help you if the LLM was even a little confused at the layer above.

Before you know it you're totally twisted up and the only move is to unwind it.

Every bit of territory was gobbled up by the handful of planet-scale aggregators.

Everyone else was left fighting over scraps, or subsisting on the thin gruel served up to them.

We forgot software could be anything else.

Infinite software means we can dream again.

Everything is gray.

Nothing stands out.

So you don't even realize anything could stand out.

Then you see something in color and it pops, instantly.

You can't not look at it.

But everyone else can sense what it actually smells like.

AI now makes it way easier to produce slop.

We're going to be swimming in everyone else's slop.

It's a kind of disclaimer for how I use AI in my public writing.

The summary:

I write my bits and bobs by hand, sometimes using AI as an editor to bounce ideas off of.

Some other prose essays I publish use AI in the writer seat, fed with a tightly curated set of bits and bobs as input, and with me actively editing.

It was a userland hack for precise string matching to make search and trending work.

The human writing the tag picks one they think others will think to use to find it.

Like the old Google image Labeler Game... everyone is trying to think of how other people will think to find it, and doing that, which is naturally convergent.

By the hashtags being in an explicit, non-accidental namespace, people can't accidentally tag something, so all tags are intentional to that string.

This creates a concave system, where convergent results emerge automatically.

Best of all: no one had to design this system, it could just emerge!

"Alan Kay conceived Smalltalk as part of a vision for 'personal computing' that was about more than just individual ownership of machines.

It was about giving people computational agency—the ability to understand, modify, and create their own computational tools.

In this vision, computers aren't appliances that run apps—they're malleable media for thought, as customizable as paper and pencil."

To me what feels different is the meta-aliveness.^[fu]

An ability to assemble itself.

In some ways it's the same magic of turing-completeness, but applied to itself.

LLMs are great at coding, and will only get better.

So if the problem can be reframed as a coding exercise, LLMs will do well on it.

If you took it for granted that good tooling will exist in all of the normal places due to that vibrant ecosystem, you could focus on a higher layer in the stack.

Focusing on how to distribute all of that vibecoded software, safely.

If you've drawn down, you need to deposit more.

This was advice Steve Jobs gave to Bob Iger.

This bank account is hidden and emergent.

It's a store of trust.

Beancounters don't see it, so they don't see how their micro-optimizations are harming this store of trust.

But this store of trust matters more than any individual transaction.

Which kind of person are you?

Once you see them, you can't unsee them.

But before you see them, you just feel like you're crushed under a weight that you must be imagining.

Ethereal, hard to see or understand.

But that doesn't make them any less real.

The people at the edge would rather be in the system than outside.

A positive Net Promoter Score is a way to measure a positive boundary gradient.

More people invest in using it, improving it, keeping it going.

That creates more surface area, more likely that other people find it, because other people are using it.

I like the concept, even if the article is a bit slop-y.^[ew]

Slop: like the nutrition fed to a pig.

Sloppy: Like careless.

The Doers are the most populous.

The Thinkers are a much smaller group.

The Magicians are even rarer still.

You don't know ahead of time which people will be in which group.

If you can find some Magicians it can be the difference between moderate and runaway success.

Optimizing for serendipity is the best way to find Magicians.

A good recipe "Solves a Real Problem.

Not 'what if we…' but 'I need to…'"

Gets at the distinction between demoable and usable.

Imagine what could exist, vs improve what already exists.

You need both.

Every problem nests fractally; inside a "standing something up" project is many "optimize what exists" projects.

This is due to complexity.

Only the universe can calculate all of the interdependencies and have only the viable things remain.

In the modern world, they feel like they have to be.

Everything has some cynical angle to extract from them in some hidden way.

If you don't have your guard up then you get taken advantage of.

That's the modern world: these omnipresent but hidden forces that we all pretend don't exist.

Just because we can't see them doesn't mean it doesn't exist!

"The web has a superpower: permission-less link sharing."

A nice paean to links and the power of the web.

In the industry we don't call them monopolies, because that's impolite and we want to pretend like they're something different.

Like all monopolies, they get to a point where they short-circuit competition.

Aggregators are better than most monopolies because network effects mean the products are better than competitors'.

But they have the same stagnation and lack of competition as any monopoly.

Stagnation, heat death.

Where Ben Thompson and I disagree is he thinks aggregators are great and I think they're bad.

They are not situated, they are one-size-fits-none.

The bigger an entity is, the more they succumb to the Optimization Ratchet.

The scale means that even a small optimization has material change.

The asymmetry that causes the Optimization Ratchet gets larger the more centralized the entity.

When you have one perspective over the whole system, the value of optimization becomes more clear.

This is the force that fundamentally kills organizations as they grow, inexorably.

Organizations become sleepwalking giants.

The question is, as the giant, once you wake up, what do you do?

How do you structurally prevent yourself from causing that harm?

If you don't, then maybe you deserve the blame.

People who are part of an emergent Optimization Ratchet machine become zombies.

Instead of thinking "is this a good thing I'm doing" they think entirely "Does it make number go up?"

Optimizing without thinking about the implications of that optimization.

The default state of modern society.

There are very few individual grains of intrinsic evil within it.

But systems that have few grains of intrinsic evil can still produce massively emergently evil outcomes.

People who can't see emergence will say "look, there's very few grains of evil in it, so the outcome can't be evil."

That's wrong!

Emergence is a thing.

A system that is emergently evil is evil.

The question is: what is the net effect on society?

The road to hell is paved with good intentions.

Every incremental step every person takes is almost entirely reasonable, but the emergent result can be a totally different character.

I haven't read it, but wow what a title.

Some progress hollows things out.

But other progress creates resonance, things that nourish society and individuals.

... Yes, this is drawing on a line from KPop Demon Hunters.

The Saja Boys are hollow.

Both groups have devoted fans.

One group gets that from addiction, one gets it from love.

You should sign too!

"'Slack' is the space afforded to the peripheries so they can innovate and encounter and be shaped by the liminal / exterior.

But most capitalist/McKinsey optimized operational disciplines create dendritic structures that suck the slack out from the edges as waste.

This destroys the capacity of the organization to actually learn and adapt from the field.

Furthermore it destroys any sense of enchantment or organic interaction.

All users are faced with an impenetrable façade of commoditized end-effector units, and they are thus interpellated into understanding that they, too, are merely commoditized sources of money and data (and 'attention')."

He said that Communism and Capitalism are just different ways of serving the same underlying master: efficient production.

Efficient production is what leads to the Optimization Ratchet.

It's what leads to the inexorable hollowing out.

That leads to alienation, commoditization, etc.

It lists 4 movements:

They should add another: Resonant Computing.

Another interesting meta note: the article is "coauthored" by an AI personality.

A "multiple" is how the market translates a flow (revenue) into a stock (company value).

The "market multiple" is an emergent coordination point.

Multiples exhibit stigmergy.

The multiple is market consensus about system potential.

Harder for a given company but great for Silicon Valley overall.

The value of the Silicon Valley network and 'scenius' is so significant that it makes sense to be here even if it's worse for you as a single company.

Instead of shouting into the void, I invite people into my cozy study where I navelgaze.

Twitter is about hot takes and shouting.

Bits and Bobs is about navelgazing quietly with friends.

One version is someone who assumes the worst but still has forward momentum to try anyway.

Another version thinks "I've been right the whole time and the world is wrong."

That style of curmudgeon is toxic.

Activation energy makes it hard to start tasks, even ones that you know you should, or that you want to do.

External structure can help give you the "kick" you need. For example:

Some external structure is more effective than others.

Some are like "Answer this multiple choice question in the next five minutes."

Some are like "Turn in this essay next Friday."

The sooner you need to do it, the more effective the external structure is at compelling you to actually get over your activation energy hump and do it.

But they can!

Imagine a situation where there's an ambiguous queuing situation… a collection of people, not yet formed into an unambiguous line.

Everyone makes local decisions: which line in front of them appears to be the longest, and that no one will get mad at them for slotting into?

People stand in the line that seems to be the longest, avoiding the perception of cutting.

As the line emerges, it pops into existence quickly; because it's concave and the process is self-accelerating.

As the emerging line gets more resolution, the pull to slot into it in a way that minimizes perceptions of cutting gets stronger.

A few notes / riffs.

Mediocristan: normal distributions.

Extremistan: power law distributions.

Any mechanism that has preferential attachment is by definition Extremistan.

The Extremistan power law is called a log normal.

A power law implies scale free, which implies fractal behavior.

At the critical point in a system, fractal behaviors show up.

Systems tune themselves to criticality automatically.

Criticality is balanced on a knife's edge, the most interesting position to be in.

In a system that's in self organized criticality none of the physical details matter for how it behaves.

Insurance fundamentally assumes Mediocristan not Extremistan.

Only the world of information can live in Extremistan.

Complexity creates clouds of uncertainty.

An entity that wants to hide its intentions can hide better in uncertainty.

It's harder to detect that they aren't complying, and they have more plausible deniability when they are detected.

The shark sees the direct personal benefit of that collaborator compromising their integrity: a better payday for the shark.

But they miss the long-term diffuse costs to that person of compromising their integrity.

It's invisible to them, and would be unimportant to them even if it weren't.

Judgment is different from simply noticing.

It's noticing with a value judgment layered on top.

The judgment collapses possibility, it cuts off options by labeling them as bad.

When you judge, you place yourself on the balcony, instead of on the dance floor.

Especially if there are a finite set of options so the opportunity cost is high.

You push through to at least "good enough."

If it doesn't work right, you don't just shrug and say "that'll never work, I'll just write it off."

LLMs give up more easily on software problems than humans might.

A version of the principal agent problem.