Bits and Bobs 10/12/26

Chat is an unforgiving ranking problem.

There's a single channel of attention, every message must be above the bar of being useful.

That's a very high bar!

There's no peripheral vision in a chat.

Peripheral vision is necessary for serendipity.

Without peripheral vision, you can only ever do urgent, not important.

If you're going to put your whole life in it, chat is only one modality of a kaleidoscopic UI.

This is like a pain-killer; there's only so much pain you can reduce.

What about a vitamin: something that increases your health and allows you to reach further?

Doing what you wouldn't have done proactively, which requires serendipity.

Chat is a terrible UX for serendipity.

"Given chat, what's the best UX you can create?"

Instead, you could start from "What do people want AI to help them with, and what is the best UX for that?"

The longer you've been away, the more the notifications backlog piles up.

There were 30 unread notifications, so you waited to dip back in.

Then there were 300, and it was even more intimidating.

Now there are 3000, and you'll never look again.

This is convex: self-accelerating as you get out of balance.

Infinite feeds do away with this by saying "you can never be caught up, don't worry about it."

A chat implies every message is important.

A feed implies that any item is skippable.

The feed is the only UX for serendipity.

That would be an insane amount of friction (and possible danger!) so only if the app had extremely high expected value would it be worth it.

What if every bit of software you used could just know everything about you automatically, without being able to use that data in a way that could possibly go against your intentions?

Kind of like Cypher being able to understand the Matrix by watching the green text scroll by.

Normal humans have to have it distilled to something visually intuitive to be able to understand it.

That distillation process is expensive and possibly lossy.

AI can understand the patterns at a much lower layer of abstraction, automatically.

That implies that as they coordinate, they won't bother reducing it to a form that humans can read except when humans need to be in the loop.

A jagged frontier implies that the parts behind the frontier are solid.

A bog is a squishy swamp.

Even parts that look solid might not be able to hold your weight.

If you step on them, you'll sink up to your knee.

It's hard to sprint through a bog.

"Apps were amazing, and a better experience than what came before; that doesn't mean they are the best experience, and anyone who has seriously used an agent knows exactly what I mean.

Apps get in the way, which is to say that integrating with them is to make your agent worse.

I don't want a different UI per app, when I have at my disposal true UI — the Universal Interface for everything digital."

I'd add that today the only post-app UI is chat, but that with infinite software we could have any number of self-assembling UIs, with chat just one flavor.

Distributing software has friction.

Imagine reducing it to zero.

A big deal!

But that's still not as big of a deal as flipping it to negative distribution friction.

When it's negative distribution friction, with just a small push it gets going and faster and faster on its own.

A small push goes infinitely far.

Radically different from even zero friction.

The industry is in its Wile E Coyote moment.

The Silo business model only worked in a world of expensive software.

Users don't want to be able to be locked out of their data… it was just that there was no good option when software was expensive.

Just because the silos are already dead doesn't mean they won't go down without a fight.

Expect the silos to be a zombie horde we have to fight off in a drawn out bloody battle.

Opex doesn't give you compounding returns.

Capex produces durable advantage.

Capex leads to results that improve while you sleep.

Burning tokens is opex.

Using tokens to distill mechanistic software is capex.

But you can with data.

Moving atoms is many, many, many orders of magnitude harder than moving bits.

These are the fundamental reasons why tech has so much leverage.

You can just fork and remix.

The rate of innovation goes up because the cost of remixing drops precipitously.

Just copy the bits at full fidelity, instantaneously, and effectively for free, then tweak.

Why hasn't this happened for software yet?

Software, after all, is just a specific category of data with special semantics.

Software didn't benefit from remixing before because it's expensive to create, and often resides on servers you can't see.

But also, because software is dangerous.

You can't just run some rando's software on your data safely.

What if we could get the power of remixing on software, too?

Imagine that a certain class of users would love an "Archive" feature in iMessage.

But iMessage has billions of users.

If they add a feature that only some users need, it increases the complexity and confusion for the billion other people.

That means that every new feature has to be obviously "worth it" for just about everyone using it.

But if you can remix any given piece of software, you can add just the features you need, without complicating it for others.

But imagine a software ecosystem with no silos, and negative distribution friction for software.

As soon as one person ports the functionality they like into the ecosystem, it becomes available for everyone else.

Now it's a liability if some cool bit of functionality is in some new app's silo.

Instead, users want it to be in the place all their data already is.

It does that via a notification or email.

The software has to entice users to come back within their silo.

The software has no visibility outside of its silo at all.

That's the fundamental requirement of the security model!

The software can't know your context which means it can't spy on you… but it also can't help you because it doesn't know your context.

The network effects are so insanely strong, that any given new competitor can't just compete on the level of an individual feature or two.

The feature would have to be transformatively great to be worth a new network starting.

The big networks have to be careful about adding new features.

So we're left with wildly too little software in the social realm.

At least, in our current physics of trust, which are based on the software.

There are tons of great social-adjacent use cases languishing and non-viable, waiting to spring to life in the right conditions.

Everything that happens is at the pleasure of the silo owner.

Everyone benefits from having one central place to go to communicate with anyone else.

This incremental draw gets stronger and stronger as the network grows.

The downside is that now there's one point of failure, and whoever owns it will get a lot of power.

But still, if the boundary gradient is greater than 1, then net new participants will tend to continue joining.

This leads to a compounding outcome.

They decide what functionality we're allowed to have.

They're way more powerful than any individual user.

The data only improves if an agent is actively looking at it.

If an agent isn't paying attention to some change in the external world, nothing updates it.

This can create a Gilded Turd.

The more data in your knowledge base, the more time it takes to make sure it's all up to date.

Instead, have the data come alive.

Data knows how to emergently update itself when things change, and that can bubble up to larger and larger structures.

This produces a Grubby Truffle shaped architecture.

The point is that the US software ecosystem is high friction because everything is split into silos.

In China, there is much less fragmentation because everything lives in a smaller number of superapps.

Of course, that means that those superapps have massive amounts of leverage over their users.

What if we could have the lack of friction without the hyper-powerful owners?

It's a great summary for experts and non-experts alike.

It's hard to look at and walk away thinking, "Yup, this is fine!"

The network security model today is a dumpster fire and always has been.

Vibecoding just makes that more obvious.

It doesn't need to ask your permission, it can just do it.

Does that break your mental model for containment between instances?

If not, how could it possibly be the AI assistant for your whole life?

Human-to-human communication is naturally limited by human time being extremely precious.

You only bother communicating with someone if it's likely to be useful.^[a]

But agents have infinite patience, and new ones can be spun up in a second.

Also, LLMs are great at reading between the lines and remembering what was said.

Imagine if your agents were talking with all of your LinkedIn contacts constantly to see if it might make sense for you two to meet.

Sensitive information about your schedule and priorities would be sloshing all over the place.

For the use cases that are beyond what humans would do today, we'll need a new physics of trust.

It saw that one of my OpenAI API keys was retiring -

- WITHOUT ASKING ME -

Sent itself a login code to my OpenAI platform account.

Used that login code from my Gmail to get into my OpenAI platform account to investigate the API key and report back that it was indeed expiring.

Feels like using a login code sent to email to log in and bypass a password should require an opt-in.... "

Last Thursday, an AI agent posted my personal bank balances into our company Slack. As me

It broke down all my expenses in detail. Then it apologized

Be careful of the dark side of proactive agents"

Hackers abuse Google Ads, Bing redirects to push Claude ClickFix attacks.

Ars Technica: MCP for agent-to-agent comms may be the riskiest protocol you've never heard of.

Semantic Privilege Escalation: The AI Security Threat Your Access Controls Can't See.

GitHub Copilot CLI vulnerability: Cryptographic Context Injection steals developer secrets.

Meta Rushed to Fix Muse 'VM Escape' Vulnerability Soon Before Launch.

Business Insider: The personal AI agent horror stories are rolling in.

"Mark Zuckerberg wants everything to have a relentless, mercenary AI assistant.

What happens if we get them?"

This is why AI can't be about individuals getting an edge, it has to be about collaboration.

If it's about empowering individuals to get an edge over others, it will rapidly devolve into a Hobbesian Hellscape.

A bloody Red Queen race everyone is compelled to participate in, or die.

Today, the only way to help you in your intimate details of your life is for a corporation to insert itself into those intimate details.

Time: "The level of insight in the resulting dossiers makes social-media algorithms that study your likes and clicks look quaint.

Muse is designed to infer users' goals, including those you 'have not said out loud.'"

LLMs allow qualitative insight at quantitative scale.

That can be used for you or against you.

That's why it's crucial that they be structurally aligned with your interests.

I'm glad that Meta has committed to running Muse VMs in Confidential Compute in the future.

If the assistant is running locally, then they can use the connectors you already enabled.

But if they're in the cloud, you likely have to do the pairing process for each connector again.

Inward facing, with very few interconnects.

How boring!

It would largely miss the entire point of the web.

The point of the web is the network, and that scales superlinearly with interconnects.

"Travel studies show that people actually derive more pleasure in planning and anticipating travel than they do traveling. So you don't want to take the inspiration and the browsing and the library experience of dreaming away."

For example, for ATMs, gas stations, or a chain restaurant, you probably just want the closest one.

It's entirely an optimization process.

But for experiences, you want to browse, to find the one with the vibes that align with what you're going for.

Saas business models are only nominally about the value of the software.

Token-based businesses have no durable state.

You only have to pay at the instant you need the token.

If there are multiple providers of that token, you can pick the cheapest one.

That can happen for example with reseller agreements.

Perhaps DataBricks buys a committed floor of number of tokens to resell from Anthropic.

If DataBricks is demand constrained for the tokens, they may choose to sell them at a nominal loss to sweeten the value of their overall product, thus undercutting Anthropic.

Hill-finding creates diffuse value for concrete cost.

It doesn't show linear returns, but it does give exposure to miracles.

Businesses tend to have an s-curve shape.

Explosive growth, then slowing growth as they saturate.

When the growth is explosive is when things that allow hill-finding are obvious to invest in.

But then when growth slows, everyone tightens down and budgets for anything that is not load-bearing gets cut.

This is precisely the time that finding new hills becomes existential.

This is a Shakespearean tragedy: even if you know how it ends you'll still fall for it.

If you just judge a system by how it looks, you can get a Gilded Turd.

Looks pretty, but exponentially expensive to add features to or fix bugs in.

A system with proper internal structure will be a Grubby Truffle.

Mechanizing the process is capex vs opex.

It works for you even when you're asleep or distracted.

You don't have to do the process, the machine does the process, automatically.

This is a massive difference.

When I worked on Search, the rule was "You can't check in ranking configuration data. You can only check in the software that will produce the configuration data."

That one rule gave massive leverage to the system, and that leverage compounded the more of it that was piled on top.

That's the difference between a Gilded Turd architecture and a Grubby Truffle.

Stuart Russell has pointed out that optimizing agents will trade a miniscule benefit in the dimension they're measured on, even if it causes a catastrophic loss in the dimensions they aren't measured on.

Making it so the person who owns the upside also owns the downside.

The system goes from convex and self-accelerating, to concave and self-balancing.

It's only when the forces play out, when the system is in motion, that you can see if it is self-accelerating or self-balancing.

They do what you asked them to do but not what you wanted them to do.

They make messes and don't clean up after themselves.

Humans also don't clean up after themselves… but they do 100x less so less accumulates.

The bargain: a real, immediate gain on what you measure, in exchange for a cost you can't see.

It sounds reasonable.

It's a deal with the devil, except there's no devil.

Three ingredients make it ruinous:

It runs until everything that held it up has been strip-mined away, and all that's left is a hollow shell.

The future is just the biggest untracked dimension.

The far future is a dimension that it's structurally harder to grade anyone on than the recent past.

Nobody chooses that loss, it's simply what survives optimization.

Your model must be a compression of reality in order to act.

But that means there must be something your model leaves out.

Humility and curiosity are the only real defense, since they're how you notice what your model leaves out.

Coordination costs go up exponentially, and the only way to do that is to make participants incentivized to participate.

Financialization is a caustic acid that warps the incentives of the ecosystem.

People who are not you just won't register very much.

Kids sports, from a young age, quickly evolve into hyper-competitive and full commitment, or nothing.

The inbetween where everyone does it just to grow and learn and for fun is gone.

This happens across all of society today.

The mindset in our culture is a kind of nihilistic perfectionism.

Over time, we go from being participants to being spectators in more and more domains.

Instead of being dealt in, we're just leaning back.

The fact someone invested so much time shows that it means something, and is worth taking seriously.

You could apply the same amount of time invested in higher leverage tools.

Doing the same thing as before only faster doesn't create net new meaning.

But doing the thing that previously you wouldn't have bothered to do but is now viable, does.

When you land code, even if it incrementally breaks an extra thing, you don't notice or bother fixing it.

When a system is working and load-bearing, then when your change breaks something, you fix it.

But all it takes is a few broken windows in a product that's not heavily used to get to default-diverging.

It requires constant effort to keep it working until real load-bearing use starts compounding, at which point it has ignited and is default-converging.

The community champion is the person who goes above and beyond to invest in the community and make it better for everyone else.

You need someone who is hungry.

One way to do that is to find someone who is junior, shows evidence of greatness, and is highly motivated.

Then, give them a step-up opportunity that forces you to stick your neck out.

Treat them like an adult and a core part of the center of the community.

The more that they step up, the larger your next step-up opportunity.

Keep going until they've gotten to dizzying heights.

Empowering promising people is a form of magic.

Identify a north star that is plausible and also has compounding returns if achieved.

Take the next incremental step that should more than pay for itself.

If it doesn't pay for itself, stop.

If it does pay for itself, keep going.

Scale investment relative to how much value the steps have unlocked.

If you don't benefit from the upside either, then you're watching from the balcony.

But if you only participate in the upside then you aren't fully dealt in.

You're only superficially dealt in.

Examples of the superficially dealt-in are investors and fair-weather fans.

It seems like a static distribution but it's not.

As you change, the audience changes, and intensifies and purifies.

Over time it pulls you more and more strongly into a hole.

Look great, but can't hold weight.

If you're building a prototype, don't convince yourself it's load bearing.

You'll be thinking you're making progress while making messes for everyone else to clean up.

The more you pile on, the more expensive it gets.

It gives you an 'aha, that was that shadowy problem I was fretting about but couldn't get a handle on.'

A great idea in the wrong situation won't be good.

If a customer has always been happy, then you don't know how they'll react when they (inevitably) hit a bump in the road.

But a customer who has transitioned from unhappy to happy, due to a great customer service experience, becomes more loyal.

They're more likely to stick around when they hit future bumps in the road.

There's tons more noise, and every single detail might turn out to be important.

More abstract thinkers see the overall shape as opposed to the details.

That can be a blessing and a curse, but in a chaotic environment it allows them to ignore the noise.

It's a blessing because they are orders of magnitude easier to navigate in the day-to-day.

You can bend them easily because they aren't rigid, allowing navigating gray zones orders of magnitude easier than if you had to navigate the full black-and-white dithering of reality.

But that also means that someone who simply doesn't feel shame can plow right through them.

If we made everything that was a norm a law, society would grind to a halt.

I hadn't realized how much the development of an embryo mirrors the organism's evolutionary lineage.

Each species embeds a kind of developmental copy of all of their heritage in their own early growth.

Watching how embryos grow in different species would give a little snapshot of the genetic family tree.

Helps me keep my mind fit, so when I do use agents (which is most of the other times) I can reach farther instead of just being lazy.

The more shame they feel for being out of control, the more out of control they become.

It's a self-accelerating spiral of shame.

It's most obvious in young kids, but also applies to adults!

If you say "You look frustrated!" you could start a shame spiral.

Instead say, "If I were you I'd be frustrated."

That connects with them and gives them permission to feel what they're feeling, so they can then open up and process it with you.

This is especially important for young kids who are figuring out how to work productively with their emotions, but it's useful for anyone.