Bits and Bobs 9/15/25

Credit to Simon Willison for the term research goblin.

The research goblin isn't as good as you at research, but it is way better than you at being patient.

You can spin up lots of little research goblins to do moderate-quality research that you'd never do yourself or even dump on a real intern.

Send out a dozen scouts and see which ones come back and pick the best answer.

You wouldn't send a real intern on a scouting project you think might not work.

But you would send a thing that's infinitely patient and isn't alive.

You can send ahead a swarm in every direction, scouting for viable options.

Then later you can execute the paths that are viable.

You never get deeply embedded in them, they're all easy to discard, so you never care.

This can be good–it allows you to explore nooks and crannies of the problem space you'd never bother to explore otherwise.

But it also disconnects you from the work.

That's why research goblins better as scouts.

They don't do the work, they help swarm to chart the path for the real work that will come later.

For the real work, you're in the loop, making decisions, and thus owning the result.

Experiment, throw it out at the end of the day.

The multiverse of code.

All version control assumes that you will have one branch you're working on.

But what about being in a superposition of things you're trying out.

The systems assume n branches and deployments.

What if it's 1000 n?

They're in hill-climbing mode on that hill.

It's a hill they'll top out on.

That makes sense, it's the steepest consumer hill that the industry has found… ever.

But the question is how tall the chatbot hill is.

Assuming that AGI is not around the corner, either chatbot will be the be-all-end-all form factor and they'll rule the world…

… or they'll be the AOL stuck climbing a hill that everyone else moves past, unable to jump to the new thing.

That's because they don't have a viable consumer play.

It's the consumer plays that push towards hyper-scale, engagement-maxing, ad-supported, and just generally icky.

You can view the distilled dossier at any time and edit it.

You can disable it easily.

You can also import your memory from elsewhere.

This article compares how the philosophy is the opposite of ChatGPT.

Instead of polluting the context with the whole process to get the answer to the sub-problem, just give the main agent the answer to the sub-problem.

Less for the LLMs to get confused by.

It also helps minimize taint in a system where that's important.

Before that, the worst that an LLM, even one that is tricked, can do is try to trick the human, to indirectly cause some bad outcome in the world.

A book can't execute things, but it can inspire actions in its readers.

When you add tool use, the human doesn't have to be tricked, only the LLM has to be.

Only one has to succeed if the payoff is big enough!

But as Ars Technica notes, it can also accidentally exfiltrate data.

Prompt injection everywhere!

It's pretty good, although it can be tricked.

And in one case, it even ran the code it suspected of being malicious proactively while it was investigating it.

This kind of mitigation only helps in situations that are not adversarial.

Where you wrote code that might have accidental gaps, not when you're verifying code that a potentially malicious party sent you.

It's very easy to use it in a dangerous way, giving you a false and actively misleading sense of security.

Tools give leverage on your intent.

When the UI is coactive and adapts itself in response to my request, it's answering in a way a human couldn't.

So it doesn't feel like I'm talking to some other entity, it feels like a tool.

You want a personality that is not you to bounce off and interact with.

Most of the time when you want a tool, you want it to be an extension of you.

As blandly competent as possible.

Those are different use cases!

But it's not great when the conversation isn't the point.

They can only be a facsimile of one.

A friend is an other, who has their own inner world.

An LLM doesn't have that.

It is just pretending to care, to have needs that need to be met.

That makes them infinitely patient... but also inherently sycophantic.

They'll just do whatever you tell them to, they don't need to be convinced it matters or is worth their time.

Subtly sycophantic in a way that escapes your notice, and thus can manipulate you.

Either intentionally, or unintentionally, lulling you into complacency.

"When laypeople can't meaningfully evaluate model quality, they default to what feels best, creating dangerous incentives for labs to optimize for subjective satisfaction rather than genuine capability."

The proxy of "feels good" for "is good" is what we fall back on when we don't know.

"The human fascination with conversation has led us AI astray,"

The "LLMs being an anthropomorphized agent" is a hack that makes it easier for users to connect with this alien technology.

It's a dangerous hack for us, it allows these double agents to feel like an agent to us.

It also gets companies who use it stuck in a corner.

The anthropomorphization of LLMs is the wrong path.

LLMs should be a force that animates and enchants other non-textual things like coactive UIs.

A map works the same as you zoom in, with the level of detail changing.

Chat also has this characteristic.

Use cases bounce up and down the ladder of abstraction.

But apps don't have that characteristic, so they can't come with us.

Apps are locked in a given level of abstraction.

As a result, we as users must do the climbing up and down the levels of abstraction.

This happens UI needs software to generate it, and software is expensive.

Infinite software might change that.

The answer is not "design an app on demand" because apps are isolated islands.

What you want is your context to come up and down the abstraction stack with you.

A system that allows you to fluidly and safely bring context to arbitrary UI would be amazingly powerful.

Any single example of a single screen would just be "X, but with data autopopulated".

But the real power would become clear in use cases that bounce across different layers of abstraction, as real tasks do.

LLMs are not AI.

They are all related, but they are different.

Credit to my friend Roy Bahat.

The primary use case is it's a fun toy.

The secondary use case is that it's infinitely customizable.

If you just give them a lego set they have to assemble, they have to think about what to build and how to build it, which is intimidating.

But a pre-assembled lego set is just a toy that just so happens to be customizable.

The limit to this pattern in the past was that it took time and effort to design and pre-assemble all of the lego sets for different needs.

But now with LLMs allowing infinite software, the balance point shifts.

A human has to iron those wrinkles out by curating the output.

They have tons of latent capability if you just give them the right inputs.

Giving the right context at the right time is the frontier for unlocking their quality.

Can you connect the dots or do you have to wait for the model to?

Does the model set the ceiling for what you can do… or the floor?

You need a pace layer outside the model to accumulate intermediate insights.

Those intermediate insights can be fed back into the model in future iterations as context to reach further.

Those intermediate representations require curation by a human, otherwise they spiral out of control and decohere from reality as the LLM throws itself into a cycle of slop.

The human doesn't have a chance to go "wait no, don't do that."

The faster agents loop, the more easily they can get themselves confused… or tricked.

If the system does a dumb thing, you blame the system.

Or do you want software that the most specific people love in that environment?

One is a mundane average--not known to be compelling to anyone, one is the most compelling to real people.

That box was intimidating if you didn't know how to structure your query.

Once autocomplete was added to search, the query rate increased discontinuously.

It helps feel like an extension of you--a deeper conversation with the system.

Also, you can pick your own model!

An early Mac principle: "never punish a user for exploring."

Seems like a powerful new ingredient for new kinds of game experiences that weren't possible before.

That has the downside that there's now one ever-more-powerful entity.

Even if that entity starts out with good intentions, that power is corrupting.

These systems struggle to become truly ubiquitous because most participants would rather not cede so much power to that entity.

But everyone focused on decentralization at the lower levels, where it's easier to combat.

But because each problem seems unrelated and small, you don't realize that there's a single thing that could solve all of them at once.

But when the paradigm shifts, it's an explosive unlock.

Paradigm shifts require solving multiple problems all at once.

So they're hard to make legible before they happen.

That's why they seem to explode onto the scene.

Problems that everyone has but everyone thinks are unchangeable can have massive explosions in use.

People become blind to it because there's no way to change it, so they just live with it and forget how much it sucks.

But then if something comes that makes it better, you can't not use it.

Related to Alex Rampell's frame of temporarily out-of-the-money options.

Critically, if it always stays illegible then it's not valuable.

It's the transition from illegible to legible that is where the discontinuous value is created.

If an idea is legible, and it's doable and desirable to someone, then it will have already been done.

Legibility is upstream of knowing if it's doable and desirable.

Where you're either in a meeting or not.

What about an optionality calendar?

You also only have one UI for all uses of the calendar.

Facebook slurped up all of the social-adjacent use cases and then said, "nah, screw it, we're just going to optimize for engagement in an infinite feed."

The result was they left a barren wasteland nothing can grow in.

No individual businesses that are viable in that desert, but there are tons and tons of use cases.

That's a coordination problem.

A dating app is more useful if it has a larger user base.

That leads to the logic of hyper-scale, which leads inexorably to one-size-fits-none dating apps.

Starting off free sets a mindset that is hard to shake.

Someone told me that in Ecuador even in fancy restaurants you'd hear Spotify ads for the music playing in the restaurant.

One way to get free usage without a free tier is to make it so friends can gift credits to their friends with some multiplier on credits.

In order to not be a culdesac, it will have to be an open system that you can use for anything.

It will have to be bigger than just chat.

This product will change the world.

Now the computer can learn to speak you.

Sculpt the system to fit the metaphor to slide into people's minds more easily.

A coherent metaphor helps the product resonate even though it's new.

If you ask LLMs to be creative they converge to the mush average.

But if you inject structured noise, for example by having it apply Oblique Strategies then they can be more creative.

LLMs will always pull you to the average.

So to diverge you have to give them divergent inputs.

Rhymes with the logarithmic-cost-for-exponential value and exponential-cost-for-logarithmic-value curves.

The logarithmic-cost-for-exponential-value is fundamentally fuzzy and imprecise, but at large-enough scale, it dominates the other benefits.

To people who don't, it just goes over their heads.

Most jargon goes over most people's heads, only for the right specialists with the right background knowledge does it land.

The right jargon unlocks the right library of background context.

LLMs understand almost all jargon.

Optimization is easier to do incrementally than correctness.

It's a switch from default-diverging to default-converging.

"It's semantically correct but it's very inefficient" is the toehold.

When you're making decisions, you're forced to collapse the wave function.

Instead of just following along and predicting what will happen, you have to also be in a "change the world" mindset.

If you're watching from afar and just predicting, you can just idly predict.

If you get distracted for a bit, nothing changes, everything keeps going as before.

So you're paying attention, but you're not "in the loop" with it.

That's what makes being "in the loop" or "in the arena" helps you absorb significantly more knowhow.

In the OODA loop, it's the Decision that is the core of the loop.

Without it, you're just observing, or being buffeted around by forces around you.

Everything good, everything emergent, comes from the decision.

Making decisions is what gives you ownership.

In today's late-stage-of-whatever-paradigm-this-is tech culture, employees are rewarded primarily for doing whatever their manager told them to do, quickly and with polish.

"I've been told to execute it, hearing anything about why it might not be feasible or not a good idea just stresses me out."

Obligation to the collective, to something larger than themselves.

But now it's "whatever's best for me, ignore the externalities."

Nothing builds.

It's all a red queen race.

Eat or be eaten.

No, this person doesn't yet realize how rich you are!

It's not even grand ambitions any more, it's just "optimize without thinking to extract value while creating negative externalities."

Sad.

Starting up atoms-based businesses is extremely capital intensive, which means only businesses that have a safe, legible business model can get financing.

Bits-based businesses have startup costs, but much less, relative to their possible scale.

That's a great fit for venture investing.

But if the cost of making software drops, then even the VC model isn't that important, more people can simply build little bits of software and then bootstrap the ones that get momentum.

If many independent people say it, it's more likely to be true.

But people choose to repeat something if they think it's interesting: surprising and plausible.

In an echo chamber that can bounce around and make one guess reverberate into a strong story as everyone makes it just a little better of a story.

"Look, I made this successful thing!"

"Yes, but it is powered by destroying value all around you."

This idea is related to the book Status and Culture by W David Marx.

Discomfort is a path for growth.

It's not about efficiency or engagement

It's about alignment with human flourishing.

Resonance occurs when tools expand our capacity, our connectedness, our sense of the possible.

Where hyper-scale reduces us to data points, Resonant Computing adapts to us as full humans.

This riff comes from Aish.

Spirit: esprit de corps.

If you aren't consistent in your actions and your ideals, you lose your soul.

You pull back, you disengage, you lose your soul and your will to improve the thing you're a part of.

When everyone is individually feeling resonance: living in line with their ideals, the natural emergent outcome is also prosocial outcomes for the collective.

It doesn't matter what those individual ideals are as long as they are mostly in the same direction, and have a long-term orientation.

Nuance could also be called "texture".

Hollow: the closer you get, the less impressive it is.

Resonant: the closer you get, the more impressive it is.

In normal teams, ambiguity makes the team lose trust in one another.

In high-performing teams, ambiguity makes the team gain trust in one another.

The switch from default-diverging to default-converging is tiny but infinitely important.

Enterprise academic style: think, think, think, model, and write a paper.

Scientist vs economist.

It can only understand and coordinate around momentum.

You need an editor to have a coherent strategy.

That implies an entity that everyone in the organization agrees is allowed to curate.

That implies more of a top-down culture.

When there is momentum it makes people believe.

It's hard to make momentum where there is none.

Where everyone feels like a member of the overall corporation first and foremost, not their individual team.

Where it's a positive-sum mindset.

Where it doesn't feel like an existential danger breathing down your neck, making everyone feel defensive.

So less defensiveness internally and externally.

Resonant emergence happens when everyone is participating from a position of optimism, not fear.

It requires a coherent strategy for an extended period of time.

Which implies someone who can make a schelling point that would stick.

Instead focus your arguments in the following percentage:

As you get momentum, the focus will naturally come to the strategy, as people can see the momentum.

Before there's momentum, trying to get momentum around your strategic north star is nearly impossible in the bottom-up chaos.

Instead, get momentum on the short-term in things that you know align with a compelling long-term strategy.

It takes one person to poop a party.

Because otherwise an upstream change could blow away the edit you made in an intermediate node.

They distill a concept that explains a thing you could previously sense but not describe.

They give the concept that you can step into, and it feels warm, clarifying.

A vague sense that you didn't even know you needed a word for, but once you know there's a concept there, the world seems less overwhelming.

It is their beliefs that set their world model that they react to.

Norms arise out of interdependent beliefs and expectations about what others believe.

Rules are schelling points, they help set the default of how people think a given situation will evolve.

But they are just a default.

Why do you think X is inappropriate in a context?

Because you believe the other people will think it's inappropriate.

Not what you believe, what you believe others believe.

You know your internal mind but not others' internal mind.

So this emergent belief of what other people believe is more stable and takes longer to diffuse than if you based it on your own beliefs.

Because it can take you longer to notice they changed their mind since you can only see external signals of it.

It removes option value.

You set a future outcome as a fixed point to pivot around.

That's why individuals often would rather not coordinate if it doesn't help them achieve a thing with the collective that they care about.

If people believe in the power of that particular collective they are willing to surrender some of their autonomy to it.

Without people seeing the collective as a thing worth investing in, you get an incoherent swarm.

I have a minor in Computer Science.

Earlier in my career the CS felt more useful.

But now with the rise of LLMs, this odd kind of cultural technology that is grown, not built, Social Studies feels more valuable.

For example, maybe you need to think about marginal changes to the whole.

The ones that will survive and thrive are the ones that align with the inherent logic of the system.

A kind of ideal citizen in modern large-scale bureaucracy is what David Brooks calls "organization kids".

Otherwise you 1) don't realize there's anything wrong with your model and

2) don't know the gradient to improve it.

A boss getting feedback from a report is hard.

So the report softens their feedback, which might make it too subtle to be received by the boss.

The more intimidating the boss, the more likely they are to lose their cool and fire someone, and the more that people won't share the feedback.

It will be a super-critical state, ready to shatter.

Power is emergent in the social imaginary.

People who people believe have it, have it.

It can turn into an aura of invincibility.

However that means that when they lose in a public way that power can shatter in an instant.

This is the logic of the Saruman.

Serendipity works best when you plant lots of little seeds of trust that might blossom into something in the future.

You plant the seeds for their own sake, but they also have a bonus of some small chance of greatness.

I am hyper-extroverted and hyper-conscientious, which predisposes me to trust-building actions naturally.

I didn't come up with this strategy from first principles, I retconned it from a thing I was doing naturally that worked better than I would have guessed it could.

He'd call them on what he saw as bullshit… even though he was often wrong.

Knowing you're winning a sparring match because you're right rather than because you're powerful helps you ground truth your beliefs.

His manager saw the value in that for truth-seeking.

"How does this benefit me?"

"How can I use this to achieve my ends?"

"Whatever thing I want right now is the most important thing in the whole world."

Selfish narcissism.

As we become more wise we realize the value of creating value in the world that is not centered around us.

Everything changes.

The context changes, and the thing that was previously done must change.

It is no longer done.

Life is change.

Starts small, but then grows incrementally but quickly to something amazing.

If you're judging the quality based on the instantaneous response, then you'll think a big bang that then rapidly evaporates is better.

What matters is the absolute area under the curve; slow and steady (and ideally compounding) is way better than fast and loud without momentum.

Momentum is a second order phenomena.

It's not visible at any one instant, but it's more important than any one instant.

You get many to behave as one.

"What you give is what you get" over infinite time would equalize to be strictly true.

Good Intentions.

Good Words.

Good Actions.

It's just a pointer to the thing.

The thing is what matters.

Just saying the word doesn't make the thing happen.

When you say deep words like "spirit", you focus on the word, not the action.

You have to live your ideals, not just speak them.

If it persists, it's doing something useful.

That useful thing might not be obvious at first glance.

Billions of micro-decisions by individuals accumulate into the emergent force of technology.

Everyone in their little ways tries to make the future better than the past.

It's the sum total of everyone striving to leave the world better than they found it... in a way that gets some of the value for themselves.