Bits and Bobs 1/21/25

On the same scope of impact as:

The closer you look, the more obvious this becomes.

Disruptive innovations scramble the competitive dynamics and enable whole new industries to emerge.^[afe][aff]

Nearly everyone today seems to be implicitly assuming it's a sustaining technology.^{[afg][afh][afi][afj][afk]}

It's impossible for that to not be disruptive.

Especially since the technology for GPT4 class quality is already commodity.

O3 in particular gets super human performance from grad student level common sense, combined with infinite patience.

But then they grow to then eat the thing that birthed them^[afm].

These kinds of disruptive technologies often emerge from environments that have significant amounts of capital to spend on open-ended R&D.

But the logic of the disruptive innovation invalidates the environment the parent assumes.

Big incumbents would rather new technologies be sustaining, because if not it's too chaotic for them.

The next generation is fertilized in the not-yet-dead corpses of the previous generation.^[afn]

Everyone learned the lesson that in a gold rush you should sell shovels.^[afp]

It seems like no one's mining for gold!^[afq][afr]

Everyone's trying to do the meta play, not just using^[afs] LLMs, a new, disruptive input.

If it were an obvious or common question^[aft], there'd be a ready, off the shelf answer.

The harder it has to chew on it, the more surprising, interesting, or novel the question is, at least in some dimension.^[afu]

If they are easy to predict once you categorize their worldview, they aren't that novel–you can accurately predict what they'll say based on the category they fit in.^[afv]

A similar test could be applied to LLMs: the novelty of an utterance has to do with the inverse of the likelihood^[afw] that that next token would be predicted by the LLM based on the preceding tokens^[afx].

Both the background knowledge that makes them smart is from culture^[afy].

But also the thing that makes their output good is the quality of the steering the user is doing via prompting.

Related to Alison Gopnik's stone soup AI parable.

Rubbing two sticks together.

Don't build a business assuming how it works now will be how it works later.

It can whip up a little interactive thing for you based on an English language prompt.

But all it has to work with is what it absorbed during training: the frog DNA of the average of the internet.

It can never do better than what the model already knows.

A low ceiling.

What if you could also give it real data^[afz][aga] to work with?

Or specific examples close to what you wanted to build to draft off, from smart people with similar needs?

It could create some spellbinding things!

Like the alien in Contact.

It's a hive mind of all human intuition, but it presents itself like just a random hyper-competent person.

That is comforting and easy to use… but also gives the wrong mental model for what they can do.

LLMs can perform feats of patience and recall that no human ever could.

If you don't think about how LLMs are different from humans, you'll never think to ask them to use their super power.

LLMs can give you disconfirming evidence to critique your idea, without any shame.^{[agc][agd][age]}

Safe. Private.

You aren't wasting anyone else's time, and you don't have an audience to be embarrassed in front of.^{[agf][agg][agh][agi][agj]}

So you can absorb the disconfirming evidence^[agk] significantly easier.

This can make your ideas significantly better than they'd be in a world with scarce or expensive disconfirming evidence.

Hobbies are work you do for its own sake, for its own enjoyment.

Hobbies are things you do just for you.

Hobbies are a form of entertainment that is not easy. They take work.

Many hobbies are very hard to start doing, and hard to get better at.

LLMs are great at being an intellectual and creative dance partner, helping you grow in a forgiving environment.^[agl][agm]

LLMs make it "cheaper" to start and stick with intellectual and creative hobbies.

They do a better job "reading between the lines" and "picking up on what you want without you even needing to ask for it."

An insight from Peter Burns in a comment on last weeks' notes.

Thought is conversation with yourself.

The more powerful the other participant the more insight.

So if one participant is the hive mind of all of humanity… wow!^[agn][ago]

They have useful things to say, and also have the patience to absorb disconfirming evidence from you.

Don't use LLMs to write books.

Books are a static communication medium, one-size-fits-all, frozen for all time.

A book author makes a static guess at who the audience is: what they already know, what they care about.^[agp][agq]

But the guess of the author directly sets who can possibly absorb the book.

The whole point of LLMs is that they're interactive and can adapt to the listener's knowledge and needs.

LLMs enable a new medium of choose-your-own-adventure, perfectly adapted to the reader to help them get just what they need to absorb the insight^[agr][ags].

The downside is that we don't all have the same experience of it, like we do with a fixed piece of media.

But the upside is that we can learn so much more, because we have a medium of information exchange that can perfectly adapt itself to the listener, efficiently transmitting way more information^[agt].

The test is, are you thinking more or less?

If you give its answer directly to others (without curation), you are giving an average, eroding yourself.

But in general, you should be applying more judgment, not less.^[agv]

Agents take actions on your behalf.

Filters^[agy] help sort information and make recommendations.

Filters, if they're not perfect, don't hurt.

Agents, if they're not perfect, can do real damage.

Downside can lead to game over.

To make agents safe requires significant, carefully controlled guardrails.

Think about the number of tasks you do in a day that could be successfully delegated to another person–even an infinitely patient, human-level-common-sense, cheap agent^{[aha][ahb][ahc][ahd][ahe]}.

Coordinating your beliefs and wants, and verifying the quality of the output, is a non-trivial coordination cost, even if you assume human-level common sense.

Humans don't re-read every previous message of the conversation before responding; they use their imperfect fuzzy memory of what's already been said.

But LLMs don't do that at all.

LLMs "read" the whole context for every single token they produce^[ahf].

Which means that as a modality you totally can go back and amend, tweak, edit the previous messages to give better background context.

You need a diversity of successful things to compare to figure out the resilient common denominator that works in most situations.

If you only have one example, you don't know which properties led to success.

Until you have multiple successes to compare and contrast, you can't tell which properties are more likely to be determinative.

If it's novel, those don't exist to triangulate against yet!

The ROI requires predictability, best practices to have been established, lots of people to have tried and compared notes for what works.

They didn't say "We'll do exactly what we did before with horses, just with less hay use."

"How can I use this deeply disruptive technology to get 30% more efficiency at what I'm doing today" seems like the wrong frame.

It's only in a hyper optimized and financialized world that we'd jump to that kind of analysis.

A better frame: "What new and transformative things are enabled by this disruptive technology?^[ahi]"

One pattern is the open-ended conversation.

One is a recipe for generating insight from interactions.

Code is unforgiving, there are some things it can do well, and some things that it can't do at all.

Today if you have a use case it's software first.

What if the software could just fade away, be an implementation detail we never really think about, because it's always created just in time to do whatever I'm trying to do?

Before LLMs, software used to be interesting.

Now it should be boring. An afterthought.

Most people can't create software, and even people who can don't necessarily care to do it in most instances because it's expensive.

AI makes it easy to extract value out of data yourself.

Vercel's V0 does exactly what you asked it for.

Websim will add things you didn't ask it for.

Not a patch job on top of the OSes we have today, but a new kind of "OS" that was AI-native, and built in a world that assumed high-quality LLMs.

LLMs are a new kind of computation.

They just get more active.

More investment of energy from the community.

But by default that more investment creates cacophonous oversupply.

As there's more and more stuff, it gets harder to find the good stuff.

For a digital commons to get better with more activity, it has to have some kind of sense-making apparatus, a quality pump.

The quality pump is the sorting process, so that more activity helps select the better stuff automatically.

In Wikipedia, this is the fact that there's a single shared namespace.

In YouTube, there is a proprietary ranking algorithm for recommendations and search results.

When you have a quality pump, more content doesn't drown out good content.

With these quality pumps, you want some way to detect if new content is good^[aho].

The main meta property of a good quality pump: more activity makes it sort better.

A good quality pump gives you upside if the new content is great, but capped downside if it's not, because few people will ever see it.

Once no one else can plausibly compete the drive for improvement is gone.^{[ahp][ahq][ahr]}

That's how you get stagnation.

The value of data is combinatorial; the combination of the right bits of data creates new value.

So as you get more of your data under one roof, it's now possible for more of the combinatorial value of your data to be activated in a way that creates value for you.

But you are likely not the only one who is storing more and more data there.

You are likely storing more data with them because they are an aggregator.

Aggregators are like gravity wells; it becomes harder and harder to resist as more people use them for more things.

But the ability of an aggregator to add value-creating features goes down with additional scale^[aht] of usage.

This is because of the tyranny of the marginal user: to grow scale, these providers need to make their products more and more lowest common denominator, dumbing them down.

Imagine a feature that would revolutionize the lives of a certain niche of people, e.g. TTRPG players.

This effect only happens when the entity deciding what kind of software to build is not the user who will benefit.^[ahu]

This is a fundamental divorce of value and incentive.

This stagnation is one of the primary problems of centralization^[ahv].

The main problem of centralization is that as companies get larger they can't do small niche value unlocks as well.

The media is published and has to be as good as it will ever be.

Very unlike the software world.

A movie is a fully static, unchanging artifact once published.

Software can be updated after the fact over the internet.

Also everyone's experience with software–even the exact same software, bit, for bit–is different, because software maintains state that is personal to a user.

Whereas a movie is always the exact same bits for every user, every time.

This allows shared cultural experiences for everyone who consumes it, shared touch points.

But it makes it much harder to iterate.^[ahy]

This generated some skepticism in the comments.

Let me try a slightly different derivation.

Learning means developing a better predictive model of the world.

Without error there is nothing in the model to correct.

Correcting weights to produce a better model is literally what learning means.

An error is a kind of failure.

Failure often hurts.

Sometimes there are situations where error doesn't feel like failure (for example, in a situation of play) or where failure doesn't hurt (for example in a psychologically safe environment) but those are special environments that have to be cultivated.

Which will give you more high quality output over some time horizon: optimizing for quantity or quality?

In practice, if you optimize for quality, you spend more time planning, debating and trying to produce a theory to then execute.

Whereas if you go for quantity you spend more time doing.^[ahz]

So quantity can often lead to better quality, too… if you have a sufficient feedback loop.^[aia][aib]

Play is how we learn best.^[aic][aid]

Evolution likes us to learn.

So evolution made play joyful so we'd do a lot of it.^[aie]

The old school or "close-ended" approach is an adversarial Dungeon Master.

The modern approach is "open-ended". ^[aif]

You could argue there was no path.

Every single part of the organization and business had been built up in a world that assumed a certain vector for gravity.

Digital photography tilted the world on its axis.

It wasn't that some part of the company wasn't a fit and could be excised.

It was that the entire company was subtly-to-significantly incorrect in that new world.

This is one of the reasons disruptive technologies tend to create new winners and winners from the previous era tend to fail to adapt.

The famous Gramsci quote feels inescapably true today.

"The old world is dying, the new one struggles to be born. Now is the time of monsters."

Incredibly legible? optimize numbers.

Incredibly illegible? vibes/qualitative.

This riff is from Ben Mathes, shared with his permission.

Focus on empowering the believers to act.^[aig]

Look for excitement and make that easier, not reducing friction where they get stuck.

The latter gives you an increasingly bland thing, vs a more impactful great thing.

Don't look for where marginal users get stuck.^[aii]

Look at the heaviest weirdest users who are using it in surprising ways and make their lives easier.

Discretionary effort is the term of art for when someone decides to go above and beyond what is required of them.

They went above and beyond because they wanted to, and if you tell them to do another thing they might just only hit the required bar and not go beyond.

Volunteer labor is free, but also more difficult to direct.

If you direct it to a thing it doesn't want, it just evaporates.

Don't try to control discretionary effort, just be thankful that it's there.

Discretionary effort creates bonus upside. The only downside is that you can't control it beyond the lightest nudges.^[aij]

This creates systems where everything within them is known to be pure, accurate, safe.

This can make reasoning within them much simpler, and allow elegant formalisms and simplifications.

The problem is that the transition from the real world to being represented in the system becomes much more onerous.

The real world is messy and impure.

To model the real world in the pure formal system requires quite a bit of work from the user to "import" it.

This work can be like a wall in front of you before you even get started.

So the tradeoff is you never run into impure states within the system, but you also might use it for far fewer real world things because it's such a pain to model real things in it.

Ideally you have a rough and ready, flexible 'inbox' where you can roughly model things and then iteratively clean them up to become more pure over time.

Ideas have people.^[aik]

Things that are outside the status quo.

Close ended systems grow with a sub linear curve.

A hyper distillation of Geoffrey West's Scale.^[ail][aim]

Zealots make a community very active... but also very intense and off-putting for people who are not zealots about that topic.

They grow quickly and stay very active, but have a hard time growing past some small size.^[ain][aio]

If it's self-evidently viable, there's a throughline that everyone can see and naturally coordinate around.

If it's not yet self-evidently viable, there's a swirl of coordination around different schelling points.

In a post-coherence team, it's more about "is this someone I can work together well with"--which often comes down to someone you've had a good experience working with in the past.^[aip]

But pre-coherence (e.g. a pre-PMF seed stage startup) "liking working with each other" is secondary. What's primary is "we all believe in the same big vision and find it deeply motivating."

Because you'll need to crawl through broken glass together.

That's only going to work if everyone really cares about what's on the other side.

Love of your fellow crawlers doesn't get you to crawl through broken glass, the prize at the end does.

People who love the vision and are committed to crawling through glass to do it, will come to love each other.

But people who love each other who are crawling through broken glass for a thing they don't care about will just get frustrated and angry.

In pre-coherence it just accentuates the lack of coherence.

Makes things more chaotic.

Detail by default slows down momentum.

Versus a thing that has momentum even if it's hideous, everyone can agree at least it's viable, taking any details of how it works as at the very least obviously producing something viable, albeit possibly ugly.

Constraints cut off possibilities; but also give you structure, groundedness, to lever off of.

Inside Out 2: "Make your curse your gift."

They're hard to enforce on yourself.^[ais]

At each point you'd rather just bend the ones that are inconvenient.

But ones that come from outside yourself you're more to take as a given, like the weather.

Impossible to change, so let's figure out how to operate within them, to use them.^[ait][aiu]

Opinions are personal.

No one can challenge them.

Opinions are divergent.

There's no synthesis, they can exist on their own forever.

If you're trying to build something together you need distilled insights the group can use to build together.

To do something together people have to choose to adopt your opinion as worthwhile.

There's a process of synthesis and group sense-making that is more than just "everyone state your opinion."^[aiv][aiw]

Shared clarity can only come from synthesis and curation, value judgments and decisions.

Momentum is everything.

Everything is ambiguous at all times.

If you want certainty, you'll never get it, you'll be frozen in place.

How do you take actions while not knowing everything with certainty?

Bad systems thinking makes it more complicated.

Good systems thinking makes it simpler.^{[aix][aiy][aiz][aja][ajb]}

"You were focusing on all of these noisy ripples, but it's actually this single hidden undercurrent that is an order of magnitude more important than everything else."

But flexibility is not free; it can be extremely expensive to create.

One way to create flexibility is to design a system that can do anything at all in theory.

Sometimes this theoretical flexibility is extremely expensive to create, and you block doing anything else while you try to build it.

Another approach is practical flexibility of being willing to have a high rewrite count.

Try something you know won't take you where you need to go and will only work for 6 weeks or so, to start absorbing knowhow and relevant insight to help you improve your accuracy on the next iteration.

That bar is much easier to hit, by orders of magnitude, and allows generating hte all-important momentum.

You have to be correct at relevant questions.

You can fool yourself easily by mixing this up.

Finding the right questions is more important than finding the right answers^[ajc].

If you and someone else see the same set of facts, you could still disagree on their importance if you have a different set of values.

Values are literally what kinds of things you value.

There's no objective weighting of the importance of a given set of facts, because you need to bring your subjective values, which might differ.

Your values are tied to your beliefs, your goals, your principles.

Because you want it to be true, or it aligns with your priors (your experience).

Those are the ideas you believe in, that you assume are correct by default.

Some ideas you are more skeptical, and say "prove it to me."

Those are the ones you don't already believe in.

We think that we're exhibiting proper skepticism in all ideas, but really we're spending time on the ones we don't believe in already and letting the ones we believe in slide.

Our values configure which things we treat with skepticism and what things we treat with credulity.

You need their beliefs to update, and if they don't understand, they won't.

If you just repeat the way it makes sense in your head, and it doesn't have an aha moment for them, you have to figure out a different way to unpack or frame it for them.

The vast vast majority of signal that is sent out in the world fails to be absorbed.

Because it assumes preexisting knowledge or motivation that doesn't match, or because it doesn't stand out in the cacophony.

The more nuanced and multi-layered and surprising the message, the less likely it's absorbed.

Finding succinct, compelling, clarifying utterances to deliver an insight is actually a large portion of having an insight that actually burrows into people's brains and changes their actions and thus produces impact in the world.

The original insight is easy.

Communicating it to others in a durable, efficient way is the hard part.

It's great to be able to absorb the perspective and insights from others to get a richer perspective.

But if you don't have your own perspective, you become just a mush of different perspectives from many others.

Those perspectives might be contradictory or incompatible and impossible to reconcile.

You'll have adopted others' perspectives and then have to retcon reasoning for it… which might be impossible given the other things in the set.

To synthesize requires curation, deciding which facts to attend to.

You should listen to everyone, but you must converge and curate and distill to synthesize into your own perspective.

If you don't you will get stuck in intellectual and emotional quicksand with no answer. Everything you try will get you more stuck.

Autonomy is about being empowered to make decisions.

In larger teams where it's past the size of two-pizza, you have to split into sub components with boundaries around the areas where that autonomy operates.

But in smaller teams you can all be aware of all decisions.

If you ask yourself a pointed question you can just ignore it.

This means that mentors can help you grow in ways that you couldn't have discovered on your own in a million years.

Questions that are uncomfortable but help you see your way out of a local maxima.

Extremely important to be able to do up to some sufficient level.

But pushing it past that point is grotesque, unnatural, and unnecessary.

You should make sure you can walk well, but don't try to be a speedwalker.

These are the kinds of skills that are best satisficed, not maximized.

Operating is one of these skills; extremely important, but if not combined with strategic judgment, can be actively dangerous.

The CEO component is the vision and external focus.

The COO component is the operational oomph to actually get something done.

You want to satisfice the COO component and maximize the CEO component.

The vision is sacred, the operations are profane.

The profane will taint the sacred.

At every point there will be an operational detail to pull you more to operations.

Operations is a means, not an end.

"you must create thick boundaries^[ajg] between the sacred and the profane

value creation (ideation, generativity, divergence, etc) is sacred

value capture (monetization, politics, operations, etc) is profane

we need both, but the moment the profane touches the sacred, it kills it"^[ajh][aji]

The product--what people do with the tech--is all that matters in the end.

The basic recipe is to find a northstar and then steadily accrete thin layers of slightly-generalized useful functionality.

But this is only if the vision is loosely held, and you're just trying to do something useful.

If you're trying to change the world with a vision in a seed stage startup, you need to take a big swing at a specific vision.

If you build broad, you'll never be able to build anything specific enough to reach very high.

That's fine in many cases where you just want anything viable that produces value, but if you have a specific thing you're trying to accomplish, you have to be specific.

In those cases you want to rough in a thin slice to lift up to the value early.

In a big company you have the patience and time to build up broad.

But if you have a specific thing you're building, you need to take your shot, and do the best shot at it.

They explode out because of some tension, some pent up energy.

But they don't have their own internal structure or support to create sustaining energy.

In the end, the system always evolves whether you like it or not.

Where do the semantics emanate from?

It can't be the layer with multiple options.

It must be the part with only one option, even in the platonic ideal expansion of the system in 20 years.

The opposite of a tired take, or something an LLM might say.

Crisp, clear, new, vital, nutritious, tasty.