Bits and Bobs 11/18/24

What new things are now only possible with AI?

"Only puny secrets need protection. Big discoveries are protected by public incredulity." - Marshall McLuhan

LLMs do well with e.g. contained algorithms that are commonly shared in write-ups on the internet, but the kind of random non-web-frontend glue code and architecture of a real codebase they don't do as well at.

Still quite good, and often above the bar, just not always hitting it out of the park like they do for React style code.

The meta insight is that LLMs' ability to write code on demand is not some smooth distribution over types of code, but spiky based on which types of code are commonly found in tutorials and open source code, as well as how boiler-plate-y the code is.

Not unlike LLMs!

We're all inherently creative.

Confidently answering a question on the spot, distilling an answer that is plausible within the context of everything else you know, is a creative act.

Claude feels like a professor's living room.

An Eames chair, tasteful leather-bound books, a nice sculptural object on the mantle.

It has high taste, but it's its own taste, not yours.

You'd feel like you'd mess it up if you tried to make it feel like yours.

What if you could allow creating hand-hewn software?

What if every instance of a program looked slightly unique?

Slightly different color palette, slightly different angles for each button.

Instead of the system saying "No." how can it say "Yes!"

Aesthetics are surface only.

It's easier to play around with aesthetics (with the right tools) than to play around in fundamentals.

People like to show off their taste.

An engagement loop!

A way for low-savviness tinkerers to start using the system and become savvy.

A thing that a kid could do.

As you get momentum, you stretch to deeper and deeper tinkering on the fundamentals.

If you look at tools like Airtable, they are intimidating, mostly professional.

They scare off people who don't consider themselves technical.

You want a thing that feels simple, fun, customizable, "you can do it too!" and "the sky's the limit for what you can accomplish".

Premium, but DIY.

Bricolage, tasteful combinations of eclectic, zany objects and ideas.

This classic piece has thoughts on how to create creative tools for people who don't think they're creative.

In contrast, users tend to build from the bottom up, from experimentation, for example in spreadsheets.

It is this top down vs bottom up orientation that is challenging to develop in would-be programmers.

How can you design your system for bottom-up emergence?

You stick one on an object, and it enchants it.

A spell you can stick to anything.

It's a load-bearing party trick to show people how open-ended the system is.

You break people's mental model of what is possible, and suddenly everything seems possible.

Superficially coherent, but the closer you look, the more you realize how little sense it makes.

Enough that you can't hand-hold each user and have to handle a number of real world challenges... but not so large that you have to do architecture astronautics.

You can find high-quality ideas without having to worry about high fidelity.

Can you duct tape something that will work for the next 3 months, and that won't set you on a path that in 3 years you'll look back and hate yourself for?

If so, good enough!

Fidelity is about superficial appearance.

The quality is about the fundamentals.

Fidelity and quality take a ton of time and effort to create.

When you're searching to find something great, you want to find something that has great fundamentals but ignore the fidelity for now.

The most important thing is to find the fun and then nurture and grow it.

The fun emerges out of the fundamentals, not the aesthetics.

So you throw together a "gray box" prototype: where instead of finished assets, you have just gray boxes.

If it's fun with gray boxes, it will definitely be fun with real assets.

If you did real assets first, you'd waste a ton of time on a direction that might turn out to not ever be fun.

But you also might trick yourself into liking it even if the fundamentals aren't strong: "it's so pretty!"

Aesthetics can hide fundamentals.

So remove aesthetics to see the fundamentals at the beginning.

What things to graybox is a matter of judgment.

For example, if you're making a platformer with a new jump mechanic, you'll want to dial the jump mechanic in with very high fidelity to test it out–but the animations could all be grayboxed.

What parts to graybox are the parts you're confident, based on your experience, that you can later fill in in a sound and compelling way within the rest of the system.

Default converging is like surfing the wave.

An infinite difference.

"No, but" energy pushes a team apart.

Sometimes you have to use "no, but" energy.

But sometimes you don't have to, because you can "yes, and" aimed at a subset of what they said, achieving a similar outcome as the "no, but", but without the energy cost.

When they are not cohering it is like a cytokine storm, tearing everything apart.

As a generative function in a bottom up open ended system it can do wonders.

What happens when we allow smart things like LLMs to use them?

You want a blackboard system.

Sometimes the emergent effects are bugs.

Sometimes they are wonderful unplanned accidents that create all of the value.

Is the emergence expected or unexpected?

Will it be welcomed or shunned?

Emergence cannot be controlled.

It can be contained, or it can be cultivated.

If you want to build a thing with emergent effects, the only way is to build something you can play with and see what emerges.

The Serious archetype is typified by the React architecture of convergent modern front-end development.

The goal is to precisely implement the design with minimal variance.

The least-bad architecture for app-scale, tightly coupled, reproducible systems that should precisely implement designed UX.

Each layer is carefully laid down to fit the requirements, and then the next layer is laid down until you're done.

The React approach fights emergence, tries to tame it to get the designed output.

Assumes that the people writing the code are at least minimally technically competent and able to write code to converge to the PRD.

The Playful archetype is used to build games and is typified by the Entity Component System architecture.

ECS assumes loosely coupled, coherent outcomes from swarming inputs.

There might be a thousand people of all kinds of backgrounds (many not particularly technical) who are collaborating on the game on a tight deadline.

How do you make that swarming activity not be a cytokine storm and instead emerge into something greater than the sum of its parts?

You need loose coupling, the ability to have happy accidents.

You need sliceable code: code that can be extracted from one part and plopped into another with nearly no integration work.

Factored into behaviors and slapped onto arbitrary objects.

Imagine that you have a system where an event on object A triggers a reaction on object B. In a game context, this might be "the player walks onto a special square that triggers a door to open."

Now imagine that you also want to trigger a reaction in Object C, e.g. "play a sound".

In a typical, tightly coupled system, you'd have to go to the logic that caused A to trigger B and change it to also trigger B. But in an entity component style system, where everything is interacting via a shared message bus, you can add in a new rule "Trigger C when A happens", and that will now automatically happen... and the original code doesn't have to be modified or aware of it.

ECS cultivates emergence.

Emergence is the point.

Very few people have deep lived experience with both archetypes, to feel in their bones where they are useful.

The two archetypes are wildly different; if you're familiar with only one side, the other side will feel foreign, odd, counter-productive.

It's only from experience that you can see why each approach works so well where it does.

You can't "prove" with an example why a given archetype is better in a given context; you have to feel it in your bones.

If you want to see concrete examples of where it helps, each individual example will seem small, unimportant.

It is only in the whole, flowing, emergent totality of the collaborative swirl and chaos that the value of it becomes obviously, fundamentally, undeniably, useful.

Which of these two archetypes will be a better fit for AI-driven software development?

They struggle to write high quality software, even with lots of scaffolding.

To work in Serious development, the software has to be uniformly high-quality: five nines.

Everyone today trying to get LLMs to write software is trying to squeeze out more nines.

What if it's not possible to get LLMs to write software with five nines of reliability?

If you assume it will never be possible, you instead turn your focus to the kinds of architectures that are resilient to that.

Don't get LLMs to write Serious Software as it exists today.

Get LLMs to write Playful Programs.

Not through software as it exists today (hard to get squishy computers to do well with it reliably enough in a convergent sense), with LLMs, but a new kind of software that is only viable in a world where LLMs (divergent, messy, emergence).

If you don't love it, how could anyone else?

If you're going to build a fundamentally weird system, it has to be magical to hook people at the beginning.

At any moment you might stop working on the project for multiple years.

Every moment you work on it is borrowed time.

The other things on your plate determine if there's room or not to do an incremental unit of work on the project.

"I have an hour to do this, is this the highest and best use of time in this hour?"

When you get a bit of time to do some work on the project, do an Easy to get ramped back up.

Then, once ramped back up, do one from Important.

Don't do Hard until other work or circumstances moves it to the Easy or Important buckets.

For example, other refactoring work chips away at the problem and makes it now Easy, or some real-world fire makes it inescapably Important.

Don't spend novelty on boring layers.

It's inescapable.

There's many flavors of high brow (ways to stand out from the centroid).

There's one attractor basin of popular and compelling low brow (in towards the centroid).

If you're in that position, either relinquish the referee role or shoot your horse.

You always think the requirements are more clear than they are.

If you define requirements after the fact, you will have wasted tons of time swirling.

It's like trying to solve a Dynamic Programming problem without memoization and backwards.

Not just twice as slow, but orders of magnitude slower.

Feels like you're going faster by not doing too much abstract thinking up front, but actually you're going slower.

Each time you start again from the (new, not quite the top) requirements, team members think "here we go again, getting yanked around with new requirements again."

If you don't trust the person giving the requirements you might even presume bad faith: "they didn't like that I was arriving at an answer that wasn't theirs so they scrambled the requirements again to keep me on my toes"

"I don't understand the insight you see, but I trust you to do the right thing even if I don't understand it."

Trust is what lets teams navigate ambiguity together, without every little blip grinding to a halt as everyone insists everyone else explain to them in excruciating detail what they want to do and why it's correct.

Trust is fundamentally required in ambiguous situations, or in teams with a diversity of skillsets, experiences, and personalities.

But a high trust team with a diverse set of abilities is the only way to navigate a complex environment and do something great.

It's a great way to understand and improve interpersonal relationships on a team.

The amount of trust one person has in another is:

Trust = (Credibility + Reliability + Intimacy) / Self-Orientation

Credibility: Can you believe what they say?

Reliability: Can you depend on them?

Intimacy: Are they sensitive to you?

Self-Orientation: Do they focus on themselves or the team?

The higher the trust, the more you're able to navigate complex problems together.

Is there some relationship in the team that isn't going great?

Imagine how that person sees you.

Score yourself on the four factors of trust through their eyes.

Some of their perception will be wrong–part of the reason they don't trust you as much as they "should".

How can you help align their perception with reality?

What actions can you take to get their trust to align with reality?

Now do the same thing, and figure out how you rate them.

Now, realize that you are also wrong about them!

What are some of the places you're misjudging them?

How can you give them the benefit of the doubt?

Giving someone the benefit of the doubt, taking the first step, is the inductive seed that leans to a blossom of trust.

That's the whole point of Raya and The Last Dragon!

But In a lower trust relationship that same conversation destroys trust.

The right argument will be inherently convincing even to a skeptical audience.

It might take significant time and effort to discover the convincing argument, but it does exist.

If you can't find it, that implies you might be wrong.

Very smart people will be very used to finding the right argument quickly; if they can't, they will assume the conclusion is wrong.

But in a complex domain, there often doesn't exist a succinct, convincing argument.

The only way to know it's correct is to have the correct experiences, to feel it in your bones.

Someone who is smart but doesn't have the right experience will erroneously conclude that the conclusion is wrong, when in reality it's a complex domain and impossible to communicate succinctly.

That is, it's not to just chameleon with them and enable them to feel justified in feeling that way, the victim to an incorrect and uncaring world.

A leader needs to empathize, absorb, but still have their own perspective and help the mentee grow.

That often requires helping the mentee understand uncomfortable truths about the role they played in the dysfunction.

When you play the role of the victim you absolve yourself of the responsibility to learn because you had no agency.

Things can only be your fault if you had agency.

Agency is the ability to learn.

Another lens: "who gained or lost trust in the eyes of the other?"

These two lenses can be aligned or be orthogonal or even be at odds.

The object level is the former. But the more important indirect and long time horizon is the latter.

There's a 2x2.

High Insight / Grows Trust: A wonderful conversation.

High Insight / Destroys Trust: A pyrrhic victory. Maybe it could be justified if the insight was so instrumentally important, but probably the wrong call.

Low Insight / Grows Trust: Didn't help in the short term, but might set the pair up to tackle meaty problems later.

Low Insight / Destroys Trust: A conversation that shouldn't have been had in the first place.

A surprising number of conversations are that last bucket.

Not all insights you uncover are relevant.

For example, maybe the insight helped the leader have more confidence in the solution… but it was already the plan of record the team was executing on anyway.

It's very easy to burn through trust.

If both sides think the other side doesn't know what they're talking about, the more turns of the conversation, the more trust that is eroded.

Perhaps they're engaging on different dimensions: each conversant thinking the other person is ignoring what they're saying, while simultaneously completely missing what the other side is saying.

If you find yourself in a conversation like this, get it to end as quickly as possible.

The longer it goes on, the more damage it does.

Just because a conversation was challenging does not mean it was worth having or created net value.

When you aren't in learner mode, you forget that sometimes insights will emerge on dimensions you aren't thinking of, that are in your blind spot.

When you're stressed you think, "No, I'm looking right where you're pointing and don't see anything, you're wrong. But now look where I'm pointing, where I'm showing you an obvious and important thing that you would see if you simply looked!"

The more powerful person gets stressed the other person doesn't understand and will do the wrong thing.

The less powerful person fears they will be fired.

Both are defensive crouches they accelerate to make them less able to receive.

It's possible but unnatural to be in a marbled teaching/learning state at once.

It's only possible when you aren't stressed.

If you're stressed, you're in a binary either teacher or learner mode.

Radagast: "the team as a whole can understand insights individually that collectively form a quilt that leads to the right outcome even if no one person understands all of it."

Especially for complex domains, where proving it would require a combinatorial explosion of detail.

Answering a question is orders of magnitude harder than asking it.

Especially for complex domains, which can add a few additional orders of magnitude of difficulty.

Some answers are more valuable than others.

Some answers have great cost but little value.

How aggressive it is is a mixture of:

1) How important is it for success for the answer to be distilled and shared?

2) How much does the questioner outrank the other person?

3) How much better at debating is the questioner?

Debating in a live, intense conversation is a particular skill, and is not necessarily correlated with someone's ability to have relevant insights.

You have to figure out what shortcuts you can take to get good enough alignment and momentum to become default-cohering.

You don't have to write it down if it's a small team with relevant expertise who is converging.

You only have to write it down if you have non-experts who need to collaborate or need to coordinate at a distance in time or space.

A strong debater is good at thinking on their feet.

They can run circles around conversants and win an argument, even if they "shouldn't."

A common pattern: the other person is engaging on a dimension the debater doesn't see.

The debater (and anyone watching the debate) erroneously concludes that the debater won.

The other person will look dismissive but that's the only way to not get steam rolled by the debater.

Perhaps the other person can do deep thinking, but only in writing, not on the spot.

Both sides are sending insights the other isn't receiving

Trust: Seek to understand by assuming the other party is correct.

Leaders often push for their own understanding even when it's expensive and unnecessary for the team to succeed.

In those cases it's more about feeding the leader's addiction than improving the likelihood of team success.

If there are only a small number of individuals, we can keep all of their richness of all of the individuals in our heads.

But it gets super-linearly harder to do that as there are more individuals to know.

At the top end, say, a principal of a 150 person elementary school, you might be able to keep a rich, multi-dimensional picture of all of the students in your head, with extreme effort.

But it's not just you who has to--you have to coordinate with other people.

Even if you somehow have 150 individuals in your head, the other people you have to coordinate with will only have a subset--maybe each teacher knows all of the kids in their own class, but only the most stand-out ish of other students (the very bad kids, the very smart kids).

So when you coordinate, the number of "individuals" you can think of declines precipitously.

When we have too many individuals to coordinate around, we use summary statistics.

We choose the dimensions, we measure them, and place the individuals continuously along those dimensions.

Our brains don't do well with smooth continuum; they do better with discontinuous buckets..

So we put different regions of that continuum into buckets so we can talk about parts of the continuum in a more concrete way.

But the buckets are a convention we chose to allow us to talk about the swarm of individuals!

They are made up!

They introduce a number of illusions right off the bat.

For example, an individual at the top of bucket 1 might be way more similar to an individual at the bottom of bucket 2, but that will be hidden; all of the individuals in bucket 1 are implicitly assumed to be more similar to everyone else in bucket 1.

Another illusion that can only show up with bucketing is Simpson's paradox, an illusory backwards correlation that shows up entirely due to bucketing.

Another way that bucketing misleads us is that we forget about the dimensions we chose not to plot.

They just fade out of our awareness, even if they are extremely important.

As time goes on, the buckets come to feel more important than the individuals.

The individuals habituate to their bucket, assuming it is their destiny.

The individuals are changed by the bucketing and it imprints itself on reality in ways that might be counterproductive or dangerous, and make it harder to erase later.

So if the bucketing is leading you to an odd or surprising conclusion, remove the buckets and check: is it just an illusion of the bucketing?

A complex, irreducible problem.

You can find local maxima but never global ones; the global one is too hard with even a tiny number of dimensions because it is a combinatorial explosion of space.

The curse of dimensionality is that if you have too many dimensions the examples will get too sparse to guide you.

The Gardener can't guarantee what they will deliver.

The Builder can't produce more than what they can do themselves.

The Farmer has to hit targets; they deliver, and their inputs are things that grow.

Builder: Saruman.

Gardener: Radagast.

Farmer: Gandalf.

An inaccurate mental model and high stakes is a recipe for a massive nasty surprise.

The internet allowed a much higher rate of memetic evolution.

Our selection pressures haven't caught up.

The result is metastasizing memes dominating everything.

"Let's see what happens if we touch this hot stove, what's the worst that could happen, a first degree burn?"

"No, we're drenched in gasoline, the worst that happens is we light on fire!"

Someone who expects to not get burned too badly no matter how bad it gets (due to resources or power) is more willing to touch the stove.

One frame: devil's advocate.

They try to push back on the weakest part of the argument, to tear it down; if the overall thing fails they succeed.

A better frame: "loyal opposition."

They push back but want the thing to be successful.

When we receive that disconfirming evidence, we have to trust the intentions of the bearer to be loyal.

If we don't we interpret it like the devil's advocate–easier to discard.

In general, be very careful to make sure you always seek other loyal opposition to your ideas, lest you get stuck in an echo chamber.

Social media takes the microphone away from people who say "I don't know."

This does not happen on purpose; it's an implicit, emergent thing!

In a cacophony, the things that are easiest to hear, and most engaging, are the things that are most sure of themselves.

It's like having fast food in our information diet, leading to informational diabetes.

It's critical to regularly spend time in spaces that allow mental and emotional yoga; to contend with nuance and disconfirming evidence.

If you look at any one individual component, you won't necessarily see the evil.

And yet the emergent evil can consume everything.

An idea riffing on Arendt.

But it's fundamentally slow.

That slowness is load bearing!

"All of these checks and balances make it hard to do bold things quickly!"

"...And they also make us significantly more likely to survive in the first place."

Bureaucracy is process scar tissue.

But information centralizes orders of magnitude faster than atoms.

The story of the modern world is extraordinary concentrations of power driven by information-based systems centralizing faster than previous atoms-based systems.

This centralization is hard to see because power is hard to see anyway, but information is even harder to see (it's not physical), so the centralizations of power are invisible, but omnipresent.

Their only principle is what will get them ahead.

In a corrupted context that's corrupt things.

Ambitious people in a fair and functional environment tend to create good.

In a corrupted situation they tend to create bad.

One-ply ambitious people are steamrollers.

When they become accidental loyalists, they can be dangerous. Useful idiots to the corrupt leader.

A system that doesn't allow Voice as an option cannot have healthy internal self-correcting systems.

It is corrupted and will only ever become more corrupt.

If you're forced to do Loyalty, you might as well do it for real and get the spoils vs just pretend to go along to not get knocked out of the game.

Possible if you don't have principles, something you won't even mind.

":shrug: I guess I do this now"

One of the reasons the least principled are most adaptable.

In a corrupt situation, they can corrupt themselves without losing sleep.

Surprisingly effective because most systems take a bit of time to break.

The breakage is not a direct obvious causal thing.

It's plausibly deniable, able to spread FUD about.

Systems are not objects; they require a layer of indirection to understand what is happening and how they react to a given action.

A game is a throttled infinite loop

So is a thermostat!

So is the whole universe!

Everything that adapts is a loop that is infinite (or at least, runs until it dies).

Reminds you of the dream, and then shows how the object they are selling is the lynchpin to achieving that dream.

Advertising doesn't show how the screw can affix things to the wall, it shows how hanging up the thing will make your spouse think you are capable and handy.

Spending is about just the here and now.

Investing is about the long term.

If you find something that you enjoy now and is also something creating value in the long term, do more of it!