Bits and Bobs 10/9/23

We knew it was where the core of the value in the company came from.

That team was known to be high performing, and use operationally mature, industry-leading approaches to the problem.

In general the core of your differentiated value--the enabler of your differential value--should be a center of excellence.

Your center of excellence should be a point of pride inside and outside the company.

Most of your effort should go into becoming even better at the driver of your differentiated value.

For everything else, the bar to clear is not approaching perfection, but "good enough".

You're likely already familiar with induction and deduction. Abduction is their lesser-known cousin.

Abduction is looking at details and trying to surmise a broader explanatory hypothesis out of them.

Whereas induction and deduction are more about certainty, abduction is more about probability.

The type of reasoning Sherlock Holmes did is a good example of the technique.

A canonically strong intro is this old Ribbonfarm essay: Startups, Secrets, and Abductive Reasoning

Abduction helps you generate hypotheses that might be true, that you can then bet on and experiment with.

If your experiments and incremental investments in the bet turn out to have good returns, that gives you increasing confidence your hypothesis is correct.

Abduction can help you make bets in places where you don't have full certainty.

Sometimes what they're doing will look crazy!

But assume that they see something others do not. They're crawling through broken glass based on the strength of that conviction and the value the behavior creates for them.

Look at what the user is doing and try to steelman it. What if their use case is not some random one, but a general one shared by many users, where this one user is the only one with a sufficiently high pain tolerance?

That is, abduct a hypothesis out of the use case for the kinds of things that might be valuable based on the existence proof of one ad hoc real world use case.

Many of those things you might build based on that hypothesis might be high-risk if the hypothesis turns out to be wrong. But some subset will be things you already knew you wanted to build.

For those no-brainers, the abducted hypothesis "this might turn out to be a powerful new user segment" adds a bonus so the no-brainer use case has more expected value, so now it's worth doing.

You do those no-brainer features, knowing that the worst case if the hypothesis is wrong you will have unlocked some value you knew about from other use cases. But if the hypothesis was right, then you should see greater-than-expected value unlocked. That's a clear signal to keep executing on that hypothesis.

This is part of the mindset and playbook I often call the doorbell in the jungle.

But it's nearly certain that any given game-changing result came from a novel idea.

When reviewing a document, the most useful thing you can give is a synthesized, high-level view:

"I found this analysis convincing for X, Y, Z reasons and as a result I think we should do A."

"Although I want this conclusion to be true, I think the weak part of the argument is X and unless we have more confidence in that we shouldn't change our plan."

Google Docs has an amazing comment feature that allows people to make concrete comments about specific runs of text.

Google Docs comments are great for detailed questions and observations. They are not great at broad, synthesized points.

Leaving a Google Doc comment on a detail is a good way of demonstrating "I am engaged and paying attention, because I am even spotting small things in the details".

In the limit though the performative, superficial aspect of "I'm paying attention" dominates the more fundamentally useful "expressing useful feedback" use case.

As a result feedback in Google Docs (and similar tools) tends to be more nitpicky and overly-detailed than it might otherwise be.

Afterwards I had a few conversations with folks about some of the downsides of an A/B testing infrastructure.

One person described a downside as "p-hacking on steroids"

Another downside is that it can create worse externalities.

A/B testing is a great way for hill-climbing a metric. A/B testing accelerates Goodhart's law.

If there are externalities that are not incorporated in your measurement loop (which there fundamentally must be) you might find that climbing the A/B test hill is actually creating small amounts of value on the hill but significantly more negative externalities.

Worse, these externalities, being outside of your measurement loop, will lurk and be invisible to you until they fully consume you.

You can create the illusion of it, but only in limited contexts for short periods of time.

Maintaining the illusion gets super-linearly more expensive with scale and time.

The illusion of perfection can be shattered in an instant with a single visible imperfection.

Sometimes someone gets their start with something that is nearly perfect, which means they will feel the desire to keep that perception going as they scale, since it was the wedge that got them momentum in the first place.

But trying to maintain that illusion becomes an all-consuming, oppressive cage that distracts from everything else you could be doing.

It's better to have beautiful fundamentals and messy optics than beautiful optics and messy fundamentals.

It takes humility to acknowledge that you aren't perfect, but doing so frees you up to add true value.

Said right before a fatal accident: "Everyone says that I need to wear a seatbelt for safety, but I haven't ever worn one and I'm just fine. They're just being a bunch of wet blankets!"

In the world of atoms, you have to invest significant energy and time to move assemblages of atoms in space.

In the world of bits information can be replicated and transmitted with ~perfect fidelity and ~infinite speed for ~free.

A lot of things that make us in the tech industry feel like geniuses actually arise mostly from "the world of bits is way easier"

"I think it can be so easy to be like 'if only there were no org dynamics' then things would be great but really, past a certain number of humans, org dynamics are the problem. And if you start seeing it as a systems problem rather than an individual person problem it is (a) much more interesting (b) much less personally taxing!"

There's an asymptotic curve of usefulness of information vs cost to get it.

If you're waiting to get to 90% you're waiting too long and investing too much effort.

More information feels superficially like it's always a good idea, but not once you incorporate the cost of getting it, it's much less clear.

If your comment in a doc reduces to "Can we double-click on this to be extra sure?" consider that you might be actively detracting value.

But rigid things have a harder time adapting.

In a changing context, adaptation is necessary to avoid death.

The higher the pace layer you operate in, the more quickly the context will change, and the more important adaptation will be.

Software (bits vs atoms) is at a higher pace layer than most other industries.

But the vast majority of hard things are challenging for totally mundane reasons.

Every fork in the road, especially when it's not clear which way to go, is an opportunity to throw up your hands and be at a standstill... or even give up.

Navigating uncertainty is a slog!

Getting someone to be intrinsically, authentically excited about a thing they weren't excited about before is extremely hard.

It's easier to find people who already happen to be intrinsically motivated by that core problem (and then fan the flames) than to try to make them excited in the first place.

That is, don't make people who are excited about the problem, find them.

But what's most important is the fundamentals, which hard harder to see.

The superficial aspects of a successful big company are things like spreadsheets, program management, and processes.

The fundamental core of what makes a successful big company is strategic clarity and a systems approach to running organizations.

Watching someone do a superficial copy is like watching a high school production of Our Town, where high schoolers play the role of the elderly characters.

Getting caught in the filter knocks you out of the game.

Successfully passing through the filter doesn't feel like anything. You just didn't die, just like the previous time steps.

You can only notice passing through a filter only by looking around you and seeing how many fellow travelers were knocked out of the game.

It's easy to not notice these because when someone's caught in the filter it's often not an attention-grabbing explosion but a quiet bowing out.

Imagine that all of the entrants are working hard but only some survive. Making it through the filter is in that case somewhat due to luck.

You can see your own hard work, but you can't see the luck required to pass through the filter.

So the winners who make it through the filter often erroneously attribute it primarily to hard work, even if luck was a large component.

That day for synthesis and reflection was the source of nearly every game-changing idea, an enormous source of leverage.

A lot of people over the years have told me "I wish I had the time to do that, too."

I didn't have the time to do it--I made the time to do it.

If you don't push back on it, the low-leverage work will expand to fill every single inch of space you give it.

A primary use case is one whose expected value for a user is greater than its expected cost.

This difference gives a gradient of activation energy to lead to more adoption.

The gradient need not be large; it's better to be dependably a little above the bar than sometimes way above the bar but often below it.

A secondary use case is one that doesn't qualify as a primary use case, perhaps because:

The quality can be great but is often below the bar

The use case isn't good enough to activate usage on its own but can ride the coattails of another feature

You can't lead a value proposition with a secondary use case; it's not dependable enough.

But often the primary long-term value arises from the secondary use case (especially when there's some kind of self-improving quality of it, e.g. a network effect).

You have to lead with your primary use case and sprinkle in the secondary use case as a bonus, at least to start.

Ethan Mollick has done a lot of research about this, for example in this old Ted Talk.

Accelerated expertise is a way of abducting playable games based on experts' intuition to help learners build knowhow in a given space much more quickly.

Most games have to have "fun" as a primary use case, and "learning" as at most a secondary use case to be viable. But games in the work context that people must play (e.g. the excellent simulations by BTS) can flip it and have the learning be the primary use case and the fun be the secondary use case.

The reason games are so hyper effective is because they require the participants to be active.

Instead of being able to passively watch a situation, they are called upon to make active decisions.

To take actions means they have to be engaged and have a mental model of the situation.

Building a robust, useful mental model is what builds knowhow.

So games create knowhow as a kind of byproduct, and that byproduct can be intentionally created and harnessed in organizations by creating games.

Within small groups, everyone can see a lot of the nuance and indirect effects of each others' actions up close and over time.

But in larger organizations, it's not possible for everyone to have that level of direct awareness of the nuance in every case.

A leader of a 1000 person org fundamentally can't know the hopes and dreams of every person in their org!

The way to be able to make decisions in those contexts is to create summary statistics, to deliberately do dimensionality reduction to collapse complexity.

This collapse is necessary, otherwise you would be frozen in place with no clarity on what to do.

Hopefully the summary statistics capture the core reality in a directionally useful way, but they can fundamentally never be a full-fidelity version.

This difference between the rich nuanced reality and the summary statistics view is what creates the space that the emergent force known as Goodhart's law will inexorably fill.

It's not possible to navigate the whole territory without a map. But never forget that the map is not the territory.