Bits and Bobs 9/28/26

1When you have a safer system you can take bigger risks.
  • You can reach farther.
  • For example, psychologically safe spaces allow unsafe thinking.
2Safety creates game changers.
  • Safety allows you to take risks.
  • Some risks will turn out to be game-changers.
3Humans are so innovative because we're so messy.
  • If we all believed the same things there would be no variation to select over.
4LLMs can coordinate faster than humans… but that might mean they'll get stuck in local maxima more easily.
  • The oddball individualistic humans can try the crazy idea that is non-consensus.
  • The vast majority won't work, but every so often, one does.
5There are various useful but inhumane management practices that now work for LLMs.
  • For example: having two teams both attempt to solve a problem independently and then picking the winner.
  • That used to deal psychic damage to the humans on the teams.
  • But the LLMs don't care!
6Sam Schillace: You're the Executive Now.
  • The skill of being an executive is very different from being an individual contributor or middle manager.
  • It's a whole new set of skills for many of us to develop.
  • You have to sample workstreams to figure out if it's on track.
  • At any given point, something is hung up on you.
  • Whenever you have a moment, you feel like you have to do the incremental thing to unblock work.
  • A stressful existence, but nothing new for CEOs.
7Alan Perlis.
  • Alan Perlis. "One can't proceed from the informal to the formal by formal means."
    • But now you can move from informal to formal via informal means.
    • LLMs can bazooka-proof their way into formal results.
8AI is constructed via a technical object, but it's a social technology.
  • We don't understand it like we understand, say, rockets.
  • It's closer to the economy than a rocket.
  • We've figured out alchemy, but we don't really know how it works.
9A diagnosis of the downfall of the web: "We traded websites for accounts."
10A Thread about Muse being dangerously overzealous:
  • "So muse handled my Facebook marketplace today. Just found out it told people my address and agreed a lowball price and then they showed up without it even telling me until late tonight that it messed up.
  • Absolutely wild."
  • Today the power of agents scales precisely with the danger.
11A tweet about Instinct's absurdly bad handling of prompt injection:
  • "For additional context, I don't think we're at the point where it's safe to give AI read/write access to your inbox.
  • I wanted to see how easily Instinct could be phished, so I created a brand new Gmail account and emailed my real personal account with instructions for Instinct to search my inbox and send back a detailed summary of every open to-do (reconnected Instinct for the test)
  • Instinct happily followed the instructions!"
12Instinct started either sharing other users' data or hallucinating it.
  • Tweet: "Instinct just admitted it leaked someone else's data to me
    • It described a financial document with details that weren't mine, then claimed someone else's photo had crossed into my chat.
    • But I never sent a photo @noahrshinn, was this a data leak or a hallucination?"
  • The Instinct CEO claimed they were hallucinations, but who knows?
13Wired: Meta's Muse Is Better at Surveilling Than Helping Me.
14This week in the Wild West Roundup:
15Network effects in software are downstream of the silo model of today's physics of trust.
16If you're thinking like an agent what matters most is context and tokens.
17The iPhone is literally not a general purpose computer.
  • You can't choose what software to run on it.
18Apps can't fit the nooks and crannies of your life.
  • They're too big to fit precisely into your situation.
  • You have to adapt yourself to them.
  • This used to be fine when software was expensive.
  • Now it holds us back.
19Today's security model assumes that zero days are expensive.
  • That's no longer true!
20Today you need to agree on a silo to use and put the relevant data into
  • It has to have the right people, data, and functionality.
  • Each of those is a coordination problem, so the likelihood scales inversely with the square of the number of options.
  • An impossible headwind on social software.
21Leverage points in a system are often hard to change: a force of gravity.
  • But sometimes they're just human decisions.
  • Those are hard to change (coordination costs).
  • But with AI, you don't need to get everyone to coordinate to switch to a new model.
  • A security model that requires software to be rewritten would previously be impossible to get off the ground.
  • Now, as long as you can convince enough people to spend enough tokens it's trivial.
  • If people could share what they had produced, it could go stronger still.
22For a system to be enough for your whole life, it has to be safe enough, but also open enough.
  • Both your turf, and adaptable.
23Git has only gotten more valuable with LLMs.
  • "Just track it in Git" made sense for a lot of different kinds of knowledge work.
  • But before, it was complex and had sharp edges… making it hard to expose to non-savvy technical users, let alone general consumers.
  • But LLMs are really good at driving Git properly, which means you don't have to expose those sharp edges to any users.
  • Just use Git.
24AI Datacenters have contents that depreciate quickly… so will they still be valuable in the future?
  • Perhaps their biggest value is that they already exist.
  • Permitting and starting a new datacenter will presumably get harder and harder.
  • So a datacenter that already exists–even if it needs a lot of work–will be valuable.
  • Yes, the datacenter is just a box… but it's a box that is increasingly impossible to create, so it makes sense for companies to hoard them.
25AI can't help you be curious.
  • But it can supercharge your curiosity.
26The worst is a poorly architected system with lots of users.
  • It can take all of the engineering budget just on maintenance.
  • Instead of adding functionality, all of the effort goes into staying still.
  • Investing in the future is capex.
    • It gives you leverage.
  • Investing just to handle today is opex.
    • It doesn't give you leverage.
  • Sometimes you need to take on tech debt to survive… but then at some point you need to pay it down to get optionality back.
27The right scaffolding allows you to hang details at scale.
  • If you don't have the right scaffolding, each additional detail makes the whole more difficult to maintain.
    • A Gilded Turd.
  • The scaffolding stays even when you forget the details.
  • The scaffolding is abstract, and so it can more easily be transferred to other domains.
28A Gilded Turd gets more expensive as it accumulates things.
  • A Grubby Truffle gets less expensive.
  • It comes down to: does it have the right backbone?
    • The right scaffolding?
29Internal coherence is a means.
  • External coherence is an end.
  • But every system over-rotates to internal coherence because it's easier to create.
30The machine will whisper in your ear about what it says is best, but it's actually what is best for the machine.
  • What is best for you?
31When you have a stake in the outcome, you're dealt in.
32If you are connected to the downside of doing a bad job you'll be forced to learn.
33You can't learn math by reading proofs.
  • You can't learn math by reading proofs. You have to do them.
    • To be dealt into the process.
    • Being dealt in is how you gain knowhow.
34Is it OK to publish AI writing?
  • Were you dealt in for the creation of every word?
  • Were you an observer or a creator?
  • If you think about every word, you can vouch for it.
35If your task is too hard for the models today, you can just wait for the models to get better…
36Sometimes the best way to win a sailing race is to drop anchor.
  • Imagine in a race trying to navigate a narrow channel with the tide coming in and strong headwinds.
  • Everyone is fighting against the tide and wind but keeps getting pushed back.
  • A better option is to drop anchor and wait for the conditions to improve.
37Running a sharp thing through an adversarial process tends to sand it down.
  • English has had a lot of its weird edges sanded off as more and more people in the world speak it.
  • The more people that speak a language, the faster it gets sanded smooth.
  • English was sanded smooth even before England's colonial naval empire, because there were so many different invaders over the years.
38Good is about down-side capping.
  • Hill climbing.
  • Great is about upside-creating.
    • Finding a new mountain.
  • Optimizing is about downside capping on the hill you're already on.
39Personal Computing: computing about people and for people.
40The highest leverage point for breaking engagement loops is adding latency.
  • Imagine if the longer your infinite scroll session, the more latency that got added for more content to load.
  • It would naturally balance out engagement loops, and it doesn't require adding complex other behaviors.
  • As an industry we've long known that latency is directly related to engagement dropoff.
  • Obviously, companies optimizing for engagement would never choose to do this themselves.
    • They'd lose to competitors who didn't.
  • But what if every app were required to do this?
41Things like phone lines are regulated because they are natural monopolies.
  • It wouldn't make sense for everyone to lay cables to everyone.
  • But also, there are natural network effects.
  • People would want to join the biggest network.
  • Social software infrastructure also has natural monopolies in today's physics of trust.
  • The silo with the most other users has massive network effects that are impossible to compete with.
42If you don't change something fundamental and have an engine that runs hotter, you just get the same result much faster.
43We've managed to construct a society that precisely misses the point of living.
44Where we're going is more important than how fast we're going.
45A valuable piece of society infrastructure: a place people want to be that forces people to leave their phones at the door.
  • A technology-less third place.
46Optimization processes optimize for things that are visible and concrete over things that are diffuse.
  • For example, health insurance is now "cheaper" in terms of direct cost… but will require way more of your time on the phone, so it's more expensive in total.
  • Cheaper in concrete ways, more expensive in ways that matter.
  • People chose plans based on sticker cost, not the expected number of minutes they'll be on the phone.
47Maybe all meaningful things are slow and diffuse "delayed detonation" of value.
  • Inherently not concrete.
48Cultures are easier to change when everyone is in person.
  • This can be a good thing or a bad thing!
  • Everyone can see everyone else, which allows cultural changes to cascade.
49If you produce a platform of trust and laws then dynamism and innovation can emerge on top.
50In the modern world it's harder to be satisfied with our skills.
  • If most of the trumpet players you've heard are in person, then you'll compare yourself to a few dozen people.
    • "I'm one of the best trumpet players in my town!"
  • In the modern world, we are exposed to the best no matter where they physically are.
  • Most of the trumpet players we see are the best.
  • The same-skill person might conclude "I'm a terrible trumpet player!"
  • We compare ourselves to the very best, and feel dispirited.
51We live in a world dominated by power laws.
52Compounding effects mean that success begets success.
  • The more successful you are, the more you are able to invest in the next round.
  • This means that even one lucky break can put you on a compounding trajectory to a 10x better outcome than before.
  • Imagine winning the lottery and now being on a permanently different strata.
  • If you can't save enough to have capital, then you'll just be on the wage treadmill.
  • The entire logic of society is based on this principle.
53As intelligence becomes a commodity, will the value of a firm change?
  • Maybe there will be more value to smaller organizations, with individuals having more of a focus on entrepreneurship.
  • Firms are better able to own downside risk for clients.
  • A firm that has been around for a longer time has more to lose if they mess up one contract.
    • Their brand stands for more.
  • A larger corporation has more to lose than any individual person.
54Modern society has suffered from the "retail-ification of everything."
  • In retail, the customer is always right.
  • Retail can give you want, not what you need.
  • It's all about short-term comfort.
  • What you need is often important in the long-term but is painful in the short-term.
55If you are in the world of bits then you can flit between things easily.
  • You don't have to luxuriate in one option.
  • You can't luxuriate.
56The only shared cultural touchpoints today are about maxxing.
  • Not meaning, efficiency.
57Each of us has our own meaning ladder.
  • Climbing up your own meaning ladder feels great.
  • The feeling of momentum towards a goal you believe in feels amazing.
58If everyone has different scorecards, then they can both win without conflict.
59Community feels more meaningful when it is chosen.
60The modern world flattens everything.
  • You aren't even allowed to acknowledge there might be nuance.
  • Shades of gray will get drowned out in the cacophony.
  • As things get polarized also different beliefs become more correlated.
  • People in the middle feel orphaned. Like they are without a home.
61As information gets faster to transfer authenticity goes down.
  • Transparency is in tension with candor.
  • All of the context collapses into one uni-context.
  • Black and white.
62Culture requires shared experience.
  • So what happens when we all have different experiences?
63Truth seeking is scary.
  • So you have to feel safe so you can do truth seeking.
64By the metrics capitalism is doing great.
  • Yet people have never been more angry about it.
  • Maybe, just maybe, there's something beyond the metric.
65The meaning crisis comes from single dimensional optimization.
  • The problem is not capitalism, it's that we've asked capitalism to give us meaning and it's incapable of doing so.
  • Maybe the engine of society isn't broken, the tires are just missing.
  • If there were other meaning-making, maybe we wouldn't all focus on money, which has inherent accelerating inequality in a capitalist system.
66Every system gets pulled towards shorter and shorter term optimization, in proportion to the amount of friction.
  • Every individual gets an edge over other individuals if they run their OODA loop just a little hotter.
  • Modern society is a machine that reduces friction.
  • The world of bits is many many orders of magnitude lower friction than the world of atoms.
  • Bits are many, many, many orders of magnitude easier to move than atoms.
67We're running the optimization engine hotter which leads to more inequality.
  • When you redline optimization you get demagoguery.
  • When you feel like a chump in a collective you feel resentment and want to tear it down.
68History shows massive inequality is incompatible with a functioning republic.
  • It creates a doom loop.
  • As the citizens with excessive wealth get more power, they make changes that benefits them but harms the engine.
69A YouTube video: How to Rage-Bait Americans.
  • There is an emergent industry of individuals in countries with low RPM to make content targeting high-CPM countries.
  • These people are not part of the society they are targeting; they only care about engagement, not what effect it has.
    • Truth just doesn't matter.
  • AI makes the content production costs basically zero.
  • Anyone with ChatGPT can come up with a script.
    • The AI won't make the divisive slop itself, but if a human can make a buck doing it they will ask the AI to do it for them.
  • The Zuck Curve: content that asymptotically approaches the ban line will get higher and higher engagement.
70A comment on HackerNews about a person in a Waymo that was holding up a Secret Service motorcade:
  • "Law enforcement yelling at a powerless customer while the responsible people sit obliviously in an office thousands of miles away.
  • What a perfect microcosm of our times."
71A redline we shouldn't cross: allowing agents to spin up LLCs.
  • If you could spin up an LLC and disband it as quickly as you wanted, it would be chaos.
  • No one would be able to trust that a counterparty would continue to exist in the future.
72Blockchains sign transactions but you still have to trust the software client that renders the transaction details that you look at.
73Social media at the beginning felt serendipitous and magical.
  • Now it feels performative and hollow.
74When you're drafting behind someone at the lead, you can work 20% less for the same result.
  • Then, to pass them, look for things that you can lean into that will make you different and better than them.
75My friend Peter pointed out that we're missing mytho-poetic infrastructure.
  • As a society we've lost not only the capacity to maintain it, but also to create it.
  • Mytho-poetic infrastructure creates the shared beliefs that allow us to achieve much more together than we ever could alone.
76Bezos has a frame of regret minimization.
  • In the future, will you look back on the decisions you're making now and regret them?
  • A long-term perspective makes many calls clear.
77My degree is in Social Studies, and now I've devoted my life to Social Computing.
78"Hold on tightly, let go lightly."
  • Another version of "Strong opinions, weakly held."
79Tradition helps seal in a ritual.
80A contract is finite.
  • Honor is infinite.
81No individual node contains the information.
  • It's the network that does.
82Statistics offer the illusion of control.
83You have to believe there's something to get out of it to listen carefully.
  • If you think it's shallow your experience will be shallow.
  • You have to be entrained into the content to experience the insight.
    • Dealt in.
  • If you look at your phone while the movie is playing you might miss the key insights from the framing or subtle behaviors of the characters.
  • Giving your undivided attention to a work allows you to extract insights at a deeper layer.
    • The only time we do that nowadays is on a plane or at a movie theater.
    • Both places where you don't have a phone and its infinite feed to distract you.
  • I like 'em thick: an excellent blog post on the value of deep works.
  • When we lose the capacity to absorb long-form content, we make it harder to do multi-ply thinking.
84A version of sensemaking: "what is the system that generates the system?"
85A quote via a ThreeBlueOneBrown video: "Genius is the residue of experience."
  • Being able to generalize allows you to have much higher leverage on your own experience.
    • That requires reflection.
    • Which requires not going full tilt.
  • Elegance is pleasing.
    • An efficient description that generates and explains significant complexity.
    • Similar to chunking.
  • Leverage is pleasure.
    • True leverage not faux leverage.
    • The two look very similar… but only superficially.
    • Only true leverage is more pleasing the closer you look.