93 chunks · 64 episodes
...f the theory of the code. The code is the map, the theory is the territory. The ground truth of what it's supposed to do and how it's supposed to work. Vibecoding makes you less aligned with the theory of the code.[fq] The map vs the territor...
Metrics can be used to help ground truth your thinking but should never replace your thinking.
Claude thinks everything it outputs is brilliant. You need an external ground truthing process. Your curation and judgment. There needs to be a quality control entity in the middle that isn't eager for your approval. A sphincter posi...
In times of disruption ground truthing is even more important. When you don't have a permeable membrane then you're a self referencing system. You're not ground truthed. "Are we good?" ...
...believes, producing a never ending stream of confirming evidence, no matter the ground truth. In a cacophony of information, you can choose which subset to pay attention to. Our brains make the implicit assumption that what we see (a subset) ...
...if you have a very long runway. The risk is that the longer you haven't touched ground truth the more likely you are to never be able to touch down again, at a compounding rate. You get more and more out of touch and then when you try to touc...
...arring match because you're right rather than because you're powerful helps you ground truth your beliefs. His manager saw the value in that for truth-seeking.
The feedback loop on model improvement is "how long does a ground truth check take" Do you need to talk to a human? Do you need to run a physical experiment with expensive inputs? If it can be done entirely at computer sp...
...ost of the input is junk. In 90 iterations, all of it is. LLMs decohere without ground truthing with real world results. That can be done automatically with things like React components. But anything that is the least bit complex has to be pu...
...iver. You can still get some emergent signals, but they are warped away from a "ground truth" of quality and can get warped by political dynamics. If there's no cost to them doing it, then the signal is superficial and performative. Meaning c...
... that doesn't have to fight to survive succumbs to kayfabe. The larger and more ground truthed the organization, the longer this process takes.
... time with LLMs than people, you could decohere from society and truth, with no ground truthing of "no, I don't think that makes sense…"
...ative even stronger. The only way to counteract this is to have the org have to ground truth constantly. An org that is so successful that they can operate poorly for years without dying but that fires people easily will have the worst of bot...
... motivated views of the same scenario, which makes it easier to triangulate the ground truth.. Perhaps Granola style transcripts could help ground truth? But only if everyone's actions didn't change given they were being recorded. Do we try t...
...ledge for what you need. Too much fidelity, not enough doing. The doing is what ground truths and helps figure out where your mental model is incorrect. It's the same asymptotic curve of perfection that is a bad idea in building products. Ove...
... unintentionally). Who should be the singular arbiters of the infinite, perfect ground truth? There is no answer. "Balancing" can itself also be bias. A point someone made this week: "If one candidate's policy was to encourage infants to eat ...
...limit. Companies are a swarm of employees making interdependent decisions whose ground truth is the business model. Ultimately the ground truth about what a company cares about is the business model. Everything else is just platitudes.
In a post truth world ground truth doesn't matter in any given interaction. But of course the ground truth never goes away; it will matter at some point. The first person who points ou...
...olitical players. The only thing that contains the expansion of politics is the ground truthed success of the company in the wild. Imagine if you had a swarm of LLM-backed agents in a company. They could conceivably absorb infinite energy, al...
LLMs will get good at everything that can be quickly ground truthed by computers. But that doesn't mean they'll do a good job at things that can't. Most meaningful things to humans can't be ground truthed entirely i...