LLMs used to be hard to get good code out of.

· Bits and Bobs 4/28/25
  • LLMs used to be hard to get good code out of.
    • They could do it, but with a lot of prompt, workflow, and UI scaffolding to give a dependably good result.
    • But RLAIF works well for writing code (especially React components) since it's easy to construct an auto-ground-truthing pipeline.
      • Write the code, try to run it (iterating until no errors) then using Playwright to visually inspect it and poke at it according to a test plan to verify it works as intended.
    • This means that models, like Sonnet 3.7, have gotten much better at writing React.
    • If your secret sauce that gives you an edge is scaffolding to wring out better coding results, your moat could be evaporated by the next model update.
    • Sonnet 3.7 made a number of vibe prompting products possible.
      • Sonnet 3.8 could make them obsolete.

More on this topic

From other episodes