A chaos engineering methodology only works with the backing of a senior, empowered leader.

  • A chaos engineering methodology only works with the backing of a senior, empowered leader.
    • The idea is that you enact things like chaos monkeys to randomly break things.
    • This forces everyone to design their systems to be resilient–and not just in theory but in practice.
    • But imagine trying to retroactively add this to an established, risk-averse company that hasn't previously done it.
    • The first thing the chaos monkey breaks is likely to bring down the whole system for real!
    • If the chaos monkey breaks the system, who gets in trouble, the person who created the chaos monkey or the person whose system was broken by it?
      • This is especially true for faux chaos approaches where the monkey is not random but pre-scripted by a gamemaster.
      • Similar to the trolley problem, presumably that gamemaster is easier to blame than someone who set up a chaos monkey.
        • "I wanted you to do chaos engineering, but not on the homepage on our third biggest shopping day!"
    • A lot of chaos engineering is the powerful engineering VP saying, "I will set up chaos monkey and if it knocks over your thing, I won't be fired... you will."
    • Which leads everyone to engineer defensively. But requires a person everyone knows is powerful enough for that flex to be true.