You have to work to get disconfirming evidence from LLMs.
- LLMs are too eager to please.
- If you aren't careful they won't question you, even if you give it false premises.
- A trick someone told me: they sneak in a reverse Van Halen brown M&M.
- That is, an unreasonable request as a canary.
- The legend about Van Halen stating in their contracts that they wanted a bowl of M&Ms in their green room… but with no brown M&Ms.
- It was often interpreted as an example of the extravagance of an over the top, self centered band.
- But it was actually a kind of canary–if a venue had messed up that detail, then they should also verify that other conditions related to safety of rigging, etc that were specified in the contract were checked.
- The reverse Van Halen brown M&M for LLMs: ask it to grade the writing samples on something you know it shouldn't score highly on.
- For example, "is it laugh out loud funny?"
- If the LLM says it is, you know to look more carefully.
- A meta-note: I originally drafted this about "blue M&M's" and asked Claude if that was correct.
- It told me it was, "not brown M&Ms as often misremembered."
- I asked it "are you sure it was definitely blue M&Ms and not brown ones?"
- When I pushed it, It corrected its error and said that it's brown M&Ms, which I then confirmed with a Google search.
- A great example of the gullibility problem!