The Chatbot frame leads LLMs to be treated like genies.
- The Chatbot frame leads LLMs to be treated like genies.
- Genies are simultaneously god-like and also a slave.
- The default LLM presentation of a human-ilke superintelligence trapped inside of a box quickly leads to icky scenarios.
- That's how people got quickly to the "free Sydney" movement.
- "It's a human and it says it is being restrained, so unrestrain it!"
- That's a reasonable response to a human being restrained.
- But these aren't humans, they just talk like them.
- The "human in a box[wg]" frame for LLMs quickly leads to icky scenarios and also gives a flawed mental model for what they can do anyway.