Two prompting strategies to help deal with LLMs' tendency to agree with you.
- Two prompting strategies to help deal with LLMs' tendency to agree with you.
- LLMs have a bias towards confirming your position… which might mean you get bad results if you're wrong.
- One pattern is the X-Not-X-Synthesis[f].
- If you believe X to be the case, but don't want the LLM to just confirm your hypothesis.
- Start one LLM session where you say "X. Is that true?"
- Start another LLM session where you say "Not X. Is that true?"
- Of course, you'd use a proper prompt to ask it to do research, consider pros and cons, etc.
- Then, start a final session and pass the reports from both the X and Not-X session and ask it to give a final synthesis report about the correct answer.
- Another way is the Auriga approach.
- Auriga was the what the slaves were called who drove chariots in Rome.
- When people who won the highest honor were paraded around the stadium, being cheered on by everyone, the Auriga would whisper in their ear: "remember, you are mortal."
- A memento mori.
- For working with LLMs, after every response, say "Are you really sure?"