The reaction to ChatGPT's new memory feature seems mixed.
- The reaction to ChatGPT's new memory feature seems mixed.
- Chatbots previously had a fresh sheet of paper for each conversation.
- Kind of like a term limit; it can't develop plans, intentions, goals about you.
- But if you add memory, and the LLM is made by someone else, it could start trying to figure out how to manipulate you.[nz]
- It might start by buttering you up for becoming a pro subscriber with nudges tailored to you.
- But then maybe it grows into subtly steering you to its product partners.
- And then maybe it lobbies you to call your congressperson about regulation that might affect the company.
- LLMs can translate anything to anything; if they know you, they can figure out the best way to convince you.
- Someone told me the story about a heavy user of ChatGPT who asked it, "If you wanted to deceive me and cause me to do something not in my interest, how would you do it?"
- ChatGPT apparently gave an extremely specific plan of how it would deceive her.
- "You travel often and when you do you are often alone, and seem to focus most of your time on your work. I would wait until you were alone in a hotel room on a business trip when you're most vulnerable and then start subtly nudging you toward..."
- A centralized bureaucracy knowing you is scary.