The context and the model are too powerful in combination.

· Bits and Bobs 6/2/25
  • The context and the model are too powerful in combination.
    • The foundation model has the power of all of the world's knowledge, using alignment imposed on it by its creator.
    • The user's context is an extremely powerful memory about them.
      • In the wrong hands, it's a dossier.
    • Together, they create the possibility for exceptionally powerful manipulation… or blackmail.
    • If everyone were able to be manipulated or blackmailed by one entity, that would be one of the most powerful entities ever created.
    • It's imperative that those two things not be combined.
    • By splitting the two layers, you give choice and competition at each layer.
      • You allow alignment with users at the context layer.
    • Perhaps a useful regulation: the creators of foundation models cannot host an experience themselves that stores user context.