How much you trust a suggestion is partially due to what context the system is drawing on.

· Bits and Bobs 7/7/25
  • How much you trust a suggestion is partially due to what context the system is drawing on.
    • Is it drawing on relevant facts about you?
      • Is it missing important ways that you differ from the general population?
      • Is it including irrelevant things that will distract it?
    • The quality of the context is a big determinant in how good the results are.
    • This is one of the reasons the ChatGPT dossier memory feels off to me.
    • You can't inspect the context to say, "include this, not that".
      • The chat is append only, not coactive.
    • You can imagine a UX where there's a coactive context drawer at the top of the interaction.
    • Things in the interaction and that context drawer are all that is given to the LLM, nothing else.
    • When the drawer is collapsed it shows a short summary.
    • When you expand it you see trees of context that you've pulled in explicitly.
    • You can add in trees of context on demand easily.
      • E.g. "Include information about my nuclear family."
      • The tree pulls in all of the sub-items that hang off of it.
      • You can choose to pull in something high up in the tree or low down.
    • You can delete any tree of context that's not relevant.
    • There's also a list of auto-included context that the system guesses is useful.
      • Those are included by default, but can be explicitly added to the included items, just like if you'd added it yourself, or deleted.
    • The ranking function is: how well does the system predict which trees of context to include, (predicting whether the user would accept or delete a suggestion?)
    • The user choosing to include or excluding bits from the suggested context is an extremely powerful ranking function.
    • The user wouldn't even realize that they're training the system for themselves and others by gardening their context.
    • Only a small number of users would need to do it to help tune it for a whole population.

More on this topic

From other episodes