A measure of the novelty of a thinker: how hard their insights are to predict.

· Bits and Bobs 1/21/25
  • A measure of the novelty of a thinker: how hard their insights are to predict.
    • If they are easy to predict once you categorize their worldview, they aren't that novel–you can accurately predict what they'll say based on the category they fit in.[afv]
    • A similar test could be applied to LLMs: the novelty of an utterance has to do with the inverse of the likelihood[afw] that that next token would be predicted by the LLM based on the preceding tokens[afx].

More on this topic

From other episodes