What is the value of proprietary information included in the training of an LLM?

· Bits and Bobs 1/13/25
  • What is the value of proprietary information[ake] included in the training of an LLM?
    • That information helps the LLM perform better, but how much?
      • How much worse would the LLM be if you hadn't included that marginal bit of data?
      • Would a human even notice?
    • Someone pointed out that the Shapely Value might be a useful conceptual lens to try to get a handle on this.