Training gives LLMs background knowledge.

· Bits and Bobs 10/7/24

Training gives LLMs background knowledge. Context gives them working knowledge.

Many people are worried about LLMs using their data in training, but there the leverage is way lower for the model creator.

Your data is just an anonymous particle of sand in a sandstorm.

It turns out that the value of the "querystream" as a data source is just not that interesting compared to all of the other data streams to feed LLMs, as evidenced by how willing the major model providers are to agree to contracts to not use any queries for training.

More on this topic

From other episodes