This week I learned about the Scots Wikipedia controversy.
- This week I learned about the Scots Wikipedia controversy.
- Scots is a language with a small number of speakers.
- A few years ago someone on Reddit noticed that the Scots Wikipedia had a high number of articles written in poor Scots.
- It turns out there was a particularly prolific American teenager with a rough understanding of Scots who had written a large number of the articles.
- Because of the prominence of Wikipedia relative to other Scots material on the web, it formed a larger amount of the LLM context on Scots.
- That means that LLMs also likely replicate Scots poorly, all because of one weird bottleneck.
- A similar kind of thing happens in evolutionary biology, a "population bottleneck."
- That's when for some reason only a small number of individuals of a species survive (or travel to, say, a new island).
- That means the rest of the species has that particular random set of individuals as ancestors, inheriting its random subset of distinguishing characteristics.