The prevalence of delve hints at the danger of hyper concentration around hyper-centralized models.

Presumably one of the human raters had a slight preference for the word "delve."

But a consistent bias in the RLHF data can create a bias that stands out from the noise.

Now multiply that one model by billions of interactions, and suddenly that one person's preference for delve has society-wide effects.