I thought this tweet about RLHF and taste was insightful:

· Bits and Bobs 2/24/25
  • I thought this tweet about RLHF and taste was insightful:
    • "the problem with RLHF is that a lot of humans:[xz]
    • A. lack taste
    • B. have different tastes
    • A makes it bad, B makes it average"