Maybe the trick to fixing hallucinations is to penalize guessing.

Makes sense to me.

Like in the SAT, where the scoring is set to deliberately penalize guessing.

Presumably you'd want the penalty to start low and ramp up as the models get deeper into training.

If you penalize guessing, hallucinations become structurally less likely.

A consistent bias in a noisy system.