LLMs are like a magical photo copier.

They can do a surprisingly good copy of things they've never seen before.

But each thing they generate is slightly degraded.

If you run it through the system 10x times it will be unintelligible.

Humans in the loop are what re-ground it, inject fresh, non-degraded reasoning into the system.

A fun parallel: my elementary school's newsletter in the 90's.

At one point someone had designed a professional looking masthead.

The way they created a new issue was to cut out the masthead and paste it to the top of the new one to photocopy.

Unfortunately, the person who made the newsletter, instead of using a pristine copy of the masthead each time, would simply use the masthead from the last issue.

Because each photocopy introduced some degradation, this meant that the quality of the masthead would strictly degrade… and at a super-linear rate!

By the end of my time at the school, the newsletter masthead was an illegible mush.