LLMs are best used to fill in a scaffolding a human sets up.

LLM generated stuff is like sand.

A good space filler, but doesn't have enough shape on its own.

You can create sandcastles, but only up to a point.

Past some critical point, the sand can't hold up weight, or even its own structure.

But if you put the right scaffolding into place and use the LLM sand to fill it in, it can be extremely strong.

With the right scaffolding, the shape is default-converging.

Just fill it up with the right material and you're done.

You can make amazing structures out of gap-fillers if you put them in the right structure.