Software written by LLMs is merely good, not great.

If it's small and similar to existing software, it's typically good enough.

But if it's larger, or unlike existing software, it's less likely to be good enough.

Imagine a system that pieces together novel software by assembling sub-components.

The sub-components could have been written by:

1) humans in the past (highest quality)

2) written by LLMs and approved/tweaked by humans (moderate quality)

3) hallucinated by LLMs on demand. (lowest quality)

If a sub-component of the third class is used to compose a larger bit of software that doesn't work, a 1% kind of user who is comfortable programming could pop the hood and tweak it.

That tweaked version can now be put on the shelf for everyone else to benefit from in the future.

This creates a kind of self-ratcheting quality.

The improvements by any human in the past improves the quality of the software generated for every human in the future.