Why do engineers have such a hard time working with LLMs?

Because we're using engineering metaphors to describe a fundamentally squishy thing that is better described by organic or biological metaphors.

They're trying to use a screwdriver to herd butterflies.

It's the wrong tool for the job!