It's possible to hill-climb skills.

All you need is a loop and an evaluation rubric.

If you store the inputs and outputs so it can see the missteps, then you can have the LLM improve the runbook as it goes.

The main thing to be careful of, as with any hill-climbing process, is over-fitting.