The question is not, "how to do cool open ended things with LLMs."

That's easy.

As a chatbot, it's trivial.

As a thing that can do tool use (and possibly have side effects), it's easy to do for enthusiasts who are willing and able to use the command line and savvy enough to make their own security tradeoff decisions.

The big question is how to allow open-ended turning-completeness in a way safe enough to be able to be mass market.