The challenge with LLMs is often giving them the right context.

It's not that they lack a baseline common sense, it's that they don't know anything about your particular situation if you don't tell them.

This is an insight from Jesse Andrews in the comments of last weeks' reflections.

Often it's hard and tiresome to tell them all the background knowledge you need them to know.

LLMs are in a dark room; each conversation is a fresh start where you have to start from scratch giving it background knowledge about your particular goal and context.

This all implies to me that the LLM "montage" kind of breakthrough will be around wrangling contexts.