I read Sam Altman's The Gentle Singularity with great interest.

I have Thoughts^[ip], which I'll share in more detail soon.

But overall while I resonate with much of the piece I think his argument ignores the danger of power concentration in such a system.

At one point he observes "a small misalignment multiplied by hundreds of millions of people can cause a great deal of negative impact."

Yes, exactly!

That's one of the reasons why vertical integration of context+model into a hyper-aggregator is so dangerous!