The Same Origin model gives origins the right to send data to arbitrary other network endpoints by default.
Code being able to transmit data to a possibly sketchy origin is the scariest permission: once it happens, there's no recalling it, no tracking what happens after that point.
This makes it quite hard to trust any given origin with sensitive data, who knows what the origin might do with it?
The more trust an incremental origin requires, the more the ecosystem will tend to stabilize on a small number of large, powerful players.