Typically the server is in charge, and the client is a dumb intermediary.

But there's no reason the client can't be in charge, and have the server be a dumb intermediary.

In that model, the question of where does the legitimate control emerge from?

The main question is: "which client does the user choose to log in on."

Logging in on a client is like leaving a horcrux of yourself, it's a high trust action.