The more powerful you are, the more that the environment will mold itself to fit what you like.
Even if you didn't ask it to!
This process will be invisible to you.
As a hyper-successful person you don't have to kick people out of your circle actively, just involuntarily raise an eyebrow and the world will adjust to kick that person out of your circle.
An example of this dynamic is, "The bear is sticky" in the show Silicon Valley.