It's critical that the model and your context are at different layers.

If your context and memories are locked at one provider, you can't easily try out a new model.

That means you slowly get trapped into your model's worldview.

Your context must be portable, and it must not be stored at any one provider.

Given how the models currently work, there's no good reason for the main UI you use–the one that stores your context/memories–to also be one of the model providers.