It will be interesting to see how inelastic the demand for frontier tokens is.
...alternatives are largely good enough for most workloads. This is one reason why model providers are desperately trying to move to harnesses that store state.
30 chunks · 21 episodes
...alternatives are largely good enough for most workloads. This is one reason why model providers are desperately trying to move to harnesses that store state.
...s, which requires leaving money on the table for the fat tails. Two options for model providers: 1) Move to a sales-gated API for all uses. This would allow detecting the fat-tail use and bucketing them into the higher margin buckets based on u...
Whenever model providers complain about others distilling their models, I think about the Project Panama images. A warehouse full of books deliberately destroyed by the inge...
... minutes to closer to 5 minutes to get more efficiency. If you want to cost the model provider a ton, send a single question about 5 minutes after the last one finished, to stay permanently in the cache.
...elected for. If vertical integration is selected for, it's conceivable that the model providers stop giving access to the models for the frontier models. Mythos appears to be going that way. Conveniently, they can tell a story of national secur...
A situation good for society is "all of the model providers have to compete and none of them win" for LLMs. Great for everyone but the LLM model providers, who are in a never-ending red ocean battle. But the ...
The model providers seem to be in a meta-stable equilibrium. None of them have any differential pricing power, since the models are practically commodity. But they do a...
The LLM model providers are like electricity providers back when electricity was new. Competing to get better quality for cheaper. Innovating on new techniques to do so. Bu...
...'t. They are bound by their fiduciary duty to continue pushing, since the other model providers are too. One apparently told him that they were hoping for a Chernobyl style disaster that would get governments to step in and stop the competition...
I continue to think the best business parallel for LLM model providers is cell phone networks. Extremely capital intensive to build out, but then much lower marginal cost to operate. Though inference has much more margi...
Tying models to UX from that model provider is dangerous. If the models are tied to the UX from the vertical integration, users get stuck to a single model. That requires that one model to be "...
...y way users interact with models is not via a UX owned by one of the foundation model providers.
...dels of generally competitive quality and lightly differentiated abilities. The model provider should not also own the context layer. That's like the cell carrier trying to dictate you can only use their super-app on your phone.
...e race between a non-trivial number of powerful LLMs, which prevents any of the model providers from trying to flex their muscle too much.
...t consumers have a subscription to that then allows them to use any of the main model providers with their context and not get stuck with any of them. But it's unclear which of those offerings will be the schelling point that starts getting com...
...heir API before ChatGPT got big. Because they set that precedent, the other top model providers also added a public API. Now, if any one of the providers got rid of their API, their competitors would push forward and scoop up the market share. ...
...models after multiple conversation turns now. This is good for everyone but the model providers; no individual model provider will have undue power by default, because there are multiple options in the same ballpark. Similar competitive dynamic...
...ion between the layers. The application layer and model layer will collide. The model providers will push hard to have their vertically integrated app used instead of the public API.
...main UI you use–the one that stores your context/memories–to also be one of the model providers.
...f features, because now users can get them for free… if they just commit to one model provider. It's OpenAI trying to change the game from stateless, easy-to-swap LLM providers, where the only competition is on quality and cost of the model, an...