LLM marginal cost will stay high even as token cost declines.

That's because of Jevon's Paradox.

As the cost gets cheaper, we'll use them for even more things.

Anything that uses LLMs will have to contend with non-trivial marginal cost, for the foreseeable future.

Electricity is cheap and yet it's still metered.