Sources and leaks from Amazon, Adobe, Atlassian, Citi, and more show what is really happening with AI right now: companies are trying to rein in AI use as costs spiral out of control.
A very large chunk of the improvements in the last year have come not from categorically better models, but from the circumstances of the models massively improving. For example, reasoning is just automatic prompt engineering, and eats a fuckton of tokens. Harnesses give LLMs tools, making it easier to turn nondeterminism into determinism (does this code compile is a decision the compiler can answer definitely). Then there’s subagents, which is just automatic context engineering.
Basically, the price per token might not have changed, but in practice, the amount of tokens used to get “SOTA” performance has massively increased.
A very large chunk of the improvements in the last year have come not from categorically better models, but from the circumstances of the models massively improving. For example, reasoning is just automatic prompt engineering, and eats a fuckton of tokens. Harnesses give LLMs tools, making it easier to turn nondeterminism into determinism (does this code compile is a decision the compiler can answer definitely). Then there’s subagents, which is just automatic context engineering.
Basically, the price per token might not have changed, but in practice, the amount of tokens used to get “SOTA” performance has massively increased.