On buying intelligence as a service

 October 19, 2026

In the last couple of years, most companies have adopted some kind of AI. Most of them have done so by buying API access from a small number of frontier labs. This approach has the lowest barrier to entry and is often the preferred starting point, but it’s problematic because of the huge, unmanageable costs it creates, the shrinking, potentially useless gap in capabilities with open-weight models, and many sovereignty issues.

A while ago I wrote about how your providers’ business models shape their behavior, and that logic applies here without modification. Closed frontier model APIs are sold on usage-based pricing. You pay for each input and output token, including those that hit the cache (often at a different price). As I noted then, a provider that makes money from usage-based fees is incentivized to drive up your consumption, and they hold the lever. The tricky part about token costs is that the bill scales with usage rather than value. An agent that reads ten documents to answer one question burns ten times the tokens of one that reads one. This holds even if the last nine documents are irrelevant and don’t contain the answer. Also, nothing prevents the AI from re-reading the same document many times, which clearly wastes tokens. Effectively, the AI provider can heavily influence how much any request will cost, as long as it instructs the AI to burn the right number of tokens. Furthermore, once people get access to a new tool, they tend to use it more often, further increasing token usage. By itself, this isn’t an issue, but in most frontier labs, users often can’t immediately see or correlate usage with cost, creating the risk of spending millions of tokens on very low-value activities. When AI is embedded in products or workflows, the bill usually climbs faster than the revenue it enables. Every company I worked with that built a product or workflow on a closed frontier model API followed the same arc: a promising pilot, a growing bill, and no measurable change in value produced.

One common narrative about using frontier labs’ models is that they’re more advanced. On the surface, this seems sensible, since frontier models tend to be more capable than other models. The problem with this narrative, though, is that not every kind of activity has real benefits from using bigger models. It is becoming standard practice to leverage the biggest models for certain complex tasks, such as defining the process. Then, smaller, cheaper, and faster models are used to go through the process and create the specific content. Another factor is the pace of model development. Frontier models can keep iterating and improving their benchmarks, but open-weight models are doing the same, and they’re matching frontier performance faster and faster. Assuming benchmarks are a valid way to assess models’ capabilities, and that no one is cheating in the benchmarking process, the capability gap between frontier models and open-weight models is shrinking.

Closed frontier models can only be used as a service, at least for now. This can be either through a contract directly with the frontier lab that created it, or through a third party the frontier lab trusts to operate the models on its infrastructure, usually a major cloud provider. Using a model with this approach can immediately expose your organization to major security and sovereignty risks. The security risk is that you send a large amount of potentially sensitive data to a third party and rely on their security and the inference provider contract. This would already be an issue, but it gets even bigger when you realize many frontier labs have shown little regard for (other people’s) content ownership when looking for new datasets to train their models. The sovereignty aspect is slightly different, albeit very related. Once your application or workflow depends on an external API to work, any disruption to the connectivity to that API, to the API itself, or the contract regulating the API will disrupt your application or workflow. In addition to technical problems, political problems can also happen. If, for any reason, the government where your frontier lab of choice is based decides that AI or specific models should not be exported to your country, they will become unavailable.

Buying AI inference from frontier labs is by far the easiest path, but also the most expensive and risky. Deploying open-weight models locally can significantly reduce some of these risks, but it isn’t always easy or economical. A mixed approach with a local model router that lets applications talk to a local API, which then routes each request, is the best option: it keeps setup simple while still letting you dynamically tweak where each inference runs. Potentially, at the start, the model router could be configured to use a local AI model or an open-weight AI model run by a company you trust, ideally in the same jurisdiction where you operate, with optional access to frontier models. This will allow you to have a clear view of which applications and workflows are generating how much consumption and the kind of models they are using. Over time, you can tweak the routing to align with your budget, internal policies, and regulations.