AI agents have tremendous potential to improve systems. There is real power in dropping an LLM-based component into a workflow and having it magically solve a problem for you.
But sometimes you get that niggling feeling that you're paying AI to solve the same problem over and over again.
Is that really the best architecture?
I've been using a simple shorthand to think about this in terms of "AI capex and AI opex".
No, this isn't about data centers. I’m using capex and opex loosely here — not as accounting classifications, but as a way to think about where the AI cost ends up: upfront in a reusable asset, or repeatedly at runtime.
I’m not going to argue “use fewer agents” (my guess is we're only just getting started with agent use!), but if the framework clicks with you, hopefully there will be at least a couple of places where you adjust how you use those agents.
Meet Herman
Let’s use an example. We have a data-processing problem where, every time certain events occur, data has to be pulled from a variety of sources and aggregated into a recommendation that then informs an important decision. The task might have been done manually to date, or not at all, because it's too complicated and the data has too many nuances.
So we build Herman the agent.
Herman can receive triggers, structured data, and multimodal inputs, and has a variety of skills to rip them apart, recompose them, and turn them into a report and recommendation. This is the kind of task happening over and over again in many organizations worldwide.
Building Herman takes some time because there are a lot of edge cases. It's important to tune the recommendation responses. We're also very concerned that the accuracy of the recommendations is high, so the recommendation isn't wrong, or at least not wrong very often. It's also important to monitor Herman's performance and retune him periodically, since the input may drift and degrade output performance.
So Herman is non-trivial to build, but it’s a massive boost: recommendations are now available that otherwise wouldn’t be, potentially at a lower error rate than human operators could achieve, and everything is fully automated.
This is a great result and actually a significant breakthrough thanks to AI!
An important thing about Herman, though, is that he is opex-heavy. A significant part of the intelligence costs money again every time he runs.
AI opex
Herman fulfills a key new role in the workflow, and even if we optimize him over time, tokens are burned every time the workflow runs. There’s also a little more "opex" hidden in the maintenance required to check performance and adjust for drift.
From here, there are a few ways to potentially reduce opex. For example: tuning system prompts, optimizing agent skills, adding tool calls, or potentially moving to a smaller, cheaper model. All of these help, but fundamentally, a significant marginal AI cost still attaches to every execution.
AI capex
But what if things could be different?
Another way to build the same data-processing pipeline would be to use coding agents to create and optimize a reusable software pipeline that captures many of the same decisions Herman makes, but instead of making them on the fly every time, hardcodes them or implements them as layers of classifiers. Such systems are complicated to build and tune, but depending on the problem, they're certainly possible.
You could also imagine a core implemented in conventional code, with a thin LLM or agentic layer around it.
(There’s obviously no need to use coding agents to build this, but given the complexity of the task, agentic coding may be one of the fastest ways to do it.)
This alternative is much more capex-heavy: spend more upfront turning what you've learned into a reusable software asset, then pay much less each time it runs.
The key is to capture work in reusable assets so we need less fresh model reasoning each time. Those assets don’t have to be code: they can also be validated knowledge, structured state, or other accumulated work that means the model doesn’t have to reason from scratch next time.
The trade-off looks like this:
- Opex-heavy: lower build cost + repeated AI execution + maintenance
- Capex-heavy: larger build cost + cheap execution + maintenance
A good way to think about it is:
Herman moves from being the clock to being the clockmaker — and eventually the maintainer.
Clearly, some of the optimizations we discussed in the opex section already move in this direction: they shift work from inline token processing into reusable code and programmatic execution. This essentially compounds value into a reusable software asset.
One example of the clock → clockmaker → maintainer transition — and part of what triggered this article — is work we've done on AI test set generators. Our initial systems relied heavily on Codex and Claude Code to build benchmarks from scratch and invent individual test cases. We then captured these activities in sets of skills.
Once we knew what we were doing, though, the next step was to move most LLM execution out of runtime and build the system around Hypothesis, a property-based testing library. There is still agent magic infused here and there, but the backbone of the work is executable software. We also use agents to change and configure the generation setup.
As system builders, we'll need to get really good at this: compounding execution paths down into maintainable, verifiable software.
So is AI capex better than AI opex?
Not necessarily. It depends on the problem, how often the workflow will run, and the token costs involved.
For one-off tasks, with few runs and uncertain outcomes, the opex path makes a lot more sense. It will likely be much faster to iterate in building opex Herman rather than capex Herman.
That said, as soon as a workflow becomes both frequent and reasonably stable, things can tilt quickly toward capex. The key things to consider include:
- Novelty in input: how broad and varied are the inputs to the workflow?
- Novelty in output: how unique and detailed are the outputs, and how precisely can we specify what makes them useful?
- Frequency: how often will the workflow be run?
- Workflow stability: how long will the workflow’s rules and requirements stay the same?
- Reliability: how accurate do the outcomes need to be? How high is the cost of failure when the results are wrong?
The more frequent, stable, and verifiable the task, the stronger the case for turning repeated reasoning into a reusable asset; the more novel, infrequent, and exploratory it is, the stronger the case for keeping LLM/agent-based execution at the core.
If the underlying rules change constantly, though, we can easily spend more maintaining the asset than we save in inference.
There are a few other things that jump out:
- When we use agents to build software, we're leaning towards the capex camp. Coding agents such as Claude Code and Codex are mostly being used to build reusable software.
- Today, much agent-generated code is still hard to maintain, and can still favor token burning if we don't understand the underlying task well.
- Over time, we'll likely need to cultivate another type of agent that focuses on maintaining and optimizing that code.
Another piece of good news is that work building agents as one-off experiments can seed a reusable software pipeline. Once we've built a version of Herman, it gives us insight into the problem and a starting point for deciding which parts to capture in reusable code.
Applying the concept
As a parting example, I’ll use a real agent use case from Safe Intelligence that I’ve worked on personally: preparing for customer conversations and discovering companies we might want to work with. Normally, this involves web searches, LinkedIn searches, and press lookups, then aggregating the information and scoring it. Using AI to do much of the heavy lifting really helps, and as AI computer use has improved, large-scale data synthesis has too.
But every search costs tokens. Searches overlap. The same company gets rediscovered multiple times. At first, we stored results as reports that were useful, but quickly went stale.
So we created something simple: Agent Universe. We stopped creating reports and built a wiki instead. Now, all of the agents doing customer research read from, edit, and update the same wiki. This is a really obvious move, but the result is extremely powerful. Token spend per query goes down because a lot of the content is already there, overlaps are reduced, and knowledge compounds over time. Humans can also annotate the wiki and trigger new tasks.
All in all, this is a simple pattern, but by moving from an opex-type model to a capex-type one, we're suddenly getting long-term investment in a system asset.
There’s a lot more scope to systematize some of the agent work we currently do from scratch each time, but we’ll get there.
Hopefully thinking about token spend in this way helps when architecting your own systems.
Is this work capex or opex? Are we building a reusable, cheap-to-run asset, or paying repeatedly for the same reasoning? What can we capture so we don’t have to solve the same problem again next time? And what will it cost to maintain what we build?
Tokenmaxxing has been one of the year's trends, but we need to think about where those tokens end up in the fossil record!
