How to build AI agents that work: start with the simplest thing
Most AI agent projects get canceled over cost, unclear value or missing controls. When you actually need an agent, what to build first, and how to shape it so it's dependable.
“AI agent” is the word of the moment, and also the one that sends the most projects to the bin. Gartner expects more than 40% of agentic AI projects to be canceled by the end of 2027, because of rising costs, unclear business value or inadequate risk controls (Gartner, 2025).
None of those three reasons is a model problem. They’re design problems: someone built an agent where something simpler would have done, or built it with no limits and no way to measure it. This is the first of four notes on how to build AI agents effectively. I’ll start with the decision that matters most: how much of an agent your agent needs to be.
Workflow or agent: not the same thing
Anthropic draws a distinction I use on every project (Building effective agents):
- A workflow connects models and tools along paths you wrote in code. The model does each step, but you decide the order.
- An agent decides its own path: which tool to use, in what order, and when it’s done.
A workflow is predictable, cheap and easy to test. An agent is flexible, but every decision it makes on its own is a decision that can go wrong, and errors compound from one step to the next. Their advice is the same as mine: find the simplest solution that works, and add complexity only when you need it.
When you do need an agent
OpenAI’s guide to building agents (A practical guide to building agents) names three situations where an agent is worth what it costs:
- Decisions with nuance. Cases with exceptions and judgment, like whether to approve a refund.
- Rules no one can maintain anymore. A system with so many rules that every change breaks something else.
- Lots of unstructured data. Reading documents, understanding what a customer writes in their own words, holding a conversation.
If your case doesn’t clearly meet any of them, the same guide says it: a deterministic solution may be enough. A form, a rule or a script isn’t less modern. It’s cheaper to run.
Climb one step at a time
In practice, I think of it as a ladder. Each step adds capability and also adds things that can fail, so you only climb when the step below isn’t enough:
- 1
A single model call
With good instructions, examples and structured output. It solves more cases than you would think.
- 2
A workflow with fixed steps
Several calls in an order you control, with checks between steps.
Do the steps change case by case in ways you can't predict? Climb a step
- 3
An agent with tools
The model chooses what to do, within a written job and with limited actions.
Can one agent no longer handle the instructions or the tools? Climb a step
- 4
Several coordinated agents
Only for very broad, parallel work, and with a budget that can carry it.
The simplest system that does the job, every time
The most useful workflows
Before jumping to an agent, there’s almost always a workflow that solves the problem. These are the five patterns Anthropic describes, with a business example for each:
| Pattern | How it works | Example |
|---|---|---|
| Chaining | Each step uses the previous one’s output, with checks in between | Extract an invoice’s data, validate it, then draft the email to the supplier |
| Routing | One step classifies the input and sends it down the right path | Separate billing, technical support and sales questions |
| Parallel | Several calls at once, then the results are combined | Review a contract for payment, term and liability clauses at the same time |
| Orchestrator | One model breaks the task down and hands out the parts | Prepare a report that needs data from several different sources |
| Evaluator | One model produces and another critiques, until it passes | Translate a legal text and check it against a list of criteria |
If your problem fits a row in this table, start there. You’ll have something you can test step by step.
The three parts of an agent
When an agent really is the answer, it’s made of three things: a model, tools and instructions. Each one has its own way of going wrong.
The model: start with the most capable
OpenAI’s advice is to build the prototype with the most capable model to set a baseline, then try smaller models to see where they still work. If you start with the cheap model, you can’t tell whether a failure comes from the design or from the model. To make that comparison you need evaluations, which is what how to evaluate an AI agent is about.
The tools: few and well made
Tools are how the agent acts on your systems. They’re the most underrated part and the one that changes results the most. They get their own note: tools for AI agents.
The instructions: a written job
An agent without a written job does a bit of everything and nothing well. The instructions say what it handles, what it doesn’t, what data it uses and when it gives up. And everything the agent sees at each step, not just the instructions, decides how it behaves. That’s context engineering.
One agent before many
Multi-agent systems sound good in a slide deck. In practice, they cost. Anthropic reports that its agents use about 4 times the tokens of a chat, and its multi-agent systems about 15 times (How we built our multi-agent research system). For broad, parallel research it pays off. For tasks where each step depends on the last, like most software work, agents coordinate poorly.
OpenAI reaches the same conclusion: get everything you can out of a single agent before splitting it up. Often, one agent with good tools is enough.
Always with a way out to a person
A dependable agent knows when to stop. OpenAI’s guide suggests two triggers for handing control to a person:
- Going past a failure limit. If the agent has tried several times and still doesn’t understand the request, it stops trying.
- High-risk actions. Canceling an order, approving a large refund or making a payment go through a person until the agent earns trust with data.
If you’re thinking about an agent that talks to customers, also read what to require before putting an AI agent on WhatsApp.