Blog
#0155 min read

How to build AI agents that work: start with the simplest thing

Most AI agent projects get canceled over cost, unclear value or missing controls. When you actually need an agent, what to build first, and how to shape it so it's dependable.

“AI agent” is the word of the moment, and also the one that sends the most projects to the bin. Gartner expects more than 40% of agentic AI projects to be canceled by the end of 2027, because of rising costs, unclear business value or inadequate risk controls (Gartner, 2025).

None of those three reasons is a model problem. They’re design problems: someone built an agent where something simpler would have done, or built it with no limits and no way to measure it. This is the first of four notes on how to build AI agents effectively. I’ll start with the decision that matters most: how much of an agent your agent needs to be.

Workflow or agent: not the same thing

Anthropic draws a distinction I use on every project (Building effective agents):

  • A workflow connects models and tools along paths you wrote in code. The model does each step, but you decide the order.
  • An agent decides its own path: which tool to use, in what order, and when it’s done.

A workflow is predictable, cheap and easy to test. An agent is flexible, but every decision it makes on its own is a decision that can go wrong, and errors compound from one step to the next. Their advice is the same as mine: find the simplest solution that works, and add complexity only when you need it.

When you do need an agent

OpenAI’s guide to building agents (A practical guide to building agents) names three situations where an agent is worth what it costs:

  1. Decisions with nuance. Cases with exceptions and judgment, like whether to approve a refund.
  2. Rules no one can maintain anymore. A system with so many rules that every change breaks something else.
  3. Lots of unstructured data. Reading documents, understanding what a customer writes in their own words, holding a conversation.

If your case doesn’t clearly meet any of them, the same guide says it: a deterministic solution may be enough. A form, a rule or a script isn’t less modern. It’s cheaper to run.

Climb one step at a time

In practice, I think of it as a ladder. Each step adds capability and also adds things that can fail, so you only climb when the step below isn’t enough:

Fig.The complexity ladder
  1. 1

    A single model call

    With good instructions, examples and structured output. It solves more cases than you would think.

  2. 2

    A workflow with fixed steps

    Several calls in an order you control, with checks between steps.

    Do the steps change case by case in ways you can't predict? Climb a step

  3. 3

    An agent with tools

    The model chooses what to do, within a written job and with limited actions.

    Can one agent no longer handle the instructions or the tools? Climb a step

  4. 4

    Several coordinated agents

    Only for very broad, parallel work, and with a budget that can carry it.

  5. The simplest system that does the job, every time

The most useful workflows

Before jumping to an agent, there’s almost always a workflow that solves the problem. These are the five patterns Anthropic describes, with a business example for each:

PatternHow it worksExample
ChainingEach step uses the previous one’s output, with checks in betweenExtract an invoice’s data, validate it, then draft the email to the supplier
RoutingOne step classifies the input and sends it down the right pathSeparate billing, technical support and sales questions
ParallelSeveral calls at once, then the results are combinedReview a contract for payment, term and liability clauses at the same time
OrchestratorOne model breaks the task down and hands out the partsPrepare a report that needs data from several different sources
EvaluatorOne model produces and another critiques, until it passesTranslate a legal text and check it against a list of criteria

If your problem fits a row in this table, start there. You’ll have something you can test step by step.

The three parts of an agent

When an agent really is the answer, it’s made of three things: a model, tools and instructions. Each one has its own way of going wrong.

The model: start with the most capable

OpenAI’s advice is to build the prototype with the most capable model to set a baseline, then try smaller models to see where they still work. If you start with the cheap model, you can’t tell whether a failure comes from the design or from the model. To make that comparison you need evaluations, which is what how to evaluate an AI agent is about.

The tools: few and well made

Tools are how the agent acts on your systems. They’re the most underrated part and the one that changes results the most. They get their own note: tools for AI agents.

The instructions: a written job

An agent without a written job does a bit of everything and nothing well. The instructions say what it handles, what it doesn’t, what data it uses and when it gives up. And everything the agent sees at each step, not just the instructions, decides how it behaves. That’s context engineering.

One agent before many

Multi-agent systems sound good in a slide deck. In practice, they cost. Anthropic reports that its agents use about 4 times the tokens of a chat, and its multi-agent systems about 15 times (How we built our multi-agent research system). For broad, parallel research it pays off. For tasks where each step depends on the last, like most software work, agents coordinate poorly.

OpenAI reaches the same conclusion: get everything you can out of a single agent before splitting it up. Often, one agent with good tools is enough.

Always with a way out to a person

A dependable agent knows when to stop. OpenAI’s guide suggests two triggers for handing control to a person:

  • Going past a failure limit. If the agent has tried several times and still doesn’t understand the request, it stops trying.
  • High-risk actions. Canceling an order, approving a large refund or making a payment go through a person until the agent earns trust with data.

If you’re thinking about an agent that talks to customers, also read what to require before putting an AI agent on WhatsApp.