How to Build an AI Agent: Architecture, Loop Economics and the Failure Modes That Come After the Demo

An AI agent is a loop. A model is given a goal and a set of tools, it decides what to do, something executes, the result comes back, and it decides again. Everything difficult about building one comes from that loop running more times than you expected, on inputs you did not anticipate, costing more than you budgeted.

This is a practical walkthrough of how agents are actually built: the architecture, the one decision that dominates your running costs, and the failure modes that only show up after the demo works.

The four parts

Strip away framework branding and every agent has the same anatomy.

The model does the reasoning. It is the only part that has judgement and the only part you pay for by the token.

Tools are the functions the model can call — a database query, an API request, a file write, a shell command. Tool design is most of the engineering. A well-described tool with a narrow signature gets used correctly; a vague one gets used creatively and wrongly.

State is what persists between steps: conversation, intermediate results, what has already been tried. This is where most production bugs live, because the model only knows what you put back in front of it.

Routing decides what happens next. And this is the decision that shapes everything else.

The routing decision, and why it is a budget decision

You have two options, and they are not equivalent.

Model-decided routing — the model chooses which agent or tool handles the next step. Flexible, expressive, and it means every task spends reasoning tokens simply working out what to do.

Code-decided routing — you define the paths explicitly, with ordinary conditionals and edges. Less elegant, dramatically cheaper, and far easier to test.

The size of that difference is measurable. On one research-and-summarise workflow, a role-delegation framework spent $4.10 in prompt tokens on orchestration alone. A graph-based framework running the same workflow spent close to nothing on routing, because routing was code. There is a latency version of the same story: roughly 120ms of orchestration overhead per node against about 450ms.

The engineering discipline that follows is simple to state and hard to hold: move every decision you can out of the model and into code, and spend model calls only where genuine judgement is required. Most steps in most workflows do not need judgement. They need a rule someone was too rushed to write.

Loop economics

A single agentic task typically produces 10 to 20 model calls. That multiplier is the thing to keep in your head while designing, because it converts small decisions into large bills.

Practical consequences:

• Every tool you add widens the decision space, and the model re-reads the whole tool catalogue on each call. Five sharp tools beat twenty vague ones on both accuracy and cost.
• Context grows as the loop runs. Unbounded state means every iteration is more expensive than the last. Summarise or truncate deliberately.
• Cap the loop. A maximum step count is not a nicety; without one, a confused agent will happily spend your budget rediscovering that it is stuck.
• Cache what does not change. System prompts and tool definitions are re-sent constantly.

Reported API spend on teams using agentic tooling runs $200 to $2,000+ per engineer per month, and inference is now around 55% of AI cloud spending. Agents are the workload driving that.

Parallelism, and when it helps

Two patterns have matured over 2026. Subagent spawning — the Claude Agent SDK supports it — lets a parent delegate independent pieces of work to run in parallel. And Google’s ADK 2.0 moved from hierarchical execution to a graph workflow engine, which is the same idea expressed structurally.

Parallelism helps when subtasks are genuinely independent and each is expensive. It hurts when subtasks need each other’s output, because you end up paying for work that gets discarded. The test is whether you could run the pieces in separate conversations without loss. If not, keep it sequential.

Humans in the loop

Any agent that can take a consequential action needs an approval step, and this is worth designing early rather than retrofitting. It is the single feature most associated with production-grade frameworks — conditional logic plus human approval workflows is precisely what graph-based orchestration is described as being best at.

Design it as a first-class state, not an interrupt: the agent should be able to pause, surface exactly what it intends to do and why, and resume from that point. Approval bolted on as a modal dialogue at the end tends to mean the work has already happened.

Reliability features worth using

The 2026 releases tell you what teams were struggling with:

• Per-node timeouts and a streaming API added to LangGraph — a hung tool call should not hang the agent, and users should see progress.
• Pluggable memory, knowledge and RAG backends plus a Chat API in CrewAI 1.14.6 — state management outgrew what the framework provided.
• Sandboxed execution with persistent workspace state in the OpenAI Agents SDK — agents that run code need isolation and continuity between runs.
• Cross-framework communication via the Agent2Agent protocol in Google ADK — agents built on different stacks increasingly need to talk.

Observability is not optional

A failing agent does not throw an exception. It produces a confident, plausible, wrong answer after eleven steps, and the only way to find out where it went wrong is to have recorded every step.

Log the full trace: each model call, each tool invocation with arguments and result, the state at each transition, and token counts per step. Token counts per step are what let you find the node quietly costing 40% of your bill.

A build order that works

1. Write the workflow as plain code first, with no model at all. Whatever you cannot express as rules is where the model belongs — usually far less than expected.
2. Define tools narrowly, with precise descriptions and validated inputs.
3. Add the model only at the judgement points.
4. Cap steps and add timeouts before you run anything at volume.
5. Instrument everything, especially per-step token counts.
6. Add the approval step for any consequential action.
7. Test against adversarial inputs — ambiguous requests, missing data, tools that fail. The demo path always works; the others are the product.

One practical note: a working two-agent pipeline in a role-based framework is roughly 25 lines of code. Getting to a demo is genuinely quick. Everything in this article is about the distance between that demo and something you can leave running.

For a side-by-side of the frameworks themselves, see our comparison of AI agent builders. If you are still deciding whether to build at all, what businesses actually use AI agents for covers the commercial case and the failure rate.

Frequently asked questions

What are the components of an AI agent?

A model for reasoning, tools it can call, state that persists between steps, and routing that decides what happens next. Routing is the decision with the largest cost consequence.

Should the model or my code decide the next step?

Code, wherever possible. Model-decided routing spends tokens on every task just to work out what to do — measured at $4.10 of orchestration tokens on one workflow where code-based routing cost almost nothing.

How many model calls does an agent make?

Typically 10 to 20 per task. That multiplier is why small design decisions produce large bills, and why capping loop iterations matters.

How do I stop an agent running up costs?

Cap the maximum number of steps, bound the context that accumulates, cache unchanging prompt content, keep the tool catalogue small, and log token counts per step so you can find expensive nodes.

When should agents run in parallel?

When subtasks are genuinely independent and individually expensive. If one subtask needs another’s output, parallelism just pays for discarded work.

How do I debug an agent that gives wrong answers?

By having recorded the trace. Agents fail by producing plausible wrong output rather than by throwing errors, so log every model call, tool invocation with arguments and results, and the state at each transition.

Technical details reflect framework documentation and published comparisons as at August 2026. Benchmark figures are vendor-reported and workload-dependent.