// blog

Agents are just loops (and that's the hard part)

2 min read
  • agentic-ai
  • fundamentals
  • llm

Every few weeks someone asks me what an "AI agent" actually is, expecting a complicated answer. Here is the whole thing:

while not done:
    response = llm(context, tools)
    if response.wants_tool:
        context += execute(response.tool_call)
    else:
        done = True

A model, a set of tools, and a loop that feeds tool results back in. That's it. Everything sold as an "agent framework" is scaffolding around those four lines.

So why do most agent projects die between demo and production? Because the loop is trivial and everything it touches is not.

Where the real work lives

Tool design. The model can only be as good as the levers you hand it. A tool called query_db(sql: string) is a footgun; a tool called get_customer_orders(customer_id) is a contract. Narrow, typed, well-described tools cut hallucinated calls dramatically — the description field of your tool schema is prompt engineering, and most people leave it one lazy sentence long.

Context management. The loop accumulates tool results, and tool results are noisy. A single API response can be 40KB of JSON when the model needed three fields. If you don't summarize, truncate, or project results before appending them, you're paying tokens to make your model dumber. The best agent codebases I've read treat context like a cache with an eviction policy, not an append-only log.

Stopping conditions. done = True is doing heroic work in my pseudocode. Real agents need budgets: max iterations, max tokens, max wall-clock, max cost. An agent without a budget is a denial-of-service attack against your own wallet.

Failure recovery. Tools fail. APIs time out. The interesting design question is what the model sees when that happens. Returning a raw stack trace teaches the model nothing; returning "error: order_id not found — did you mean to call list_orders first?" turns a failure into a course correction.

The uncomfortable implication

If the agent is a loop, then most of your engineering effort should go into things that look boring on an architecture diagram: tool schemas, result truncation, error messages, budgets, traces. The teams that succeed with agents are the ones that treat those as first-class code, with reviews and tests, rather than prompt-and-pray.

The loop is four lines. The moat is everything else.

← All posts