Gen AI

AI Agent Architecture

An agent is a model wrapped in a loop with tools and memory, deciding at each step whether to act, remember, or give a final answer, unlike a plain chatbot.

A model sits in a loop with tools and memory, and the model itself decides at each step whether to call a tool, update state, or stop and answer

Why Does This Exist?

Ask a plain chatbot to book you a flight and it does exactly one thing: it turns your request into a text response. It might write something convincing — a fake confirmation number, a made-up flight time — but it has no way to actually go check anything. One input, one output, and the interaction is over.

A travel-booking task doesn't fit that shape. Booking a flight means checking real availability, comparing a couple of options against your stated budget, maybe searching again because nothing under $400 has a direct route, and only then telling you what it found. That's several steps and at least one action reaching outside the conversation entirely. An agent is what you get when you wrap a model in something that can run more than one step: a loop that can call tools, hold onto state between steps, and let the model decide when it's done. Remove the loop, and you're back to a chatbot that can only ever guess.

Think of It Like This

A travel agent on the phone, not a brochure

A brochure tells you about flights. You read it, and that's the whole interaction — one static answer, no matter what you actually need. A travel agent on the phone is different: they call the airline to check real seats, hear that $600 is too much, put you on hold, check again, and only then read you a final itinerary. Each step depends on what the last one turned up.

A chatbot is the brochure. An agent is the travel agent on the phone: checking, comparing, sometimes trying again, and stopping only once it has an answer worth giving.

How It Actually Works

The four components, and what breaks when one is missing

An agent has exactly four moving parts. The model is the reasoning core — it reads the current state and decides what to do next. Tools are its only way to affect or query anything outside the conversation: check flight availability, run a search, hit an API. Memory is whatever persists across steps — the budget you gave it, the options it already rejected, the fact that it already searched once. The loop is the control flow: the code that keeps calling the model, feeding it results, and deciding when to stop.

Pull out the tools, and the model can still reason but can never check a real flight — you're left with a chatbot that talks about booking instead of doing it. Pull out memory, and every step starts from zero, so it can search once but can't compare that result against a budget it's already forgotten. Pull out the loop, and you get exactly one model call, which is a chatbot again. The loop is what turns a single guess into a process that can check, compare, and retry before answering.

What "the model decides" means, mechanically

None of this requires the model to execute anything itself. At every step the model produces one output — text, in a known format — and the harness, the calling code that runs the loop, inspects that output for whether it looks like a tool call or a final answer. A request to check flight availability gets run for real by the harness, which feeds the result back in and calls the model again. A finished answer makes the harness stop the loop and return it.

So the model doesn't "call a tool" literally — it writes something that says, in effect, "check flights from here to there," and the harness is the only thing that can turn that into a real request. The model chooses; the harness acts.

Why this isn't a chatbot with extra formatting

The difference between a chatbot and an agent isn't nicer output formatting versus calling functions. It's what happens to the model's output after it's produced. A chatbot's output is the end of the interaction, full stop. An agent's model output can also be an instruction fed right back into another round of the same process: model reads state, model decides, harness acts, harness updates state, model reads that new state, and so on.

That's the mechanism that lets the travel agent check flights, notice $600 is over budget, and search again before answering. One pass in, one pass out is a chatbot. Feeding a step's output back in as the next step's input is what makes it a loop, and the loop is what makes it an agent.

Show Me the Code

1from typing import Literal, TypedDict2
3
4class ModelOutput(TypedDict):5    kind: Literal["tool_call", "final_answer"]6    payload: str7
8
9def run_step(output: ModelOutput, memory: list[str]) -> str | None:10    """Harness logic: inspect the model's output and route it.11
12    Returns a result to feed back in, or None if the loop should stop.13    """14    if output["kind"] == "tool_call":15        result = f"checked: {output['payload']}"  # stand-in for a real tool call16        memory.append(result)17        return result18    memory.append(output["payload"])19    return None  # final answer -> harness stops the loop here

The model never runs run_step itself — the harness does, on the model's behalf, every time it produces an output.

Watch Out For

Bolting tools onto a chatbot and never bounding the loop

Wrapping a model in a tool-calling harness without a step limit or a budget check turns a small mistake into an expensive one. If nothing stops the loop, an agent stuck between "search again" and "still over budget" keeps calling the flight-search tool until someone notices the bill — it needs an explicit ceiling on steps, cost, or time, not an assumption that the model will eventually decide to stop.

Assuming the model 'knows' it called a tool

The model has no memory of its own actions beyond what the harness hands back in the next message. It doesn't experience calling the flight-search tool — it produces text that requests a check, and the only way it learns what happened is by reading the result in its next input, exactly like anything else in the prompt. Treat the model as stateless between calls; the harness carries state forward.

The Quick Version

An agent is a model plus tools, memory, and a loop; a chatbot is a model alone, doing one input-to-output pass.
The model doesn't execute tools — it produces output, and the harness inspects that output to decide whether to call a tool or stop.
Memory is what lets a later step know what an earlier step already tried, like a rejected flight over budget.
Removing any one of the four components — model, tools, memory, loop — collapses the system back to a plain chatbot.
What makes it a loop, not a single pass, is that a step's output can be fed right back in as the next step's input.