Outside the Agentic Loop
"Agents are LLMs with the ability to take actions (tool use)" "Agents are fine-tuned models for specific tasks" "Agents are the harness" "Agents are just loops"
Above are some of the definitions I keep running into for what an AI agent actually is. They look like they contradict each other, but they don't. They are describing the same thing at different resolutions. Here is how I have come to hold it.
An agent is a model that can take actions (tools), running in a loop toward a goal, wrapped in a harness that keeps it working over long, complicated tasks. Every agent has that loop, and the loop is the easy part. The hard part is everything Claude Code, Codex, Hermes and the rest have quietly figured out. The tools, the prompts, the context, how memory gets saved, how information streams back, how the context gets compressed before it overflows.
This is my attempt to lay the whole thing out plainly, because the space is genuinely confusing right now and most explanations either wave their hands or bury you in jargon. It is the version I wish someone had handed me six months ago.
Let me start at the bottom and build up. For the loop, we need to go back to 2023.
Most people's first interaction with an LLM was through a chat window. Behind that chat box, it is just an HTTP call. You POST some JSON, you get JSON back. The "OpenAI format" everyone talks about is just the shape of that JSON.
Let me motivate it with a simple example. Say you want to specialize in email replies. To the model it just looks like a list of system and user messages lined up:
draft = completion(
model="claude-sonnet-3.5", # or gpt or gemini, the model itself doesnt matter.
messages=[
{"role": "system", "content": "You help draft emails."},
{"role": "user", "content": f"Template:\n{template}\n\nEmail:\n{email}"},
],
).choices[0].message.content
That is one call. You ask, it answers. Most of what people call "AI features" are exactly this, a single call with a good instruction behind it. Summarize this, classify that, draft this email. No loop, no agent, just one request and one response. Worth sitting with that, because it means most of the AI in most apps is not an agent at all.
Now fast forward to 2025, when everyone is talking about agents. What actually changed? What is the so called agentic loop?
It is when a model has to decide and act over several rounds, using tools. Simply, it looks something like this:
messages = [{"role": "user", "content": "Triage my inbox and reply where needed."}]
tools = [read_inbox, check_calendar, send_reply] # things you let it DO
while True: # THE LOOP
resp = completion(model=..., messages=messages, tools=tools)
if resp.stop_reason != "tool_use": # model says "done"
return resp.content[0].text
for call in resp.tool_calls: # model asked to use a tool
result = run_the_tool(call) # YOUR code runs it
messages.append({"role": "tool", "content": result}) # hand result back
# loop again. The model now knows the result, picks next move
The thing to notice is that last comment. Nobody wrote out the order "first read the inbox, then check the calendar, then reply." The model picks its next move every time through the loop, based on what it just saw. That is the whole difference between an agent and what people call a workflow. If you script the steps yourself in code and the model just fills in the blanks, that is a workflow. If the model decides the steps, that is an agent. Same tools, the only question is who is holding the wheel. Anthropic draws exactly this line in their Building Effective Agents post, and it is the cleanest cut I have found through the confusion.
Here is the part that took me a while to sit with. That loop, the thing the whole valley argues about, is maybe five percent of what makes an agent good. I could write the loop above in an afternoon. The other ninety five percent is why Claude Code feels like magic and the agent I threw together at a hackathon felt like a toy.
So how do you make good agents?
| Weak agent | Strong agent | |
|---|---|---|
| System prompt | "You are a helpful assistant" | precise role, constraints, examples, failure handling |
| Tools | vague names, dump raw JSON back | sharp names, descriptions the model understands, results shaped for the model to use |
| Context | stuff everything in, blow the window | curate what goes in, when to trim, what to summarize |
| Loop | same while | same while |
The loop is identical in both columns. Everything that separates the two lives in the other three rows, so let me walk them, because this is where the real work is.
Take the system prompt. The lazy version is "You are a helpful assistant." The version that actually works reads more like a job description written for someone with no memory and no common sense. It says who the agent is, what it may and may not do, what its tools are and when to reach for each one, how to behave when something fails, and it usually carries a couple of worked examples so the model can pattern match instead of guess. Whenever I open up a strong agent's system prompt it is rarely clever. It is just thorough in a way I was too lazy to be. Most of the quality you feel from a good agent is sitting right there, invisible, in a wall of careful instruction nobody ever sees.
A good system prompt makes the agent smart. It still cannot do a single thing on its own. It can only talk.
Which is the whole reason tools exist. A tool is just a function you let the model call. But handing it a function is not enough. The name matters, the description matters, and the shape of what you hand back matters most of all. If a tool returns a raw blob of JSON, the model burns attention parsing it. If it returns something small and clean and obvious, the model can just act on it. Writing good tools turns out to be mostly writing good descriptions, which is prompt engineering wearing a different hat. Do this well and the agent can finally act in the world. Do it and it will forget it ever did, the moment it moves to the next turn.
Because the model has no memory between calls. None at all. The harness rebuilds that entire message list every single turn, from scratch. So context is a curation problem, not a storage one. What do you put in front of the model this turn, what do you trim, what do you compress into a summary so the important parts survive without overflowing the window. This is the part that felt like sorcery to me, and it turned out to be bookkeeping. When Claude Code "remembers" your project, something behind it is deciding what to keep and what to drop, and when it runs low on room it summarizes the old material and carries on.
Memory across sessions is the same idea one level up, and the answer is quieter than you would think. It is mostly files. The labs converged on this through 2026. Anthropic's memory tool writes notes into a folder the agent can read back later. A memory tool sounds mystical, and under the hood it is a function that appends to a file. That is genuinely all it is.
Once you have written the same block of instructions into three different agents, you start wanting to package it, and that package has a name now. A skill. A skill is a reusable set of instructions, sometimes with a script or two bundled in, that the harness loads only when the task calls for it. That last part is the trick. You cannot cram every instruction you own into one system prompt, it would overflow and muddle the model. So the harness keeps a shelf of skills, shows the model only their names and a one line description of each, and pulls the full thing off the shelf when it is actually needed. I have been writing these by hand for my own setup for months without clocking that they were the one piece of the ninety five percent I had already been practicing.
By now you can build something that looks like it works. The trouble is you have no real way to know if it does. That is what an eval is. A graded test set. A pile of real inputs, each with the right answer or a way to score the answer, run the agent across all of them, and read the number. It is the gap between "it worked the two times I tried it" and "it works." It is the least glamorous thing in this whole piece, it is what actually separates a demo from a product, and if I am honest it is the part I skip most.
A smaller annoyance shows up fast once you have tools. Every agent needs them, and if every app rebuilds the same ones from scratch, that is a lot of wasted work. That is what MCP solves. Write a tool once as an MCP server and any agent can plug into it, Claude, ChatGPT, Gemini, Cursor, all of them. It started inside Anthropic and at the end of 2025 they handed it to a neutral foundation, so it is now the closest thing the space has to a standard. There is a sibling protocol, A2A, for agents handing work to other agents, but MCP is the one you touch first. The short version is that MCP is how an agent reaches its tools, and A2A is how agents talk to each other.
All of this has a cost you feel almost immediately. Remember that the loop rebuilds the whole message history every single turn. A long task resends the same growing pile of text over and over, and you pay for every token of it, every time. The fix is prompt caching. The providers let you mark the stable front of your prompt, the system prompt and the tool definitions, so it is not billed at full price on each pass. Reads off the cache run around ninety percent cheaper on Anthropic. If you ever build an agent that runs for more than a few steps, this is the line between viable and absurd, and it is the least discussed number in the whole bill.
Put all of it together, the loop, the prompt craft, the tools, the memory, the skills, the evals, the plumbing, and what you have built is a harness. And of course, that is now something you can just buy. OpenClaw is a wildly popular open source one you run yourself. Hermes, from Nous Research, self hosts and improves itself as it watches you work. Claude Managed Agents is the opposite bet, where Anthropic runs the hands for you, the sandboxing and the credentials and the tracing, and bills you by the hour. OpenAI has AgentKit and Google has its Gemini agent platform. They all agree on the loop. They differ on who runs it and where it lives. The thing worth noticing is that this layer is filling up fast, viral open source coming from one side and the labs themselves from the other.
One more, because you will be tempted by it. Everyone wants to build a swarm of agents talking to each other. The current wisdom, and I think it is right, is to start with one. Cognition wrote a whole piece called Don't Build Multi Agents, the argument being that separate agents make decisions in isolation and those decisions end up colliding. Anthropic's take is gentler, reach for more than one agent only when you have a real reason, like isolating messy context, doing genuinely parallel work, or holding more tools than one agent can keep straight. The funny footnote is that Cognition themselves shipped a multi agent setup a few months later, once they had a real reason for it. So the lesson is not never, it is not by default.
The last piece rarely gets counted as part of the stack, and it should. How does a normal person, someone who is never going to open a terminal, actually reach one of these things. In 2026 the answer finally got good. Voice models can sit on a real phone number now, take the call, use the same tools a text agent uses, and keep talking to you while they work in the background. That is the bridge between "an agent" and something my family's clinic could actually pick up and use.
So that is the whole shape, bottom to top. And the thing I keep circling back to is that the loop, the part everyone fixates on, is the least interesting part of the whole machine. It is a while statement. The hard, valuable, unglamorous work is all of the stuff wrapped around it, the prompts and the tools and the context and knowing whether any of it actually works.
Here is a reference guide of sorts for every layer above. Having your favorite coding agent write the above for you is possible, but the act of doing it by hand is what separates good AI Agents from mediocre ones, in the next few weeks I intend to dive deeper into these.