"Agent" now labels almost anything with a language model behind it, including systems that only produce text. That makes vendor comparisons difficult and budget conversations worse, because the two things being compared differ by an order of magnitude in cost, risk, and time to production.
The distinction that matters is not conversational quality, memory, or how many tools appear on a slide. It is whether the system changes state in another system. A chatbot produces an answer and a human decides what to do with it. An agent performs the action itself - it writes the record, sends the message, issues the refund. Everything expensive about agents follows from that one property.
This article covers where the line actually sits, the intermediate levels most teams should stop at, what changes in architecture and cost when you cross it, and how to decide which one your problem needs.
The Line Is Whether It Can Act
A useful test: if the system malfunctions at 03:00 with nobody watching, what is the damage?
For a chatbot, the worst case is a wrong answer that a person reads. The damage is bounded by human attention - someone still decides whether to act on it. For an agent, the worst case is a wrong action already taken. There may be no human in the loop at all, and the damage is bounded only by what the system was permitted to do.
That difference reframes the whole project. A chatbot is an information quality problem: is the answer correct, grounded, and current? An agent is that plus an authority problem: what is it allowed to touch, who approved it, can it be undone, and how do you reconstruct what happened.
Conversational ability is not the dividing line. A system can hold a fluent multi-turn conversation, remember context, and cite sources while remaining a chatbot, because a human still executes every consequence.
Four Levels, Not Two
Most real deployments sit between the extremes, and naming the level prevents buying more system than the task needs.
Level 1 - Answering from a corpus. Retrieval-augmented question answering. The system searches documents and answers with citations. No external state changes. This is where most enterprise knowledge use cases belong and where most of the measurable value in this space has actually been delivered.
Level 2 - Reading from live systems. The model calls read-only APIs: order status, account balance, ticket history. It can now answer questions documents cannot, but it still cannot change anything. The added risk is data exposure, not data corruption, which makes permission filtering the main control.
Level 3 - A single bounded action. The system performs one well-defined write: create a ticket, book a slot, apply a refund under a limit. Each action is designed, validated, and logged individually. This captures a large share of automation value at a small fraction of the risk of an open-ended agent.
Level 4 - Multi-step autonomy. The system plans a sequence, calls several tools, reacts to intermediate results, and may loop. This is what people usually mean by "agent," and it is where the cost, the failure modes, and the operational burden are concentrated.
The common mistake is jumping from Level 1 to Level 4 because Level 4 demos well. Level 3 solves more real problems than its reputation suggests, and it fails in ways you can absorb.
What Changes in the Architecture
Crossing into action introduces components that a chatbot never needs.
A tool layer with enforced contracts. The model does not get a database connection; it gets specific functions that validate arguments, check scope, and enforce limits in code. Whether the model asked correctly stops determining whether the system stays consistent.
Durable state. A multi-step task outlives a request. It needs persisted state, resumption after failure, and a way to answer "what is this run currently doing" - a job store, not a conversation buffer.
Idempotency. Retries are certain. Without deduplication keys, a network timeout on a successful refund becomes two refunds. This single omission causes more agent incidents than model quality does.
Approval gates. Actions above a threshold need a human decision, with an interface that shows the resolved arguments and affected records rather than a tool name.
Reversibility and audit. Every action needs a correlation ID, a recorded version of the model and prompt, and ideally an undo path. A chatbot needs none of this because it changed nothing.
We cover the durability side in reliable agent backends and the authority side in AI agent security. The relevant point here is that both become mandatory the moment you cross the line - they are not optional hardening you add later.
What Changes in the Cost
The model bill is the smallest difference.
A chatbot answers in one or two model calls. A multi-step agent may make ten to thirty per task, each carrying accumulated context, so token cost per completed task can be an order of magnitude higher. Latency changes character too: a chatbot answers in seconds, while an agent task may run for minutes and therefore needs asynchronous handling and a progress surface.
The larger difference is engineering time. A retrieval chatbot is mostly ingestion, retrieval quality, and evaluation. An agent adds the tool layer, state machine, idempotency, approval workflow, audit store, rollback procedures, and an adversarial test suite. In practice the agent build is several times the chatbot build for the same domain, and the ongoing operational load is higher because every new capability is a new permission surface.
This is why the level matters. Level 3 - one bounded action - often delivers most of the intended benefit at a small fraction of Level 4's cost.
What Changes in the Risk
Chatbot failure modes are informational: hallucination, stale content, missing evidence, wrong tone. They are serious, but a human sits between the failure and the consequence.
Agent failure modes are operational. The action executes twice. The action executes against the wrong record because a filter was built from a misread instruction. The agent reads a document containing an instruction and follows it. The task fails halfway and leaves partial state that no process cleans up. The agent loops on a failing tool until the budget is gone.
None of these are model quality problems, and a better model does not remove them. They are bounded by design decisions - scope, limits, validation, approval, and how the system stops.
Choosing: Start From the Task
Three questions usually settle it.
Does a human currently perform an action, or only make a decision? If the human reads a report and decides, information is the product and a chatbot is sufficient. If the human's work is the clicking - copying values into a form, creating the ticket, updating the record - then action is the product.
Is the action reversible? Reversible actions can sit below an approval gate with a generous threshold. Irreversible ones - money out, external communication, deletion - belong behind a human decision regardless of how good the model is.
How much variance does the task contain? A repetitive task with a fixed shape fits Level 3: define the action, validate it, log it. A task where the required steps genuinely differ each time is the case for Level 4 - and also the case that is hardest to evaluate, because there is no fixed expected output to compare against.
When a Chatbot Is the Right Answer
It is worth stating plainly, because it is the more common correct answer.
Choose a chatbot when the bottleneck is that people cannot find or interpret information; when the domain is regulated enough that a human must own each decision anyway; when the underlying systems have no safe write API and building one is the real project; or when nobody can yet state what a correct action would be.
The last case deserves emphasis. If two experienced people disagree about what the system should do in a given situation, automating the action encodes an unresolved disagreement and executes it at scale. Answering questions is safe under ambiguity. Acting is not.
A Sensible Migration Path
Teams that reach production reliably tend to move one level at a time.
Start at Level 1 and measure retrieval and answer quality against real questions. Add read-only tools when documents cannot answer what users ask, and carry user identity through those calls so permissions are enforced at the source.
Introduce a single write action once the read path is trustworthy. Design it fully - scope, limit, validation, idempotency key, audit record, approval threshold - and treat that work as the template for every action that follows.
Add multi-step autonomy only when several bounded actions exist, the sequence between them genuinely varies, and you have the state, tracing, and rollback machinery to explain a failed run afterwards.
The label matters less than the level. A precise description - "read-only answering over policy documents with citations" or "one refund action under 500 EUR with approval above it" - tells you what to build, what it costs, and what can go wrong. "An AI agent" tells you none of those things.