“We’d like an AI agent to handle our customer requests.” We hear this more and more often at the start of a project. And in many cases, after an hour of discussion, the conclusion is the same: what the company needs is not an agent, but a well-designed workflow, with a language model at one or two specific steps.
The distinction is not a technical detail. It changes the cost, the timeline, the reliability and how easy the system is to maintain. This article explains the difference and offers a simple grid to work out what you really need.
Two very different ways to use an LLM
In a workflow, you define the path. The steps are set in advance: receive an email, extract the information, classify the request, route it to the right team, prepare a draft reply. The language model steps in at certain points, to understand or write text, but it does not decide the sequence.
In an agent, the model decides the path. You give it a goal and tools (search a knowledge base, query a CRM, send a message), and it chooses which actions to take, in what order, and when to stop. It can backtrack, try something else, combine several sources.
Both approaches use the same kind of model. What changes is who is at the wheel: your business logic, or the model.
A workflow follows a path you drew. An agent looks for its own path. Most business processes already have a path.
Why workflows often win
- Predictability: the same inputs produce the same route. You know exactly what the system will do, which is essential as soon as a process touches customers, money or sensitive data.
- Cost: an agent often chains several model calls for a single task, sometimes many. A workflow only calls the model where it adds value.
- Speed: fewer calls, less waiting. For high-volume processing, the difference quickly adds up to hours.
- Debugging: when a workflow gets it wrong, you know at which step. When an agent gets it wrong, you have to reconstruct its reasoning.
- Security: a workflow only performs planned actions. An agent that picks its own tools has a wider attack surface, especially against prompt injection.
When an agent really pays off
Agents are not a fad: they are the right answer in specific situations. Those where the path cannot be written in advance, because it depends on what you discover along the way.
- Open-ended tasks: research that requires consulting several sources, cross-checking, and deciding when you know enough.
- Highly varied cases: when requests are so diverse that a decision tree would become unmanageable.
- Adaptive sequences: when the result of one step completely determines the next, and you sometimes need to backtrack.
Even then, a good agent stays bounded: limited tools, a capped number of steps, human approval for sensitive actions.
The decision grid we use
Before choosing, we ask four questions about the target process:
- Can the process be drawn on a whiteboard? If the steps and branches fit on one page, it is a workflow.
- Is the number of distinct cases limited? If the vast majority of requests fall into a dozen categories, it is a workflow, with an “other” queue handled by a person.
- What does an error cost? The higher it is, the more you want to keep control of the path.
- Does the path depend on what you discover? This is the only question where a clear “yes” really argues for an agent.
In practice, the most common answer is hybrid: a workflow that reliably handles most of the flow, and an agent reserved for one specific step where autonomy brings real value.
A concrete example
A distribution company receives several hundred emails a day from business customers: quote requests, order tracking, complaints, address changes. The first project considered was an agent able to “handle everything”.
After analysis, most messages fell into five well-defined categories. The chosen solution: a workflow where the model classifies each email and extracts the key information, then simple rules route it to the right process. Order tracking is answered automatically from the ERP, quotes and complaints reach the right person pre-filled. Only ambiguous messages go to a human queue. The system is faster to build, cheaper to run, and every decision can be explained. An agent can come later for complex cases, if the need is confirmed.
Our recommendation
Start by describing the process as it exists today, step by step, before talking about technology. If you can write it down, automate it with a workflow, and add the language model where it understands or writes better than a rule. Keep the agent for the steps where the path truly cannot be drawn in advance. You will get a system that is more reliable, cheaper, and much easier to evolve.