The enterprise use cases that generate the most business value from AI are rarely single-turn queries or simple document retrieval. They are multi-step workflows: research and synthesise, validate and route, generate and review, escalate and track. Single-agent LLM systems handle these poorly. Multi-agent architectures handle them well — but they come with their own set of design challenges that most teams underestimate.
When single agents are not enough
A single LLM agent with tool access works well for bounded tasks: answering a question, summarising a document, drafting a reply. Its limitations surface when tasks require sustained reasoning over many steps, parallel sub-task execution, specialised expertise in different domains, or reliable error recovery.
The classic failure mode is context window exhaustion — the agent accumulates tool call results and intermediate reasoning until the useful context is lost in a sea of prior steps. Multi-agent systems solve this by decomposing work across specialised agents, each operating with a focused context and clear responsibilities.
The decision to move from single to multi-agent is not primarily about capability — it is about reliability and maintainability. Multi-agent systems are more complex to build, but they are more reliable at scale because each agent has a clear, testable role.
Orchestrator-worker: the pattern that survives production
The most robust multi-agent pattern for enterprise workflows is orchestrator-worker: a planner agent that decomposes tasks and routes work, and specialised worker agents that execute specific subtasks.
The orchestrator holds the task plan and tracks state, but does not perform substantive work itself. Worker agents are specialised — a research agent, a data extraction agent, a validation agent, a drafting agent — each with a focused prompt, limited tool access, and clear output schema.
This separation of concerns makes the system debuggable. When something goes wrong, you can inspect which agent's output caused the downstream failure. It also makes the system extensible — adding a new capability means adding a new worker, not modifying a monolithic agent.
State management is the hardest part
The biggest engineering challenge in multi-agent systems is state management: maintaining a consistent shared understanding of task progress, intermediate results, and accumulated context across agents.
Avoid implicit state passing through natural language. Instead, define explicit state schemas — typed data structures that agents read from and write to — and treat inter-agent communication as structured API calls, not chat messages. This makes the system predictable and testable.
For long-running workflows, implement checkpointing: persist agent state at meaningful boundaries so that the workflow can be resumed from the last checkpoint on failure, rather than restarting from scratch. This is non-negotiable for workflows that take minutes or longer.
Human-in-the-loop at the right granularity
Fully autonomous multi-agent workflows are appropriate for low-stakes, reversible actions. For high-stakes or irreversible actions — sending communications, modifying records, initiating transactions — build structured human review gates into the workflow.
The key design decision is granularity. Too many human review steps negate the automation value. Too few introduce risk. Map your workflow to a risk matrix and require human confirmation only for actions above a defined risk threshold.
Async approval patterns work well for many enterprise workflows: the agent proceeds to the approval gate, notifies a human, and waits. Synchronous interrupts are appropriate for time-sensitive decisions. Design these gates explicitly — agents that silently halt on uncertainty are harder to debug than agents that surface their uncertainty explicitly.
Reliability patterns for production multi-agent systems
Multi-agent systems amplify both capability and failure modes. A single agent failing silently can cascade. Build reliability in at three levels: agent-level (retries with exponential backoff, fallback to simpler models for low-stakes subtasks), workflow-level (timeout handling, partial completion recovery), and system-level (dead letter queues for failed workflows, monitoring and alerting).
Idempotency is critical for agent tool calls that have side effects. Each tool call should be idempotent or should produce an explicit confirmation that can be checked before re-execution. This prevents duplicate actions on retry.
Agent evals are qualitatively different from single-model evals because you are evaluating emergent behaviour. Run end-to-end workflow tests with defined expected outcomes, not just unit tests of individual agents. Regression testing a multi-agent system requires a representative set of full workflow scenarios.
The framework question
LangGraph, CrewAI, AutoGen, and similar frameworks provide useful abstractions for multi-agent orchestration. They handle message passing, tool call integration, and state persistence — reducing the boilerplate of building from scratch.
Our recommendation: use a framework for the orchestration layer, but treat your agent prompts, tool implementations, and state schemas as first-class code. Avoid letting the framework's abstractions leak into your business logic.
For enterprise deployments requiring fine-grained control, auditability, and custom state management, we typically build a lightweight custom orchestration layer rather than adopting a full framework. The frameworks move fast — which is good for capability but can create migration burden for production systems.