Architecting Production Multi-Agent Swarms with LangGraph and CrewAI

Why prompt chains break in production

Most first-generation agent systems are a single long prompt chain: plan, call a tool, summarise, repeat. They demo well and fail quietly. A chain has no explicit state, only a growing transcript, so when a tool returns something unexpected the model improvises, and the improvisation compounds across every subsequent step. By step nine the agent is reasoning about its own earlier guesses as if they were facts, and nothing in the system can tell the difference.

The failure modes we see most often are not exotic. Agents loop on the same tool call because nothing records that it already failed. Two agents overwrite each other's notes in shared memory. A planner produces a twelve-step plan for a two-step problem and burns a day's token budget before anyone notices. A retry after a timeout sends the same purchase order twice, because the chain has no concept of an action that has already happened.

Each of these is a control-flow bug, not a model bug, and a better model does not fix it. Swapping in a stronger model often makes things worse for a while, because it is more persuasive when it is wrong. The fix is to stop treating the conversation as the program. The program should be ordinary code with explicit states and transitions, and the model should be called inside it to make bounded decisions.

When multiple agents are worth the overhead

Multi-agent is not automatically better than a single well-tooled agent. Every extra agent adds latency, tokens, a new prompt to version and a new place for errors to hide. Before we design a swarm we ask whether one agent with good tools, a clear system prompt and a validator would do the job. For a surprising number of workflows, such as document triage or a support assistant with five actions, it would, and we say so.

Several agents earn their keep when the work genuinely splits into skills that conflict inside one context. A route optimiser wants aggressive cost reduction; a compliance checker wants to refuse anything ambiguous. Put both instructions in one prompt and the model averages them. Give each to a specialist with its own tools and evaluation set, and you get a real argument that a deterministic rule or a human can settle.

The other legitimate reason is parallelism. When a task can be decomposed into independent sub-problems, such as scoring hundreds of candidate routes or checking every clause in a contract bundle, fanning out to worker agents and merging the results is faster and easier to test than one agent working through a list.

• Use one agent when the task needs a single skill set and fits comfortably in one context window.
• Split into specialists when instructions conflict, such as optimising for cost while enforcing compliance.
• Fan out to parallel workers when sub-tasks are independent and their results can be merged by code.
• Add a separate validator whenever an output will trigger a side effect in a real system.

Model the workflow as a graph, not a conversation

LangGraph lets us express an agent system as an explicit state machine: nodes are specialists (researcher, planner, critic, validator), edges are the only legal transitions, and the shared state is a typed object rather than a transcript. That single change makes the system inspectable. You can draw it, test each node in isolation with a fixed input state, and replay any run from a checkpoint. Conditional edges are plain functions over state, so the routing logic is code you can unit test, not a sentence in a prompt.

Cycles are allowed, which is what makes LangGraph suitable for agents, but every cycle is visible on the diagram. If a critic can send work back to a planner, that edge exists explicitly and carries a counter in state. Fan-out to parallel workers uses the Send API, and results merge through reducers you define, so ten workers finishing in random order still produce the same final state.

CrewAI is useful one level up, for describing roles, goals and delegation in terms a product team can reason about. In practice we use CrewAI-style role definitions to author the specialists, review them with domain experts, and compile them into a LangGraph graph that owns execution, retries and persistence. Domain experts edit the role; engineers own the graph.

• Typed shared state with explicit reducers, so concurrent writes merge predictably instead of clobbering each other.
• Checkpoints after every node, persisted to Postgres rather than process memory, so a failed run resumes from the last good state.
• Hard recursion limits and per-node token budgets, enforced by the runtime rather than requested in a prompt.
• Routing functions with unit tests, so a change to an edge condition is reviewed like any other code change.

Coordinating memory across dozens of agents

Memory is where swarms get expensive and unpredictable. We split it into three tiers: a short-lived scratchpad per agent, a run-scoped blackboard that every node can read but only writes through reducers, and long-term episodic memory in a vector store such as Qdrant that is queried, never appended to directly, during a run. Each tier has its own owner, retention period and access rules.

The scratchpad is where an agent thinks, and it is thrown away when the node finishes. Only a structured result, validated against a schema, is written to the blackboard. That keeps the shared state small, which matters for cost: if every agent passes its full reasoning to the next one, context grows with every hop and you pay for the same tokens many times over. It also stops one agent's half-formed idea becoming another agent's premise.

Writes to long-term memory happen after the validator signs off. That rule alone removes a whole category of hallucination, because unverified intermediate reasoning never becomes something a future run can retrieve as fact. Each stored memory carries its run ID, the evidence it was based on and a timestamp, so stale or disputed entries can be found and removed rather than quietly influencing decisions for months.

Bounded loops, budgets and idempotent actions

Infinite loops are the most common production incident in agent systems, and the cheapest to prevent. LangGraph enforces a recursion limit on every run; we set it per graph from the longest legitimate path plus a small margin, not to a large round number. On top of that, every cycle in the graph keeps its own counter in state, so a critic can return work to a planner at most a fixed number of times before the graph routes to escalation instead.

Budgets work the same way. Each node has a token ceiling and a wall-clock timeout, and each run has a total cost ceiling checked before every model call. When a budget is exhausted the run stops in a known state with a clear reason, which is far easier to handle than a run that simply never finishes. Track cost per completed task, not cost per call, because a cheap model that needs four retries is not cheap.

Retries are only safe if actions are idempotent. Every tool that changes something outside the graph takes an idempotency key derived from the run and the step, and the receiving service refuses a second request with the same key. Combined with checkpointing, this means a run that crashes halfway through can be resumed without sending a duplicate booking, payment or email.

• Set the recursion limit from the longest valid path in the graph, and alert when a run comes close to it.
• Keep a counter in state for every cycle, and route to escalation when it is exhausted.
• Enforce token, time and cost ceilings in the runtime, and stop with a typed reason when one is hit.
• Pass an idempotency key with every write so a resumed or retried run cannot repeat a side effect.

Deterministic consensus before side effects

Nothing touches a production system until a validator node agrees. The validator does not ask a model whether the answer is good; it runs checks: schema validation on every tool payload, policy rules for the action type such as spending limits or permitted ports, and, for high-stakes actions, a second independent agent that must reach the same conclusion from the same evidence without seeing the first agent's reasoning.

Independence is the point. If the second agent reads the first one's argument, it tends to agree with it, and you have paid twice for one opinion. We give the checker the original inputs and evidence only, and where possible a different prompt or model, then compare structured outputs field by field. Agreement on the decision and its key parameters passes; anything else is a disagreement, not a near miss.

When agents disagree, or a policy rule fails, the graph pauses at a human-in-the-loop checkpoint using LangGraph's interrupt mechanism, with the full trace attached. The reviewer approves, edits or rejects, and the run resumes from exactly that point. How often this path is taken is a number to watch, not to drive to zero: if humans never see a case, the thresholds are probably too loose.

Observe every thought step, then replay it

Each node emits OpenTelemetry spans with the prompt version, model, token counts, tool calls, budget remaining and validator verdict, nested under a single trace for the run. Because the graph is explicit, the trace reads like the diagram: you can see exactly which edge was taken and why. That is what turns an agent incident from a mystery into a short investigation, and it lets you send agent traces to the same backend your platform team already uses.

Checkpoints double as a test harness. When a run goes wrong in production, we export its input and checkpoints into a regression case, fix the node or routing rule, and replay the run to prove the fix. Over time this builds an evaluation set made of real failures, which is worth more than any synthetic benchmark. Every prompt change, model upgrade or new edge runs against that set before release, with pass rate, cost per task and escalation rate as release gates.

• Version every prompt and record the version on every span, so behaviour changes can be traced to a deploy.
• Log validator verdicts and disagreement reasons as structured fields, not free text.
• Turn every production incident into a replayable regression case before closing it.

How we would approach this

We start by mapping the workflow before choosing agents: which decisions are made today, by whom, with what data, and which actions are irreversible. That map becomes the first draft of the graph, and it usually shows that some steps need a rule, not a model. The OmniSwarm project, our Autonomous Supply Chain Swarm case study, followed this pattern: weather, customs and fuel optimisation agents collaborating through LangGraph, with Kafka, Qdrant and PostGIS underneath, and 94.6% of routing decisions automated. In systems like this the agents are the visible part; shared state, validation gates and traces are what make a number like that safe to rely on.

Multi-Agent AI Systems covers the orchestration, consensus and observability described here; AI Agent Development covers single agents, tool integration and human-in-the-loop safeguards, and is often the right first step. Our Budget Calculator gives a rough estimate in a few minutes, and a scoping call with one real workflow is the fastest way to find out whether you need a swarm or one good agent.