How Token Costs Grow at Multi-Agent Scale — And Where to Control Them

The problem isn't one agent

When you run a single agent, token costs are easy to reason about. System prompt × 1, user message × 1, completion × 1. You can estimate before you run.

Multi-agent pipelines break that model in four ways.

1. Context fan-out. Each agent in a crew gets the full conversation history by default. Three agents in a sequential pipeline means the same context block sent three times. At 4,000 tokens of system prompt + history, that's 12,000 input tokens before any of the agents does useful work.

2. Retry amplification. Low-confidence results trigger retries. If your pipeline has no confidence threshold, every agent retries independently. A 3-agent crew with 2× retry rate produces 9 model calls for one logical task.

3. Model selection mistakes. Most teams hardcode gpt-4o everywhere because it's the safe choice. For a crew where 70% of tasks are simple routing decisions, that's frontier model spend on tasks that llama3.2 could handle at $0.

4. Hidden async costs. Parallel crews run simultaneously. You don't notice the cost until the bill arrives, because there's no synchronous cost accumulation you can watch.

The five MeshFlow controls

MeshFlow addresses each of these with enforced controls at the kernel level — not optional utilities you can forget to call.

1. CostCap — hard budget enforcement

wf = Workflow(cost_cap=CostCap(usd=0.50)) wf.add(Agent("researcher"), Agent("analyst"), Agent("writer")) result = wf.run("Analyse Q3 results") # Raises CostCapExceededError if spend hits $0.50 before completion ```

CostCap runs inside `StepRuntime.run()` — the kernel that every agent step passes through. It cannot be bypassed by user code. The cap applies across all agents in the workflow, not per-agent.

2. ModelRouter — route by task complexity

router = AdaptiveModelTierRouter( tiers=[ ModelTier("fast", "llama3.2", max_tokens=512), # $0.00 local ModelTier("smart", "mistral", max_tokens=2048), # $0.00 local ModelTier("large", "gpt-4o", max_tokens=4096), # cloud, pay here ], adapt_every=50, )

cascade = CascadeRouter(router, escalation_threshold=0.65) ```

The cascade router starts with the cheapest model and escalates only when confidence is too low. Thresholds adapt automatically every 50 routes based on actual outcomes. In our benchmarks, approximately 90% of tasks stay on local models.

3. ContextCompactor — deduplicate before sending

compactor = ContextCompactor(CompactionConfig( strategy=CompactionStrategy.SLIDING_WINDOW, max_tokens=4000, preserve_last_n=4, )) ```

ContextCompactor runs between steps. When the conversation window exceeds `max_tokens`, it removes the oldest messages while preserving the last N turns. For a 10-message history, this can remove 6–8 messages before they get sent to the next agent.

4. cache_control — reuse stable context

MeshFlow sets Anthropic's `cache_control: { type: "ephemeral" }` automatically on system prompts longer than 1,024 tokens. Subsequent calls that share the same system prompt pay cache read rates (~10% of creation cost) instead of re-computing.

# No code change needed — AnthropicProvider does this automatically
agent = Agent("analyst", model="claude-sonnet-4-6", role="researcher")

5. WorkflowAnalytics — make costs visible

result = wf.run("task")
print(result.total_cost_usd)      # $0.042
print(result.agent_costs)         # {"researcher": 0.0, "analyst": 0.0, "writer": 0.021}
print(result.cloud_agents)        # ["writer"]   ← only one hit cloud
print(result.cache_read_tokens)   # 3200         ← tokens served from cache

The cost profile you should aim for

With all five controls active on a typical mixed workload:

vs always-gpt-4o: same output quality, with cloud spend concentrated on the tasks that actually need it.

What MeshFlow Cloud adds

The OSS layer makes costs visible per-run. MeshFlow Cloud aggregates across your entire organization — cross-team cost dashboards, ModelRouter analytics, spend-by-workflow trending, and CI cost gates that block PRs when a change increases org token spend beyond a threshold.

If your LLM bill is already large enough to have a FinOps conversation with your CFO, the dashboard pays for itself in the first month.