I’m curious how people here are thinking about managing agentic LLM systems once they’re running in production.

[bryanadenhq]

Beyond basic observability, things like instrumentation, runtime control, and cost management seem to get complicated quickly as soon as you have multiple agents, tools, and models involved. In particular, it feels hard to reason about cost and token usage at the agent level, apply guardrails or budgets at runtime, or debug and compare agent runs in a structured way rather than just reading logs after the fact. I’m interested in hearing how others are approaching this today. What parts are you building yourselves, what’s working, and where are you still feeling friction? This is just for discussion and learning, not pitching anything.

[KeepALifeUS]

Great questions - these are exactly the challenges I ran into when building a production multi-agent system. Here's what worked for me:

Token Usage & Cost Management

The biggest cost driver in multi-agent systems is usually agent-to-agent communication. Every time agents talk directly, that's API calls on both sides. I moved to a stigmergy pattern (indirect coordination through a shared environment, inspired by how ants use pheromone trails) - agents read/write to a shared state rather than messaging each other directly.

Result: 80% reduction in API token usage because:

• No back-and-forth conversations between agents
• Each agent only reads what it needs, writes what others might need
• The "coordinator" agent only steps in when genuinely needed

Runtime Control

For guardrails and budgets at runtime:

• Token budget per agent per run (tracked in shared state)
• Circuit breaker pattern: if an agent exceeds threshold, it gets paused
• Structured output schemas so agents can't go off-script

Debugging Agent Runs

What helped most:

• Every action gets logged with agent ID, timestamp, input/output tokens, and cost
• Shared state has version history, so you can replay what each agent saw
• Separate "runs" with unique IDs for before/after comparison

Where I Still Feel Friction

• Comparing runs when the underlying prompts or models change
• Attributing cost when one agent's output triggers expensive downstream work
• Knowing when to retry vs. escalate to human

I documented the stigmergy approach here if useful: https://github.com/KeepALifeUS/autonomous-agents

Curious what patterns others have found helpful!

[metawake]

Great question! One thing I've found critical is run recording for debugging.

When a crew fails in prod, being able to:

1. Replay the exact execution locally (without API calls)
2. Diff against a working run to find what changed

...saves hours vs. digging through logs.

I built Work Ledger (github.com/metawake/work-ledger) for this -
wraps CrewAI crews with ledger.wrap(crew).

Curious how others are approaching debugging/observability for multi-agent systems?

[devonakelley]

The retry vs. escalate problem @KeepALifeUS mentioned is the one that kills me. You can log everything perfectly and still not know whether to retry or swap models until it's too late.

I've been building Kalibr for this. It tracks outcomes you define and shifts routing automatically when a model+provider starts degrading. So instead of manually diffing runs to figure out what changed, traffic just moves. Not rules-based fallback, more like "this path stopped working well, here's a better one right now."

Still early but curious if anyone else is trying to close the loop between observability and actually acting on it automatically.

[xXMrNidaXx]

Production agentic systems are a different beast than prototypes. Here's what we've learned:

1. Observability is everything

• Log every LLM call, tool use, and decision
• Trace full execution paths
• Set up alerts for unusual patterns (loops, excessive retries)

2. Graceful degradation

• What happens when an API is down? Have fallbacks.
• Timeout everything. Agents can get stuck.
• Build "safe mode" that restricts dangerous tools

3. Human escalation paths

• Not every decision should be autonomous
• Define clear triggers for human review
• Make it easy to pause and resume

4. State persistence

• Don't rely on in-memory state
• Persist to DB so you can recover from crashes
• Version your state schema

5. Cost monitoring

• Per-task, per-agent, per-user budgets
• Real-time dashboards
• Automatic throttling when limits approach

We run production agent systems at RevolutionAI and these patterns came from painful experience. The observability piece is probably the most underrated — you WILL need to debug weird agent behavior at 2am. Make it possible. 😅

[aniruddhaadak80]

From my point of view, the hardest production problem is reconstructing why an agent chose a path, not just seeing that it did.

Raw traces help, but once multiple agents, tools, and model routes are involved, I usually want a run ledger that captures prompt version, tool policy, model selection, retry history, and budget consumption as first class state. Without that, comparing two runs becomes guesswork.

I also think replay with frozen policies matters a lot. If I cannot rerun the exact decision context that produced a bad action, debugging turns into reading logs and hoping the failure reproduces.

[Nyrok]

One thing that's bitten us in prod: the prompt itself is a source of complexity that's hard to version and audit. When an agent behaves wrong, you're reading logs trying to figure out which part of a 400-token prose prompt caused it. Role? Constraints? Output format? All mixed together.

What's helped is treating the prompt as structured data from the start. Explicit blocks for role, constraints, output format, chain of thought. When each concern is isolated, you can diff them separately, swap one block without touching others, and actually know what changed between runs.

I built flompt (github.com/Nyrok/flompt) for this. Visual canvas, 12 typed blocks, compiles to XML. Prod debugging gets a lot easier when your prompt has structure. Open-source, a star is the best way to support it.