Skip to main content

Command Palette

Search for a command to run...

Day 54: Why multi-agent systems need decision rules

Why agents need arbitration rules before disagreement reaches the final answer.

Updated
3 min readView as Markdown
Day 54: Why multi-agent systems need decision rules
V
I’m Vamsi, a builder focused on production AI systems. I write Production AI Field Notes to break down the architecture behind reliable LLM apps, RAG, agents, multi-agent workflows, evaluation, observability, safety, and governance. My goal is simple: help builders move beyond impressive AI demos and design systems that can be tested, operated, trusted, and improved in the real world.

Agent A says approve it. Agent B says block it. What does your system do?

If you cannot answer instantly, the real answer is "whatever happens to happen", and undefined conflicts still resolve. They resolve by accident: last writer wins, highest confidence score wins, or the system stalls in a polite standoff until a timeout makes the choice nobody designed.

Disagreement is scheduled, not exceptional

Give two agents different objectives and different context and collision is inevitable. The risk checker optimises for safety; the growth agent optimises for conversion. Both are working correctly. Eventually they meet on the same case and give opposite answers.

That is not a bug in either agent. Treating it as one (tuning prompts until the disagreement goes away) suppresses a signal you actually want.

Four arbitration mechanisms, in order of preference

  • Policy lookup first. Many "conflicts" were already settled by compliance, legal, or an SLA. Retrieve the existing ruling rather than re-litigating it per request. This is faster, cheaper, and consistent with what the rest of the company does.
  • Priority rules. Safety constraints outrank optimisation goals, categorically, with no scoring involved. Some hierarchies should not be negotiable.
  • A designated decider for each remaining conflict class: agent or human, named in advance rather than discovered during the incident.
  • Escalation for genuine ties, delivered with both positions and their evidence attached. "The agents disagreed" is not an escalation; it is a shrug with a ticket number.

Log every conflict and its resolution

This is the part with compounding value, and it is almost always skipped.

Recurring disagreements are design feedback. Two agents colliding weekly on the same case class means their roles overlap, and arbitration is treating the symptom while the boundary defect stays in place. Fixing the role definition eliminates the conflict rather than resolving it repeatedly.

Conflict logs are also the best evaluation data you will get for free: real cases, genuinely ambiguous, with a recorded outcome.

Closing thought

Multi-agent systems need decision rules. They feel bureaucratic right up until the first production conflict, at which point they are the difference between an incident and a log entry.

Day 54 of 60 Days of Production AI Systems.

Who wins ties in your system today, and did anyone decide that, or did it just turn out that way?


Previous: Day 53: Why handoffs need contracts
Next: Day 55: Why observability is not optional for AI systems

Day 54 of 60 · Agent Coordination Contracts (chapter 9 of 10)

Agent Coordination Contracts

Part 6 of 6

A practical series on the contracts that let agents coordinate without chaos. It covers decentralized behavior, role design, shared communication, shared versus private memory, handoff payloads, and decision rules when agents disagree.

Start from the beginning

Day 49: When decentralized agent behavior makes sense

Why decentralized agent behavior needs convergence rules before it earns production trust.