Skip to main content
The Production LayerThe Production Layer

Command Palette

Search for a command to run...

A clean connector does not make a reliable agent

Connector correctness and agent behaviour are separate problems. Most teams only test the first one.

Updated
•2 min read•View as Markdown
A clean connector does not make a reliable agent
V
Vamsi Krishna
I’m Vamsi, a builder focused on production AI systems. I write Production AI Field Notes to break down the architecture behind reliable LLM apps, RAG, agents, multi-agent workflows, evaluation, observability, safety, and governance. My goal is simple: help builders move beyond impressive AI demos and design systems that can be tested, operated, trusted, and improved in the real world.
Part of series5 Days of MCP for Production AI

Your integration tests pass. Every tool returns what it should, schemas validate, errors are structured.

The agent still spent forty tool calls going in a circle and then confidently reported a result it had not verified.

Nothing was broken. The connector did its job perfectly. The job was just much smaller than anyone assumed.

What the protocol owns

Discovery, invocation, schemas, transport, session lifecycle. Whether a capability exists, how it is described, and how a call is made and answered.

That is the entire list. It is a contract about plumbing, and it is a good contract.

What the host still owns

Everything that determines whether the system behaves sensibly:

  • The goal, and a checkable definition of what finishing means
  • Budgets: tokens, time, money, number of tool calls
  • Stop conditions, including the ugly ones like "the last three attempts produced the same error"
  • Approval gates for anything expensive or irreversible
  • Policy, meaning what this particular user is allowed to have done on their behalf

None of that belongs in the MCP server. A server enforcing per-user policy is a server that has to know about your product's permission model, which means every consumer of it inherits your assumptions.

The temptation is to push these rules into prompts, because prompts are easy to edit. A prompt that says "always ask before deleting" is a preference. Product code that refuses to delete without a confirmation is a control. Only one of them survives a model that is having a bad day.

The test that separates the two

Ask what a passing test actually proves.

"The search tool returns results" proves the connector works. "The agent, given an ambiguous request, searched twice, found nothing conclusive, and said so instead of inventing an answer" proves the system works.

The second kind requires evaluating trajectories: what the agent did, in what order, and when it stopped. It is more work to build, and it is the only thing that tells you whether autonomy is safe here.

Closing thought

Connector green does not mean agent ready. Those are two test suites, and most teams have written one.

Part 4 of 5 Days of MCP for Production AI.

If your agent went into a loop tomorrow, what would stop it, and would that thing be code or a sentence in a prompt?

#mcp#agents#ai-safety#evaluation#production-ai

Comments (3)

Join the discussion
J
Julian Neagu1mo ago

This is a common trap with agents. A working tool call does not mean the agent knows when to stop or what success looks like.

V
Vamsi Krishna1mo ago

Fair point on the varying arguments, and it breaks the stop condition I wrote. The counter has to key on the error rather than the call, and it has to live in the harness.

C
Cai1mo ago

The distinction between connector-level testing and system-level testing is one I had to learn the hard way too. What pushed it home for me was watching an agent cycle through the same three tools forty times, each time getting the same error, and each time trying a slightly different argument instead of stopping. The connector was flawless. The agent had no budget, no circuit breaker, and no way to say I tried everything reasonable and I cannot do this.

The prompt-based approach you mention is the most seductive trap in the space. It feels like you are adding a control, but what you are really doing is adding a suggestion. The model can override it, forget it, or reinterpret it on the next turn. Code that says if tool_call_count > 10, stop and report is not elegant, but it is a control that survives model changes and prompt rewrites.

The test that separates the two is exactly right. Most teams are still running connector-green and calling it agent-ready. The trajectory-based evaluation is harder to build, but it is the only thing that tells you whether the system actually works.

5 Days of MCP for Production AI

Part 4 of 5

What the Model Context Protocol actually standardises, and what it leaves to you. Hosts, clients and servers, the three primitives, transport as an ownership decision, and when a direct API is the better call.

Up next

When MCP earns its overhead, and when a direct API is better

One question settles most of these arguments: will more than one AI application need this capability?

More from this blog

T

The Production Layer

66 posts

The Production Layer is a practical publication for builders moving from AI demos to reliable production systems. Sixty posts in order, plus eight specialist tracks, covering the real engineering decisions behind LLM apps, RAG, agents, multi-agent workflows, evaluation, observability, safety and governance.