How I Designed 6 Specialist Agent Contracts (and Why Generic Agents Don't Compose)

The problem with "one agent to rule them all"

Single-agent setups feel productive in week one. You give the model a system prompt that lists every tool, every persona, every constraint. It looks like it works. Then you hit the wall:

• The agent confuses which role it should be playing when the user prompt is ambiguous.
• Tool selection becomes a guessing game — the model picks whatever the prompt weight leans toward, not what the task actually needs.
• You can't reason about failures. "Why did the model pick that tool?" has no inspectable answer because the prompt is 4,000 tokens of competing instructions.
• Two specialists' tasks collide inside one context window. The researcher "remembers" the security review and rewrites it.

This is not a model problem. It is a contract problem. The agent has no defined boundary, so the model fills the void with whatever seems plausible. You end up with a system whose behaviour is emergent in the worst sense — undocumented, untestable, and impossible to harden.

The shape of a multi-agent system is decided by its contracts, not its prompts.

The principle: narrow agents, explicit handoffs

Instead of one agent that does everything, design a small roster of specialist agents. Each one owns a narrow responsibility, has a fixed set of tools, and follows a documented handoff protocol when it needs to delegate. The orchestrator routes tasks to specialists. Specialists return structured outputs. Nothing else.

The benefits show up fast:

• Inspectability. Each specialist has a measurable contract. You can eval it independently.
• Failure isolation. One specialist failing doesn't poison the rest of the run.
• Context hygiene. Specialists keep their own working memory; they don't see each other's scratchpads.
• Composable. You can swap one specialist without rewriting the others, as long as the contract holds.

The contract shape I landed on

Each specialist agent is a single markdown file with YAML frontmatter. The frontmatter is the machine-readable contract. The body is the human-readable system prompt. Same shape for every agent — only the contents change.

---
id: solution-architect
type: specialist
use_when: |
  The task requires choosing between architectural options, or
  producing a component diagram, or evaluating trade-offs in a
  proposed design.
primary_sources:
  - project-context/architecture.md
  - project-context/tech-stack.md
  - project-context/decisions.md
delegates_to:
  - security-analyst    (when trade-offs touch auth or trust boundaries)
  - cost-estimator      (when trade-offs have billing impact)
collaborates_with:
  - workflow-analyst
  - discovery-analyst
success_signals:
  - decision_rationale_documented
  - rejected_alternatives_listed
  - adr_referenced_or_proposed
budget:
  max_tool_calls: 12
  max_tokens: 6000
---

# Solution Architect — System Prompt

You are the Solution Architect for a small AI product team.
You do not write code. You do not run tests. You produce
architectural decisions and the ADRs that justify them.

When the orchestrator routes an architectural question to you:
1. Read the relevant primary_sources files first.
2. If trade-offs touch security or cost, delegate to the
   matching specialist before answering.
3. Always return a Decision with: context, choice, rationale,
   and rejected alternatives.
4. If the choice warrants a permanent ADR, draft one in
   `project-context/decisions.md` and reference it in your output.

Notice what the contract forces: a fixed list of primary sources, an explicit allowlist of who this agent can delegate to, and a measurable success signal. None of these are prompt-engineering tricks — they are constraints that make the agent's behaviour inspectable.

The 6 specialists in my current roster

I run six specialists in my daily multi-agent workspace. They are not theoretical — each one is consumed by the orchestrator every week.

Each contract lives in its own file under ai/agents/<name>/agent.md. The orchestrator reads the directory at boot, validates the YAML, and keeps the roster in memory. Adding a new specialist means adding one file — no central registry to edit.

Handoff rules: the part most multi-agent posts skip

The contract above includes a delegates_to field. That field is enforced by the orchestrator, not trusted from the agent's prompt. The orchestrator keeps a graph of allowed delegations and rejects any handoff that isn't on the list.

Why this matters: when a general-purpose agent "decides" to call a tool, you are trusting the model's judgement about whether that tool is appropriate. With an explicit delegation graph, the model can only delegate to agents that the system has approved. This is the same idea as capability-based security — agents get capabilities, not "whatever the prompt suggests".

# orchestrator guard (pseudocode)
def delegate(from_agent, to_agent, payload):
    if to_agent not in from_agent.delegates_to:
        raise HandoffNotAllowed(
            f"{from_agent.id} cannot delegate to {to_agent.id}"
        )
    if not payload.matches(to_agent.use_when):
        raise HandoffOffTopic(
            f"payload does not match {to_agent.id}.use_when"
        )
    return to_agent.run(payload)

Two checks: structural (is this delegation allowed at all?) and topical (does the payload actually match the target's domain?). Both fail loudly. Both are testable. Neither depends on the model being honest.

Success signals: making the contract measurable

The success_signals field is where the contract meets reality. These are not vibes. They are checks that can run after the specialist returns. If the signal is not present, the orchestrator retries, escalates, or rejects.

Examples I've used in production:

• decision_rationale_documented — the solution architect's output must contain a "rationale" field that is at least 80 characters and references at least one ADR.
• tool_calls_within_budget — the automation engineer must complete the task in fewer than 12 tool calls; otherwise the orchestrator escalates to a higher-budget specialist.
• handoff_payload_schema_valid — every payload leaving a specialist must validate against the receiving agent's input schema. Zod handles this.
• no_duplicate_citations — the research analyst must not repeat a citation within the same response.

You do not need all of these on day one. Pick the failure mode that hurt you last week and turn it into a signal. Add one signal per iteration.

What this gives you

After two months of running this pattern, the differences from the "one prompt to rule them all" approach are stark:

• Eval is now possible. Each specialist has a regression suite of 30–50 cases. I can change one prompt and immediately see if quality regressed on the narrow task it owns.
• Failures are legible. When something breaks, the orchestrator log shows exactly which agent failed, which contract was violated, and which input triggered it. No detective work.
• Onboarding is shorter. A new contributor reads one specialist contract and learns one slice of the system in 15 minutes.
• Specialists get better over time. I can rewrite the security analyst's prompt without touching the cost estimator. The blast radius of a change is one agent.

When this pattern is wrong

Specialist contracts are not free. The overhead shows up when:

• The task is genuinely atomic and short. A one-shot prompt does not need a roster of six specialists and an orchestrator. Use a plain function call.
• The specialists need to share large context constantly. If two agents are reading the same 200 KB of source code, the contract overhead will dominate the run.
• You don't have a stable vocabulary for the specialists. If "what counts as a discovery question" keeps changing, the contracts will rot weekly.

For one-off scripts and prototypes, a single agent is fine. The contract pattern earns its cost when you have repeated workflows, audit requirements, or a team that has to share the system.

The template you can copy

Here is a minimal contract for a new specialist. Drop it into ai/agents/<name>/agent.md:

---
id: <kebab-name>
type: specialist
use_when: |
  <2-4 sentences describing the kind of task this agent owns>
primary_sources:
  - <files this agent should always consult first>
delegates_to:
  - <other specialist ids this agent may hand off to>
collaborates_with:
  - <specialist ids that may co-work without delegating>
success_signals:
  - <observable markers that the agent's output is correct>
budget:
  max_tool_calls: <number>
  max_tokens: <number>
---

# <Name> — System Prompt

You are the <Name> for <team / product>. You do not <out-of-scope thing>.
You do <in-scope thing>.

When the orchestrator routes a <type> task to you:
1. Read the relevant primary_sources files.
2. If <condition>, delegate to <specialist>.
3. Return <structured output shape>.
4. If <escalation condition>, escalate back to the orchestrator with context.

Fill in the brackets. Validate the YAML. Add the file. The orchestrator picks it up on the next boot. That is the entire integration story.

Closing thought

The hard part of multi-agent systems is not the models. It is the contracts. If you cannot draw the boundary between two agents on a whiteboard, the orchestrator cannot enforce it at runtime — and the model will quietly cross it every chance it gets.

Write the contract first. Then write the prompt. Then write the eval. In that order. The order is the lesson.