What Does a Good Agent Spec Look Like? A PM's Framework for Designing AI Agents

Your engineer says the agent design is done — but you don't know what questions to ask to verify it. That's not your fault. Almost every AI agent tutorial available is written for engineers: LangGraph code walkthroughs, framework comparisons, deployment pipelines. There's almost nothing written from the PM's perspective on design specification. The result: PMs write requirements using PRD logic, engineers make architectural decisions based on technical intuition, and escalation conditions never get discussed until after launch.

This guide fills that gap: a three-layer decision framework to help PMs understand where their decisions belong, an eight-dimension deployment checklist to verify spec completeness before sprint planning, and 10 questions you can bring into your next spec review.

TL;DR

• Agent design has three layers: strategy (why build, who owns it), architecture (tools/memory/escalation), implementation (prompt/testing/monitoring). PMs own strategy and co-decide architecture.
• A complete Agent Spec must answer eight dimensions before launch: reliability, guardrails, success criteria, tool integration, cost/latency, human-in-the-loop, error recovery, and observability.
• Research cited by MorphLLM (via Princeton) shows agent systems with AGENTS.md run 28.6% faster and use 16.6% fewer tokens — and human-written specs outperform auto-generated ones.
• First-version agents should only suggest, never execute. Climb the Autonomy Ladder only after task completion rate meets your target.

Why PMs and Engineers Talk Past Each Other on Agent Design

The traditional PRD framework breaks down for agent design at a fundamental level. A PRD describes a deterministic system: the user does A, the system responds with B, all paths are enumerable. But an agent has autonomous judgment — you've given the system a license to make decisions under uncertainty.

If the scope of that license isn't explicitly defined, the problem isn't that the system will break. It's that the system will make decisions you didn't anticipate. As Mind the Product notes, the core challenge in agent design isn't which model to choose — it's defining "goals and guardrails," two things a PRD never asks you to specify.

Here's how one engineer put it: "My biggest fear is a spec that says 'let the agent automatically reply to customer service emails' with nothing else. No guidance on when to escalate to a human, no fallback if something fails, no list of what tools the agent can and can't use. So I guess — and when I guess wrong, I get blamed. But the spec never said."

The four production gaps PMs most commonly leave out of specs:

1. Context engineering: What information can the agent see at each step? What format, and at what precision?
2. Observability: Can the agent's behavior be traced and debugged?
3. Guardrails: What actions are permitted? What requires human approval?
4. Cost architecture: What's the token budget per task? What's the cost ceiling?

None of these appear in a standard PRD template — but leave them unanswered and the operational cost of your agent will far exceed your projections.

The Three-Layer Decision Framework

The starting point for fixing the PM-engineer communication gap is clarifying who decides what, and at which layer.

Layer	Core Questions	Who Decides
Strategy	Why build this agent? What does success look like? Build vs. buy vs. configure? Who is the owner?	PM leads
Architecture	Tool boundaries? Memory strategy (context / retrieval / external memory)? Escalation conditions? Autonomy level?	PM + Engineer co-decide
Implementation	System prompt design principles? Testing framework (multi-scenario accuracy validation)? Monitoring metrics?	Engineer leads

Strategy is the PM's most critical contribution. Before design begins, the PM must clearly answer: what problem does this agent solve, what is explicitly out of scope, and what does success look like? One CTO described the cost of skipping this: "Last month, a PM proposed an 'automatic customer behavior analysis agent' and said it was simple. Engineering estimated three months. Two of those three months were spent filling in design decisions that should have been answered in the spec."

Architecture is where communication most often breaks down. Three decisions in particular:

Tool boundaries: What tools is the agent authorized to use, and what are the constraints on each? Anthropic's tool design principle says tools should be "task-shaped," not "API-shaped." Tool boundaries should reflect business decisions, not implementation details.

Memory strategy: What belongs in the current context (needed right now), what gets retrieved on demand (needed occasionally), and what lives in external memory (retained long-term)? Anthropic's context engineering research introduces "context rot" — when context balloons as a task progresses, agent performance doesn't degrade linearly; it collapses after a certain threshold. The right approach: "smallest set of high-signal tokens."

Autonomy Ladder: Agent autonomy should escalate gradually — start with suggestion only, move to partial approval, then full autonomy. Mind the Product calls this the Autonomy Ladder, and the timing for moving up should be driven by whether task completion rate has hit your target — not by the calendar.

What a Good Agent Spec Looks Like: Eight-Dimension Deployment Checklist

This is the core deliverable. The eight-dimension framework from Product School, combined with Anthropic best practices, with specific PM-actionable verification questions for each dimension.

1. Reliability testing Have you tested in a production-like environment? Do you have specific latency and throughput targets (for example, "250 concurrent requests, P95 latency under 300ms")?

2. Guardrails Which actions are irreversible? Do those actions have an approval gate? What are the agent's minimum necessary permissions — what can it do, and what is explicitly off-limits?

3. Success criteria What is your task completion rate target? How do you measure tool correctness? What is your cost-per-successful-task target? As Product School puts it: "If you can't clearly define 'good,' you can't safely ship."

4. Tool integration What is each tool's boundary? Does each tool have timeout, retry, and backoff strategies? Anthropic recommends designing for idempotency to prevent duplicate operations on retry.

5. Cost and latency budget What is the token budget per task? Where is the cost ceiling? What's the fallback if the budget is exceeded? Engineers rarely ask this proactively — but without an answer, there's no way to design a cost-aware context strategy.

6. Human-in-the-loop What conditions trigger human intervention? When handing off to a human, does the agent pass the full context? What's the SLA for human response once escalation is triggered?

7. Error recovery What's the fallback if the agent fails? Have you done a tabletop drill — a simulation of agent failure scenarios? Does a rollback mechanism exist?

8. Monitoring and observability Is there end-to-end tracing? Are the engineering dashboard (for debugging) and the product dashboard (for business metrics) separate? What's the first metric you'll look at after launch?

A lesson from running our own agent fleet: the consequence of not defining observability before launch isn't system failure — it's spending far too long figuring out where the failure is when something goes wrong. Every agent in our system now has structured logs and a trace ID, which cut diagnosis time from "unknown" to a few minutes.

10 Questions PMs Must Ask Engineers

Engineers won't bring these up on their own, but you need to ask. Bring this list into your next sprint planning or spec review:

1. Under what conditions should this agent stop and ask for human input instead of deciding on its own?
2. If a tool call fails, what's the agent's fallback?
3. What does the agent need to remember? For how long? Who can clear it?
4. What are our success metrics? What's the task completion rate target?
5. What tools does the agent have? What is each tool's boundary — what is it explicitly not allowed to do?
6. What happens when the context window is full? Is there a compaction strategy?
7. How does the agent know it made a mistake? Is there an evaluator?
8. How will we monitor the first version after launch? What metrics matter?
9. Which operations are irreversible? Do those operations have an approval gate?
10. Where does the agent spec live? Is it in AGENTS.md, or somewhere else?

Question 10 isn't just about file location — it confirms that a spec document exists and has a clear format. If the answer is "in a Confluence page somewhere" or "in the engineer's head," that's a problem to solve, not a state to accept.

Three Spec Files: The Role of AGENTS.md / SKILL.md / DESIGN.md

Between 2025 and 2026, an industry-wide answer emerged to the question of "how do we make agent behavior specifiable?" A three-layer spec file convention has taken hold.

AGENTS.md = behavior layer Defines the agent's overall context, role, prohibited actions, and operating principles. Research cited by MorphLLM (via Princeton) shows that agent systems with AGENTS.md run 28.6% faster and use 16.6% fewer tokens than those without. More importantly: human-written AGENTS.md outperforms auto-generated versions — auto-generation actually reduced agent success rates by 2% and increased costs by 23%.

This means AGENTS.md is more than a human-readable spec document. It's a context anchor for the agent itself, and its quality directly affects agent behavior. Maintaining a well-written AGENTS.md is a quantifiable technical investment, not just a good habit.

SKILL.md = task layer Defines reusable skills and procedural knowledge. If AGENTS.md is "who," SKILL.md is "how" — the step-by-step workflow for specific tasks, decision rules, and output formats.

DESIGN.md = presentation layer Google Labs' DESIGN.md standard provides a dual-track format: machine-readable design tokens (YAML) plus human-readable design rationale (Markdown), giving AI agents a persistent, structured understanding of a system's visual design.

The DEV Community (AWS Builders) article on the three-layer framework offers a useful principle: formally verifiable content goes in the spec file; judgment-based content stays in natural language. Confusing these two is the most common reason spec files lose their effectiveness.

Our own agent fleet at Shareuhack runs on this exact three-layer architecture: CLAUDE.md as the global behavior spec, agents/AGENTS.md defining each agent's role and operating principles, and .claude/skills/ as the skill layer. In practice, this separation has made each agent's behavior more predictable and made it much faster for any agent to locate its own decision boundaries when given a new task. If you want to go deeper on memory architecture, we've written a separate piece on AI agent memory design.

Common Design Mistakes: Over-Engineering and Other Traps

Unclear business value and insufficient risk controls are the leading causes of agentic AI project failure — not technical capability shortfalls. Both point to design-phase problems, not implementation problems.

The four most common traps:

Trap 1: Starting with multi-agent architecture Multi-agent architecture solves the problem of a single agent hitting its capability limits — but if you haven't tested what those limits are yet, you're just multiplying your unknowns by N. Anthropic's recommendation: first confirm whether prompt-response is sufficient; then consider a single agent; only then consider multi-agent.

Trap 2: Too many tools When the number of tools exceeds the agent's selection ability, it becomes more likely to pick the wrong one. Anthropic's tool design principle is "consolidation over proliferation" — merge tools where possible, keep each tool's boundary crisp and non-overlapping.

Trap 3: Context overload (context rot) Anthropic's context engineering research names this "context rot": as context accumulates over a task, agent performance degrades non-linearly — it holds steady until a threshold is crossed, then collapses. The most common mistake is putting every possibly-relevant piece of information into context. The correct approach: "smallest set of high-signal tokens."

Trap 4: No escalation path This is the gap engineers, PMs, and CTOs all mention. The engineer can't guess it. The PM didn't think to specify it. The CTO finds out after launch. Productside's "Agent Journey Map" puts escalation design before task design — not after — for exactly this reason.

The fix is simple: the first version of any agent should be suggestion-only. Wait until the indicators are stable before climbing the Autonomy Ladder. That's not being conservative — it's avoiding the discovery of failure costs in production.

Important: "Simplicity first" and "eventually scaling to multi-agent" aren't contradictions — they're a sequencing decision. Build the simplest effective version first, then let data drive complexity increases. That's a sustainable path for agent design.

When Not to Build an Agent

Before committing to build an agent, answer three questions. If two or more are "no," you probably don't need one:

1. Is the task multi-step or does it involve branching decisions? (If the flow is fixed, a workflow handles it better.)
2. Does the task require tools or cross-session memory? (If it's a single query, RAG + prompt is sufficient.)
3. Does the execution path vary significantly with context? (If the path is fixed, workflow is more reliable than agent.)

Only if all three are "yes" is an agent worth building.

Build vs. buy vs. configure decision dimensions:

• Customization needs: If an off-the-shelf tool covers 80% of the requirement, configure first. Build only for core differentiating capabilities.
• Maintenance cost: The real cost of a custom-built agent begins after launch — model version updates, API changes, edge case accumulation. Off-the-shelf tools outsource that maintenance to vendors.
• Compliance requirements: For sensitive data or specific regulatory requirements, building gives you control over data flows. Off-the-shelf tool data handling requires a ToS review.
• Existing tool coverage: Before deciding to build, evaluate whether Zapier, Make, n8n, or similar no-code/low-code tools can already meet the requirement.

If you're evaluating security design for your agent, the OWASP Agentic AI Maturity Assessment Framework is a useful supplement, particularly on guardrails and monitoring design.

Conclusion

A PM's job isn't to learn LangGraph — it's to translate "decision boundaries" into specifications that engineers can execute. The three-layer framework tells you what you should decide. The eight-dimension checklist tells you whether your spec is complete. The 10 questions align your team on critical decisions before sprint planning even starts.

When the next agent requirement lands, here are two paths forward:

If you're a PM, start with the three-layer framework. Confirm that the strategy layer's three core questions have answers, then bring the 10 questions into sprint planning. If you're a CTO or tech lead, use the three-condition framework to determine whether an agent is warranted in the first place — and if it is, require PMs to complete the eight-dimension checklist before the sprint starts.