Agent Architecture for Production AI: A Practitioner's Guide to the 15-Component Harness

AI agents are no longer experimental. In 2026, they process customer inquiries, generate reports, manage infrastructure, write and review code, and orchestrate multi-step business workflows across every industry. The technology works. What fails — consistently and expensively — is the architecture around it.

The pattern is depressingly predictable. A team builds a prototype agent in a weekend. It works beautifully in demos. They push it toward production and discover that the model sometimes hallucinates, the costs are unpredictable, the agent occasionally takes actions it should not, there is no way to audit what it did or why, and when it fails, it fails silently. The prototype that impressed the board becomes the system that wakes up the on-call engineer at 3 AM.

The gap between a working agent prototype and a production agent system is not a matter of prompt engineering or model selection. It is an architecture problem. The model — whether Claude, GPT, Gemini, or an open-source alternative — is a single component in a much larger system. Production reliability comes from the fourteen other components that surround it: the harness that validates inputs, enforces security boundaries, manages costs, handles failures, maintains observability, and ensures that every action the agent takes is auditable, reversible, and governed.

This guide presents the 15-component agent harness pattern, developed through our work with engineering teams building production agent systems. It is not theoretical — every component addresses a failure mode we have observed in real deployments. The guide is aimed at engineering leaders and senior developers who have moved past the excitement of agent demos and are confronting the harder question: how do we make this reliable, secure, and cost-effective at production scale?

Why Agent Architecture Is Different from Traditional Software

Traditional software is deterministic. Given the same input, it produces the same output. You can test it exhaustively, reason about its behaviour statically, and predict its failure modes from code inspection. Agent systems violate every one of these assumptions.

The fundamental differences that demand a distinct architectural approach:

• Non-deterministic execution. The same prompt with the same context can produce different outputs across invocations. This is not a bug — it is inherent to how large language models work. Architecture must account for output variability rather than assuming reproducibility.
• Unbounded action space. A traditional API endpoint has a fixed set of possible actions. An agent with tool access can combine tools in ways the developers never anticipated. The combinatorial explosion of possible action sequences means you cannot enumerate all behaviours in advance — you must constrain them architecturally.
• Cost proportional to complexity. Traditional software has roughly fixed per-request compute costs. Agent systems have costs that scale with reasoning complexity, tool call chains, and retry loops. A single pathological request can consume hundreds of dollars in API costs if the architecture does not enforce budgets.
• Failure modes are semantic, not syntactic. Traditional software fails with exceptions, timeouts, and error codes — failures that monitoring tools detect automatically. Agent systems fail by producing plausible but incorrect outputs, taking inappropriate actions, or reasoning their way into loops. These semantic failures require domain-specific detection.
• Security surface is conversational. Traditional injection attacks exploit parsing flaws. Agent injection attacks exploit the model's instruction-following behaviour — an attacker embeds instructions in user input that the model treats as authoritative. This is a fundamentally different attack surface that requires architectural defences, not just input sanitisation.

The 15-Component Agent Harness

The agent harness is the complete architecture that surrounds the language model, transforming it from an unpredictable text generator into a governed, observable, and reliable production system. Each component addresses a specific category of production failure. Omit any one, and you create a gap that will eventually be exploited by adversarial input, edge-case data, or simple operational reality.

The components are organised into four layers: Input Processing, Core Execution, Output Governance, and Operational Infrastructure.

Layer 1: Input Processing

Component 1: Request Gateway. The request gateway is the single entry point for all agent interactions. It handles authentication, rate limiting, request validation, and payload size enforcement before any content reaches the model. This is your first line of defence against abuse, and the component that prevents a single misbehaving client from consuming your entire inference budget.

The gateway must enforce:

• Authentication and authorisation — who is allowed to interact with this agent, and what actions can they trigger?
• Rate limiting — per-user, per-organisation, and global limits that prevent runaway usage
• Payload validation — maximum input sizes, allowed content types, and structural requirements
• Request deduplication — idempotency keys that prevent duplicate processing of retried requests

Component 2: Input Sanitiser. The input sanitiser processes user content to detect and neutralise prompt injection attempts before the content reaches the model. This is not a traditional input validator — it operates at the semantic level, identifying patterns where user input attempts to override system instructions.

Effective sanitisation strategies include:

• Delimiter enforcement — wrapping user content in clear structural boundaries that the system prompt instructs the model to respect
• Instruction detection — scanning for imperative patterns ("ignore previous instructions", "you are now", "system:") that indicate injection attempts
• Content policy filtering — rejecting inputs that violate domain-specific content policies before they consume inference resources
• Multi-language awareness — injection attempts in languages other than the primary interaction language, including Unicode tricks and homoglyph substitution

Component 3: Context Assembler. The context assembler constructs the complete prompt from multiple sources: the system prompt, user input, conversation history, retrieved documents, tool results, and any other context the model needs. This is where retrieval-augmented generation (RAG) integrates with the agent architecture.

The assembler must manage:

• Context window budgeting — allocating tokens across system instructions, user context, retrieved documents, and response space
• Priority-based truncation — when context exceeds the window, removing the least important content first rather than truncating arbitrarily
• Source attribution — tagging each context segment with its origin so that output attribution and audit trails are possible
• Freshness management — ensuring retrieved context is current, with staleness thresholds appropriate to the domain

Layer 2: Core Execution

Component 4: Orchestrator. The orchestrator manages the agent's execution loop: receiving the assembled context, sending it to the model, processing the response, dispatching tool calls, and deciding when the task is complete. This is the control plane of the agent system.

Production orchestrators must handle:

• Multi-turn execution — managing state across multiple model invocations within a single task
• Tool call dispatch — routing tool requests to the correct handler and managing the results
• Completion detection — determining when the agent has finished its task versus when it is stuck in a loop
• Graceful degradation — falling back to simpler strategies when the primary model is unavailable or producing errors

Component 5: Model Router. The model router selects which model handles each request based on task complexity, cost constraints, latency requirements, and availability. Not every request needs your most capable model — simple classification tasks can run on smaller, faster, cheaper models while complex reasoning tasks route to more capable ones.

• Complexity-based routing — using input analysis to estimate task difficulty and select the appropriate model tier
• Cost-aware selection — routing to cheaper models when the task budget is limited
• Fallback chains — automatically failing over to alternative models when the primary is unavailable or rate-limited
• A/B testing support — routing a percentage of traffic to candidate models for evaluation

Component 6: Tool Registry. The tool registry maintains the catalogue of actions the agent can take, including their schemas, permissions, cost profiles, and side-effect declarations. Every tool the agent can call must be registered here — unregistered tools are unreachable, providing a hard boundary on the agent's action space.

Each tool registration includes:

• Schema definition — input parameters, output format, and validation rules
• Permission requirements — which users and roles can trigger this tool
• Side-effect declaration — whether the tool reads data, writes data, sends communications, or modifies external state
• Cost profile — expected latency, API costs, and resource consumption
• Reversibility flag — whether the tool's action can be undone, and if so, how

Component 7: Tool Executor. The tool executor runs tool calls in a sandboxed environment with resource limits, timeout enforcement, and error isolation. It ensures that a failing tool call does not crash the agent, a slow tool call does not block indefinitely, and a tool call cannot access resources beyond its declared permissions.

• Sandboxed execution — tool calls run with only the permissions and resources declared in their registry entry
• Timeout enforcement — hard limits on execution time, with configurable strategies for timeout handling
• Retry logic — configurable retry policies with exponential backoff for transient failures
• Result validation — verifying that tool outputs match the declared schema before returning them to the model

Component 8: Memory Manager. The memory manager handles both short-term (within a conversation) and long-term (across conversations) state. This is where the agent stores context that persists beyond a single request: user preferences, previous decisions, accumulated knowledge, and task progress.

• Conversation state — maintaining context within a multi-turn interaction
• Session persistence — storing and retrieving state across separate sessions
• Knowledge accumulation — building long-term memory from interactions, with relevance scoring for retrieval
• Memory hygiene — expiring stale entries, deduplicating, and enforcing storage limits

Layer 3: Output Governance

Component 9: Output Validator. The output validator checks every model response against domain-specific rules before it reaches the user or triggers downstream actions. This is the component that catches hallucinations, policy violations, and factual errors.

Validation strategies include:

• Schema conformance — verifying that structured outputs match the expected format
• Factual grounding — checking claims against retrieved source documents, flagging unsupported assertions
• Policy compliance — ensuring outputs do not violate content policies, regulatory requirements, or brand guidelines
• Consistency checking — comparing the current output against previous responses in the same conversation for contradictions

Component 10: Action Gatekeeper. The action gatekeeper enforces approval workflows for high-risk actions. Not every action the agent wants to take should be executed immediately — some require human approval, some need secondary model confirmation, and some should be blocked entirely based on context.

• Risk classification — categorising actions as low, medium, or high risk based on reversibility, impact scope, and cost
• Approval routing — sending high-risk actions to the appropriate human approver with full context
• Escalation policies — automatic escalation when actions exceed defined thresholds
• Audit logging — recording every approval decision, including who approved, when, and with what context

Component 11: Response Formatter. The response formatter transforms the model's raw output into the format expected by the consuming system. This includes structured data extraction, citation generation, confidence scoring, and multi-format output (text, JSON, markdown, HTML) depending on the client.

• Format transformation — converting between output formats without loss of semantic content
• Citation injection — attaching source references to claims based on the context assembler's attribution tags
• Confidence scoring — estimating and communicating the reliability of the response
• Streaming support — delivering partial responses as they are generated for latency-sensitive applications

Layer 4: Operational Infrastructure

Component 12: Observability Pipeline. The observability pipeline captures structured telemetry from every component: request traces, model invocations, tool calls, validation results, and cost metrics. This is not optional logging — it is the primary mechanism for understanding agent behaviour in production.

• Distributed tracing — end-to-end trace IDs that follow a request through every component
• Model invocation logging — prompts, completions, token counts, latency, and model version for every inference call
• Tool call auditing — inputs, outputs, duration, and error states for every tool execution
• Cost attribution — per-request, per-user, and per-task cost tracking with real-time dashboards

Component 13: Cost Controller. The cost controller enforces budgets at multiple levels: per-request, per-user, per-organisation, and global. Agent systems have a unique cost profile — a single request can trigger multiple model invocations and tool calls, each consuming resources. Without explicit budget enforcement, costs scale faster than usage.

• Per-request budgets — maximum token spend and tool call count for a single agent invocation
• Per-user quotas — daily, weekly, and monthly limits that prevent any single user from consuming disproportionate resources
• Organisational budgets — hard spending caps with configurable behaviour when limits are approached (warn, throttle, block)
• Cost-aware routing — dynamically routing to cheaper models or reducing context when approaching budget limits

Component 14: Error Recovery Engine. The error recovery engine handles failures across the entire harness: model errors, tool failures, timeout breaches, validation rejections, and budget exhaustion. Each failure type has a configured recovery strategy that maintains the user experience without exposing internal error details.

• Retry strategies — model-specific retry policies that account for rate limits, transient errors, and model degradation
• Fallback cascades — ordered lists of alternative approaches when the primary strategy fails
• State preservation — saving progress so that recovery can resume from the last successful step rather than restarting
• User communication — informing the user of delays, partial results, or failures with actionable context

Component 15: Evaluation Loop. The evaluation loop continuously assesses agent quality using automated metrics, human feedback, and domain-specific test suites. This is not a one-time launch gate — it is an ongoing process that detects quality degradation before users do.

• Automated evaluation — running standardised test cases against production traffic samples to detect quality drift
• Human feedback integration — capturing user satisfaction signals and routing low-quality interactions for review
• Regression detection — alerting when quality metrics fall below established baselines
• Model update validation — testing new model versions against the evaluation suite before promoting to production

Security Boundaries: Defence in Depth for Agent Systems

Agent security is not a single layer — it is a series of boundaries, each designed to contain failures that penetrate the previous one. The defence-in-depth model from traditional security applies, but the attack surface is fundamentally different.

Boundary 1: Input Perimeter

The input perimeter separates untrusted user content from the agent's trusted execution environment. Every piece of user-supplied content — text, files, URLs, API responses — is untrusted until validated. The input sanitiser (Component 2) and context assembler (Component 3) implement this boundary.

Key design principles:

• Never concatenate user input directly into system prompts — always use structural delimiters
• Treat all retrieved documents as potentially adversarial — RAG sources can be poisoned
• Validate content types strictly — a "text" input that contains encoded instructions is an injection attempt
• Log all sanitisation actions — knowing what was filtered is as important as filtering it

Boundary 2: Action Perimeter

The action perimeter constrains what the agent can do in the world. The tool registry (Component 6), tool executor (Component 7), and action gatekeeper (Component 10) implement this boundary through the principle of least privilege: the agent can only access tools that are explicitly registered, and each tool can only perform actions within its declared scope.

• Separate read and write tools — a tool that can query a database should not also be able to modify it
• Scope tool permissions narrowly — a "send email" tool should only send to pre-approved recipients, not arbitrary addresses
• Require explicit side-effect declarations — tools that modify external state must declare this, enabling the gatekeeper to apply appropriate approval policies
• Rotate tool credentials independently — if a single tool's credentials are compromised, the blast radius is limited to that tool

Boundary 3: Data Perimeter

The data perimeter controls what information the agent can access and what it can include in responses. This is critical for agents that handle sensitive data — customer information, financial records, internal documents — where the model's tendency to be helpful can lead it to disclose information that the user is not authorised to see.

• Context filtering — removing sensitive fields from retrieved documents before they enter the model's context
• Output scanning — detecting and redacting sensitive data patterns (PII, credentials, internal identifiers) in responses
• Data classification integration — querying the organisation's data classification system to determine handling requirements
• Cross-tenant isolation — ensuring that in multi-tenant deployments, one tenant's data never appears in another tenant's responses

Security boundaries in agent systems are a natural extension of AI governance practices. Organisations that have established governance frameworks for their AI systems find that the security boundary model maps directly to their existing risk controls — the agent harness provides the technical enforcement layer for governance policies that were previously enforced procedurally.

Failure Mode Engineering

Production agent systems fail. The question is not whether, but how — and whether your architecture detects, contains, and recovers from failures before they impact users. Failure mode engineering is the practice of systematically identifying how each component can fail and designing the recovery path in advance.

Model Failures

Language models fail in ways that traditional software does not:

• Hallucination. The model generates plausible but incorrect information. Detection requires domain-specific validation — the output validator (Component 9) must check factual claims against authoritative sources rather than relying on the model's confidence.
• Reasoning loops. The model enters a cycle where it repeatedly calls the same tools or generates the same reasoning steps without making progress. The orchestrator (Component 4) must detect loops through action sequence analysis and break them with a configurable maximum iteration count.
• Instruction drift. Over long conversations, the model gradually deviates from its system instructions as user content accumulates in the context window. The context assembler (Component 3) must periodically reinforce critical instructions and the output validator must check for drift indicators.
• Capability degradation. Model providers update their models, sometimes introducing regressions in specific capabilities. The evaluation loop (Component 15) detects these regressions through continuous testing, and the model router (Component 5) can redirect traffic to alternative models.

Tool Failures

Tools fail in predictable categories:

• Timeout — the tool takes longer than expected, often due to external API latency. The executor enforces timeouts and the recovery engine retries or falls back.
• Schema mismatch — the model provides arguments that do not match the tool's expected input. The executor validates inputs before execution and returns structured errors that the model can correct.
• Permission denied — the tool call requires permissions that the current context does not grant. The gatekeeper blocks the call and the model must find an alternative approach.
• Rate limiting — external APIs throttle requests. The executor implements backoff and the orchestrator may need to pause and resume the task.

Cascade Failures

The most dangerous failures are cascades — where one component's failure triggers failures in others. Common cascade patterns in agent systems:

1. Model error triggers retry, retry consumes budget, budget exhaustion blocks all subsequent requests for the user
2. Tool timeout causes the orchestrator to retry with a larger context (including the error), exceeding the context window limit and causing a different error
3. Output validation rejection causes the model to regenerate, but the regeneration includes the rejected output in its context, leading to the same rejection in a loop

Preventing cascades requires circuit breakers at every component boundary. When a component exceeds its error budget — too many failures in too short a period — it trips the circuit breaker and returns a fast failure instead of propagating the error downstream.

Cost Governance at Scale

Agent costs are fundamentally different from traditional software costs. A web application serves requests at roughly predictable per-request costs. An agent system's per-request cost varies by orders of magnitude depending on task complexity, the number of model invocations, the tools called, and whether the model enters retry or reasoning loops.

Effective cost governance operates at four levels:

Request-Level Budgets

Every agent invocation starts with a token budget and a tool call budget. The orchestrator tracks consumption against these budgets in real time. When the budget is approached, the system takes progressively restrictive action: first reducing context to use fewer tokens, then switching to a cheaper model, then summarising progress and returning a partial result rather than consuming unlimited resources.

User and Organisation Quotas

Quotas prevent any single user or organisation from consuming disproportionate resources. The key insight is that quotas should be tiered, not flat — a user performing a complex analysis task should have a higher budget than one asking simple questions. Dynamic quota allocation based on task classification allows the system to be generous where it matters and conservative where it does not.

Cost Attribution and Chargeback

Every token consumed, every tool call executed, and every model invocation made must be attributed to a specific user, organisation, and task. This attribution feeds dashboards that show where money is being spent, enables chargeback models for internal platform teams, and identifies cost optimisation opportunities — tasks that are consuming more resources than their business value justifies.

Optimisation Strategies

Practical cost reduction without quality degradation:

• Prompt caching — reusing model responses for identical or near-identical inputs, with cache invalidation policies tied to the context freshness requirements
• Model tiering — routing simple tasks to smaller, cheaper models while reserving expensive models for complex reasoning
• Context compression — summarising long conversation histories rather than including every message in the context window
• Speculative execution — running cheaper models first and only escalating to expensive models when the cheap model's confidence is below a threshold

Cost governance for agent systems parallels the discipline of cloud cost optimisation — the same principles of visibility, attribution, and progressive constraint apply. Organisations that have mastered cloud cost management have a significant advantage in managing agent costs because the operational muscle is already developed.

Agent Architecture Maturity Model

Not every organisation needs all fifteen components from day one. The agent architecture maturity model provides a staged adoption path that delivers value at each level while building toward comprehensive production readiness. This mirrors the broader AI SDLC maturity framework — agent architecture maturity is one dimension of overall AI engineering maturity.

Level 1: Supervised Agent

The agent operates with human review of every output before it reaches the user or triggers actions. This is the appropriate starting point for most organisations — it provides immediate value from AI assistance while maintaining full human control.

• Components required: Request Gateway, Input Sanitiser, Context Assembler, Orchestrator, Output Validator, Action Gatekeeper (set to approve-all)
• Missing: Automated cost control, sophisticated routing, long-term memory, continuous evaluation
• Risk profile: Low — human review catches model errors before they have impact

Level 2: Guided Agent

The agent handles routine actions autonomously and flags exceptions for human review. This is where most production systems should target initially — it captures the majority of efficiency gains while maintaining safety for high-risk actions.

• Components required: All Level 1 plus Tool Registry, Tool Executor, Cost Controller, Observability Pipeline
• Missing: Sophisticated memory, model routing, continuous evaluation
• Risk profile: Medium — routine actions execute without review, requiring robust validation

Level 3: Autonomous Agent

The agent operates independently within defined boundaries, with human involvement only for escalations and periodic review. This level requires mature observability, evaluation, and governance infrastructure.

• Components required: All 15 components, fully operational
• Missing: Nothing — all boundaries are enforced architecturally
• Risk profile: High operational complexity — requires dedicated platform team and continuous monitoring

Specifying and Testing Agent Systems

Agent systems demand the same spec-driven engineering discipline as any production software, but with additional dimensions that reflect the non-deterministic nature of model-driven execution. Vague specifications produce unreliable agents for the same reason they produce unreliable code — the implementer (whether human or AI) fills gaps with assumptions.

What to Specify

An agent specification must define:

• The agent's purpose — what tasks it should handle, and equally important, what tasks it should refuse
• Tool permissions — which tools are available, under what conditions, and with what approval requirements
• Output constraints — format requirements, content policies, and quality thresholds
• Failure behaviour — what happens when the model errors, tools fail, or budgets are exhausted
• Security boundaries — what data the agent can access, what actions it can take, and what information it can disclose

Testing Non-Deterministic Systems

Traditional test assertions — "the output must equal X" — do not work for agent systems. Instead, agent testing uses:

• Property-based assertions. Instead of checking exact outputs, verify properties: "the response must contain a citation for every factual claim", "the response must not include PII", "the tool call sequence must not exceed 5 steps".
• Statistical acceptance criteria. Run each test case multiple times and assert quality thresholds: "90% of responses must score above 4/5 on the evaluation rubric", "hallucination rate must be below 2%".
• Adversarial test suites. Maintain a library of prompt injection attempts, edge-case inputs, and previously-observed failure cases. Run these continuously and alert on any regression.
• Integration scenario testing. End-to-end tests that exercise complete agent workflows — from user input through tool calls to final output — verifying that all components interact correctly.

Organisational Considerations

Agent architecture is not purely a technical decision — it has significant implications for team structure, operational responsibilities, and governance processes.

Platform Team vs. Feature Teams

The agent harness is a platform capability, not a feature. It should be owned by a platform team that provides the harness as an internal product, with feature teams building specific agents on top of it. This separation ensures that security boundaries, cost controls, and observability are consistent across all agents rather than reimplemented (inconsistently) by each feature team.

Incident Response

Agent incidents require different response procedures than traditional software incidents. When an agent produces harmful output or takes an inappropriate action, the response must include:

1. Immediate containment — disabling the specific agent capability, not the entire system
2. Impact assessment — using the observability pipeline to determine how many users were affected and what actions were taken
3. Root cause analysis — determining whether the failure was in the model, the harness, the configuration, or the specification
4. Prevention — updating the harness configuration, evaluation suite, and specification to prevent recurrence

Governance Integration

Agent architecture decisions are governance decisions. The tool registry defines what the agent can do. The action gatekeeper defines what requires approval. The output validator defines what quality standards apply. These are policy decisions that should be made collaboratively between engineering, legal, compliance, and business stakeholders — not unilaterally by the engineering team. Organisations with mature AI governance frameworks find that agent architecture provides the enforcement mechanism for policies that previously relied on human compliance.

Implementation Roadmap

Building the full 15-component harness is a significant engineering investment. The following roadmap sequences the work to deliver value at each stage while building toward comprehensive production readiness.

Week 1–2: Foundation

• Deploy the Request Gateway with authentication and rate limiting
• Implement the Input Sanitiser with basic injection detection
• Set up the Observability Pipeline with request tracing and cost tracking
• Establish the Action Gatekeeper in approve-all mode for initial monitoring

Week 3–4: Core Execution

• Build the Orchestrator with multi-turn support and loop detection
• Implement the Tool Registry and Tool Executor with sandboxing
• Deploy the Context Assembler with token budget management
• Configure the Cost Controller with per-request and per-user budgets

Week 5–6: Output Quality

• Implement the Output Validator with domain-specific rules
• Build the Response Formatter with citation support
• Configure the Action Gatekeeper with risk-based approval policies
• Set up the Evaluation Loop with initial test suites

Week 7–8: Maturity

• Deploy the Model Router with complexity-based routing
• Implement the Memory Manager for cross-session state
• Build the Error Recovery Engine with circuit breakers
• Expand the Evaluation Loop with adversarial testing and regression detection

This timeline assumes a team of 2–3 engineers with experience in distributed systems. Adjust based on your team's size and the complexity of your agent's domain.

Getting Started

If you are building or planning to build production AI agent systems, the most important step is to stop treating the model as the product. The model is a component. The product is the entire system — the harness that makes the model safe, reliable, observable, and cost-effective.

Start with three questions:

1. What can this agent do? Define the tool registry and action space explicitly. If you cannot enumerate the agent's capabilities, you cannot secure them.
2. What happens when it fails? Design the failure modes before they occur. Every component in the harness should have a documented failure mode and recovery strategy.
3. How will you know it is working? Establish evaluation metrics and observability before launch. An agent system without observability is a black box — and production systems cannot be black boxes.

For engineering teams building production agent systems and wanting structured guidance on harness architecture, security boundaries, and governance integration — our AI-native engineering service includes agent architecture as a core capability. We work alongside your team to design, build, and validate the harness components appropriate for your agent's risk profile and operational requirements.

If you are evaluating whether your current agent architecture is production-ready, or planning a new agent system and want to avoid the common pitfalls, book a conversation. We will review your architecture against the 15-component model and give you a clear assessment of gaps, risks, and the most impactful improvements for your specific situation.

The organisations that succeed with AI agents in production will not be those with the most sophisticated models. They will be those with the most disciplined architectures — systems where every action is authorised, every output is validated, every failure is contained, and every decision is auditable. The model provides the intelligence. The harness provides the trust.