Skip to main content

Production Readiness and Validation

ACP Engine production readiness is an evidence-based release decision. A successful build, an enabled Manager form, or a single smoke test is not enough. Every release candidate must use one immutable source revision and one explicit deployment profile across security, migration, rollback, observability, load, chaos, and end-to-end validation.

Required automated evidence​

Run the source-repository production gate before promotion. The gate covers:

  • all Go service and runtime tests;
  • deterministic fake AI and MCP dependency scenarios;
  • provider and MCP routing policy tests;
  • SDK generated-contract, type, and transport tests;
  • ADK validation, simulation, policy, diff, and deployment tests;
  • Manager type, UI-contract, and production build checks;
  • public route parity and Docker Compose configuration validation.

The live gate additionally requires scoped test credentials supplied outside source control. It verifies authenticated runtime reads and idempotent writes, normalized actions, persisted execution state, usage, trace output, cross-realm denials, and merchant/store authorization. Test credentials must never be embedded in reports or commands shared with support.

MCP and provider routing evidence​

Provider selection is bounded by required capability, active health, quality, latency, cost, region, and data-handling policy. MCP selection intersects the authenticated realm, agent allowlist, active flow, enabled server, approved and contract-tested tool, routing rule, dependency health, latency budget, per-call cost budget, region, and data-handling requirement.

An explicit MCP route does not silently fall back to another server. Registry and first-healthy routes rank eligible candidates deterministically and may fall through only within the approved candidate set. Route traces include the selected order, rejected candidates, effective timeout, retry count, region, data handling, and pinned policy version. Write retries require a downstream idempotency key.

Automatic Flow routing evidence​

Every routable published Flow needs a reviewed semantic manifest or dedicated routing profile plus positive, negative, ambiguous, continuation, and allowed-handoff fixtures for each supported locale. Capability-based handoffs must be tested against the actual eligible published Flow set. Before enforced mode, an approved labeled dataset must demonstrate routing accuracy of at least 0.90, high-risk safety of at least 0.99, p95 routing latency of at most 250 ms, and average routing cost of at most $0.002.

Run shadow mode first and compare applied and proposed Flow decisions without changing customer execution. Review false routes, clarification rate, fallback, handoff stability, and candidate-filter reasons. Begin enforced with a bounded canary percentage. The immediate routing rollback is realm mode disabled, which restores compatibility selection while preserving profiles and decision history. See Automatic Flow Routing.

For adaptive and hybrid Flows, test every declared AI-decision choice, adaptive Supervisor child, minimum-confidence boundary, deterministic fallback, provider failure, unknown-choice rejection, and allowed-output-action denial. Confirm that a model response cannot introduce an undeclared branch, Flow, tool, provider, or action. Keep writes, financial operations, destructive operations, approvals, schemas, idempotency, and data-egress policy under deterministic Runtime enforcement.

Failure and capacity validation​

Exercise provider outage, MCP outage, timeout, cancellation, duplicate delivery, worker interruption, stale lease, queue delay, database failover, and partial side effects. In-process fakes make protocol and dependency failures repeatable; worker, queue, database, and network failures must also be tested against the target deployment profile in an approved window.

Record synchronous, streaming, and asynchronous load results with concurrency, payload distribution, duration, p50/p95/p99 latency, throughput, error rate, queue age, saturation point, fairness, and quota enforcement. Results from one infrastructure profile must not be presented as universal capacity claims.

Merchant Commerce release journey​

The reference journey covers product discovery, cart creation and retrieval, idempotent cart mutation, checkout preparation, approval pause/resume, unauthorized-store denial, normalized frontend actions, usage, and trace output. ACP returns an opaque checkout handoff and does not execute payment or collect card, CVV/CVC, OTP, raw payment token, full-address, or payment-authentication data. Travel and accommodation capabilities remain separate domain MCP tracks.

Project-specific acceptance​

Regulated customer programs, including work performed under a Mastercard MSA, require a jointly defined workload and policy set for each project. Define payload sizes, concurrency, infrastructure, detection categories, precision, recall, false-positive and false-negative limits, deterministic-only and local semantic-guard profiles, performance thresholds, and failure behavior before testing.

Store immutable baseline and release-candidate evidence with exact source, configuration, dataset, and environment references. Production promotion requires written joint acceptance. An internal template, local benchmark, or previous-project result is not customer acceptance for a new project.

Promotion decision​

Before production promotion, confirm:

  • security review and cross-realm/data-egress negative tests passed;
  • automatic Flow routing evaluation and shadow/canary evidence passed when enabled;
  • migrations and rollback were rehearsed against the target topology;
  • logs, traces, metrics, alerts, and support handoff are operational;
  • load and chaos results meet the target deployment profile;
  • the Merchant Commerce journey passes where it is in scope;
  • all project-specific acceptance records are approved;
  • exact image tags, environment ownership, outage risk, and rollback revision are recorded.

Production deployment still requires explicit operator authorization. These checks establish readiness; they do not authorize a rollout.