Skip to main content

Operational Guide

This guide is for QA, UAT, go-live, and support teams. It explains how to validate the customer environment, monitor runtime behavior, troubleshoot common issues, and prepare for production handover.

Environment Validation​

Start every environment validation with these checks:

  1. Confirm base_url is reachable.
  2. Run token exchange.
  3. Run authenticated status check.
  4. Confirm runtime realm, agent, provider, flow, and MCP IDs.
  5. Confirm the independent merchant, store, and merchant API-key scope when Catalog is in scope.
  6. Run the runtime provider list and independently verify the merchant Enrichment provider capabilities.
  7. Run GET /api/v1/catalog/stores with the merchant key and perform a Catalog search when the journey uses ACP Catalog.
  8. Run the downstream MCP server list and ensure required servers are active.
  9. Run downstream MCP test and discovery when external tools are required.
  10. Run public MCP initialize and tools/list when MCP clients are in scope.
  11. Run tool-selection preview.
  12. Run runtime chat.
  13. Confirm runtime and store Enrichment usage records are visible in their separate reporting surfaces.

Manager operators can run status in the embedded CLI as a read-only cross-check of realm runtime resource counts. Use merchant/store dashboards for Catalog and Enrichment checks.

Smoke Test Requests​

Run these requests in order using the Enterprise API Documentation:

  1. GET /api/v1/health
  2. POST /api/v1/auth/token
  3. GET /api/v1/auth/is-authenticated
  4. GET /api/v1/runtime/providers
  5. GET /api/v1/catalog/stores with a merchant API key
  6. GET /api/v1/catalog/search?storeId={store_id}&limit=10 with the same key
  7. GET /api/v1/runtime/mcp/servers
  8. POST /api/v1/runtime/mcp/servers/{id}/test when a downstream server is configured
  9. POST /api/v1/runtime/mcp/routing/preview
  10. POST /api/v1/runtime/sessions and retain session_id plus resume_token
  11. POST /api/v1/runtime/ai/chat without flow_id for automatic routing
  12. GET /api/v1/usages/summary

The optional Postman companion collection linked from the API documentation can execute the same sequence during development and UAT.

UAT Scenarios​

Plain Assistant Response​

Goal: confirm the assistant runtime works without a tool call.

Example user prompt:

Hello, can you help me choose a gift?

Expected behavior:

  • Runtime returns a friendly response.
  • Assistant runtime metadata is present.
  • Usage is recorded.

Product Search Through MCP​

Goal: confirm approved tool selection works.

Example user prompt:

Find black running shoes under 150 USD.

Expected behavior:

  • Tool selection uses the approved commerce MCP server.
  • Tool execution returns product results.
  • The response includes customer-facing content.
  • Usage records include tool metadata.

Built-in Catalog MCP Write Validation​

Goal: confirm the unified ACP MCP service can execute a Catalog mutation within the merchant key's allowed store scope.

Example:

  • Initialize the public MCP endpoint and call tools/list.
  • Execute a controlled catalog_upsert_product in a staging merchant/store with a stable SKU and an optional image gallery.
  • Confirm the product is visible through GET /api/v1/catalog/product and Manager Catalog Products.
  • Exercise cart and order tools only when inventory mutation and outbound events are approved for the test.

Optional Downstream Merchant MCP and Webhook Validation​

Goal: confirm an approved Merchant Services write tool performs the mutation and emits the expected subscribed outbound event.

Example:

  • Register https://merchant.thyris.cloud/mcp with a scoped tr_live_... key.
  • Discover the downstream server's tools and record the exact approved set. ACP's built-in Merchant Commerce profile does not expose procurement or payment-provider execution tools.
  • Execute a controlled catalog_upsert_product in a staging store with an optional two-item imageUrls gallery.
  • Confirm the product response preserves gallery order and uses the first URL as primary.
  • Confirm the merchant webhook receiver gets product.created or product.updated with direction=outbound and source=mcp.

Repeat only the downstream write operations approved for UAT. A separately registered Merchant Services MCP may expose procurement tools, but those tools remain downstream capabilities outside ACP's built-in Merchant Commerce contract. They can modify inventory, create business records, start supplier workflows, and emit additional outbound events.

Flow Capability Journey​

Goal: confirm a selected Flow contributes the intended bounded behavior without changing the client API.

Example:

  • Configure and publish a deterministic, adaptive, or hybrid Flow with a reviewed semantic manifest or dedicated routing profile.
  • Send a representative domain request without flow_id and confirm the automatic decision selects the expected capability module.
  • Send a separate protected request with flow_id and confirm explicit selection bypasses classification.
  • Verify the same subject-bound session and resume token preserve authorized continuation and handoff behavior.
  • For adaptive or hybrid Flows, exercise every bounded AI decision, low-confidence fallback, adaptive Supervisor child, denied output action, and provider-failure path.

Goal: confirm parallel execution and aggregation.

Example:

  • Execute a search against multiple MCP servers.
  • Confirm results are deduplicated and ranked.

Observability​

Use supported usage endpoints and the deployment's logging, tracing, metrics, and alerting platform for operational visibility:

  • GET /api/v1/usages
  • GET /api/v1/usages/summary
  • GET /api/v1/usages/tools/summary
  • GET /api/v1/observability/logs
  • GET /api/v1/observability/traces/{trace_id}
  • GET /api/v1/observability/metrics
  • GET /api/v1/observability/alerts

Manager exposes realm-scoped overview, dashboards, metrics, traces, logs, and alert lifecycle views. Prometheus, Grafana, Alertmanager, OpenSearch, and the configured telemetry exporter remain deployment-specific operational services; their presence in Compose does not prove the target environment is ready.

Recommended operational metadata:

  • realm_id
  • agent_id
  • flow_id
  • session_id
  • routing.decision_id
  • trace_id
  • mcp_server_id
  • tool_name

Common Error Categories​

401 Unauthorized : Token is missing, expired, or invalid. Re-run token exchange.

When ACP authentication succeeds but MCP connection/discovery or tool execution receives a downstream 401, check the registered MCP API key separately. For Thyris Merchant MCP, the key may be invalid, expired, or revoked; rotate the registered credential and rerun the MCP connection test and discovery.

403 Forbidden : The token is valid but does not have access to the requested realm or resource.

404 Not Found : The referenced ID does not exist in the target environment.

409 Conflict : The requested operation conflicts with current state, such as duplicate names or active dependencies.

422 Validation Error : The request body is missing required fields or contains unsupported values.

429 Too Many Requests : Rate limits or operational throttles are active. Retry with backoff.

5xx Runtime Error : Retry only when the operation is safe. Capture trace ID and contact support with request details.

Support Handover Checklist​

Before production handover, confirm:

  • Customer base URL is finalized.
  • Customer credentials are provisioned securely.
  • Realm and agent IDs are confirmed.
  • Assistant backend is active.
  • MCP servers are registered and tested.
  • Tool discovery is complete.
  • Routing rules are configured.
  • Flows are active.
  • Frontend chat flow is tested.
  • Usage reporting is visible.
  • Observability endpoints are accessible to authorized support users.
  • Customer support team knows how to provide trace_id, session_id, and timestamp when reporting issues.

Production Readiness Notes​

For production environments:

  • Use HTTPS for all external connections.
  • Avoid sharing secret keys in chat, tickets, or screenshots.
  • Use separate credentials per environment.
  • Keep UAT and production realms separate.
  • Rotate integration credentials on a regular schedule.
  • Monitor error rates and latency after go-live.
  • Confirm MCP owner contact paths before launch.
  • For Thyris Merchant MCP, record the key expiration/rotation owner and verify merchant outbound webhook subscriptions for every approved write tool.