Skip to main content

Research and monitoring

Research model​

Topics describe customer questions or information domains. Prompts are market- and language-aware measurement inputs with intent and tags. Prompt sets create a repeatable measurement unit used by manual runs and schedules.

Avoid prompts that name only the owned brand when measuring competition. Include category, problem, comparison, recommendation, product, campaign and factual prompts so the sample represents real discovery behavior.

Providers and repetitions​

A monitoring run selects a project, prompt set, one or more providers, optional targets and a repetition count. Repetitions expose answer variance. Every answer snapshot records status, provider/component, model, market, language, latency, token counts, raw evidence policy and observation time. Each direct provider call also creates an independent workspace usage event so monitoring cost remains queryable even when raw provider-response retention is disabled.

Schedules​

Supported schedules are hourly, daily at 09:00 UTC, weekly on Monday at 09:00 UTC and monthly on day 1 at 09:00 UTC. Each schedule contains timezone, provider IDs, prompt-set IDs, optional target IDs, maximum requests and maximum cost.

The scheduler uses leases/locks, bounded retries, backoff and failure state. Disabling a schedule removes its next run; enabling it queues an immediate next run. Budgets are evaluated before work is claimed.

Monitoring screens​

  • Overview: provider health, freshness, coverage and run state.
  • Runs: manifest, prompts, providers, repetitions, cost/latency and failures.
  • Answers: immutable answer snapshots and detected mentions.
  • Evidence: citations, source URLs, extracted claims and replay context.
  • Usage: workspace provider requests, normalized tokens, web searches, estimated cost, pricing coverage and recent metered operations.

Evidence rules​

Store raw provider responses only when policy permits. Evidence must be traceable to a run, prompt, answer and provider. Replaying a run uses its manifest and creates a new observation; it does not rewrite history.

Run lifecycle​

The run summary must expose requested, completed, failed and excluded sample counts. A partial run can produce directional intelligence only when the metric contract permits it and the low-sample state remains visible.

Search-required measurement must select a direct provider profile with web search enabled. Provider-native API search is not evidence that the same answer appeared in the provider's consumer interface. Review Provider Usage and Cost Management before high-repetition or scheduled measurement.