Canary analysis
An agent compares the canary with the baseline fleet and recommends promote or abort.
Source: scripts/dev/review-runbooks/canary-analysis.yaml. CI checks it with runbook spec validate; teams,
people, skills and called runbooks it names resolve against your server
when you publish it.
apiVersion: therunbook.app/v1kind: Runbookmetadata: id: canary-analysis title: Canary analysis description: An agent compares the canary with the baseline fleet and recommends promote or abort. owners: [team:oncall] tags: [deploy, canary, agentic]spec: execution: title: "Canary ${{ inputs.service }} ${{ inputs.version }}" inputs: service: { type: enum, options: [api, web, worker, billing], required: true } version: { type: string, required: true } defaults: { actor: agent } steps: - id: compare_errors title: Canary error rate within 10% of baseline check: true instructions: Compare the canary's 5xx rate with the baseline fleet over 20 minutes. outputs: canary_rate: { type: number, required: true } baseline_rate: { type: number, required: true } - id: compare_latency title: Canary p95 within 15% of baseline check: true instructions: Compare p95 latency of canary and baseline over the same window. outputs: canary_p95_ms: { type: integer, required: true } baseline_p95_ms: { type: integer, required: true } - id: recommend title: Recommend promote or abort instructions: Recommend promoting the canary to the full fleet, or aborting, with one sentence of reasoning. outputs: recommendation: { type: enum, options: [promote, abort], required: true }Publish it to a project:
runbook push canary-analysis.yaml --project <project> --draftrunbook publish canary-analysis --project <project>