CodeRabbit Observability

Overview

Use documented metric definitions and preserve denominators. Avoid scraping bot comments as a substitute for authoritative reporting.

Prerequisites

Identify the CodeRabbit organization, Git provider, repository, plan, and accountable owner.
Read references/official-docs.md and re-check any time-sensitive contract before execution.
Use synthetic or read-only evidence until the approval boundary is satisfied.
Preserve the repository's independent CI, security, and human-review requirements.

Current Contract

Git-platform and IDE or CLI metrics have separate definitions.
The Usage dashboard explains events, limits, and the Review Log.
Data export supports offline analysis.
Authenticated metrics APIs depend on plan and permissions.

Authentication

Treat Git-provider sessions, CodeRabbit web sessions, CLI credentials, and CodeRabbit API keys as separate credentials. Use only an already-approved session or secret-manager reference, never print a secret, and do not place credentials in .coderabbit.yaml, source files, logs, or deliverables.

Instructions

Define decision, population, provider, repositories, UTC window, and metrics.
Export with timestamps, filters, pagination, and exclusions.
Compute coverage, response, resolution, and usage with denominators.
Compare cohorts and publish thresholds only after baseline evidence.

Tool Discipline

Use Glob to locate candidate configuration and evidence files without widening scope.
Use Grep to find relevant fields, commands, identifiers, and stale claims.
Use Read to inspect the smallest required files and authoritative evidence.
Use Write only for a new approved local draft or evidence artifact.
Use Edit only for a bounded approved change whose rollback is known.
Do not use these file tools as a substitute for authenticated CodeRabbit or provider operations.

Approval Boundaries

Require data-owner approval before user-level exports, scheduled reports, or API credentials. Keep analysis and drafts local until approval is explicit, and record who approved the action and its scope.

Output

A reproducible metric spec, source receipt, dataset summary, dashboard, anomalies, and actions. Include source dates, unknowns, and the exact boundary between observed fact and recommendation.

Error Handling

Condition	Response
Current contract is unclear or docs disagree	Stop mutation, cite both sources, and request owner resolution.
Required access or approval is missing	Produce a draft and evidence plan only.
Validation or pilot behavior differs from expectation	Restore the prior state and retain the failed evidence.
Output contains secrets or private code	Stop, quarantine the artifact, redact it, and notify the data owner.

Examples

Example 1

Compare coverage before and after central config rollout.

Example 2

Reconcile CLI use with Review Log before buying credits.

Validation

Confirm every claim against the dated sources in references/official-docs.md.
Verify the requested scope, owner, approval, happy path, failure path, and rollback.
Re-read the effective configuration or provider state after any approved change.
Report unsupported fields, undocumented endpoints, and unverified assumptions as failures.

Resources

Official documentation and contract notes
Re-check the dated contract before any live operation.
Treat unresolved or changed vendor behavior as a stop condition.

coderabbit-observability