Eve library

Quality Metrics Dashboard Runbook

Runbook

Revision 1

On this page

Use this runbook to capture and review quality signals across the monorepo. The dashboard job is lightweight and runs in CI (quality-metrics in .github/workflows/quality-gates.yml), but can also be executed locally.

What is collected#

  • Coverage: coverage/coverage-summary.json (generated by your coverage run) lines percentage.
  • Suppressions: Counts of eslint-disable and @ts-ignore usages.
  • Technical debt markers: TODO, FIXME, and BUG tag counts.
  • Dependency health: Number of outdated packages from npm outdated.
  • Documentation coverage: Ratio of Markdown docs in docs/ vs. code files in services/, sdk/, web/, and mobile/.

Prerequisites#

  • Node 18+ with npm.
  • Repo dependencies installed (npm ci at repo root).
  • Optional but recommended: a recent coverage run that produced coverage/coverage-summary.json.

How to run locally#

  1. Ensure coverage exists (optional but recommended):
    bash
    npm run coverage:ci # or any task that writes coverage/coverage-summary.json
    
  2. Capture the metrics snapshot:
    bash
    npm run metrics:dashboard
    
  3. Review the outputs:
    • Latest snapshot: reports/metrics/dashboard-latest.json
    • History (appends on every run): reports/metrics/dashboard-history.json

CI behavior#

  • Workflow: quality-metrics job in .github/workflows/quality-gates.yml.
  • Artifacts: quality-dashboard (contains dashboard-latest.json and dashboard-history.json).
  • Coverage will be null if no coverage/coverage-summary.json is present; run a coverage task before invoking the dashboard to populate it.

When the job fails#

  1. Inspect the job logs for the failing probe (coverage missing, npm outdated failures, or filesystem errors writing reports).
  2. Download the quality-dashboard artifact to see the partial results and confirm which metric is missing.
  3. Common fixes:
    • Re-run coverage before metrics: npm run coverage:ci && npm run metrics:dashboard.
    • Regenerate reports locally and commit if the script was updated.
    • If npm outdated fails due to registry/network, re-run with npm config set fetch-retries 3 or retry the workflow.

Ownership and thresholds#

  • Owners: Platform DX (default reviewers for .github/workflows/quality-gates.yml).
  • Targets: Keep coverage ≥70%, drive suppression counts down over time, and avoid growth in TODO/FIXME tags week over week. Escalate in #eng-quality if debt metrics increase for two consecutive runs.

Interpreting results#

  • Rising suppression or technical debt counts indicate code quality drift—schedule remediation.
  • A non-null dependency_health.outdated_count > 0 means npm outdated found packages to upgrade.
  • Low documentation.doc_to_code_ratio signals missing docs; prioritize writing or updating guides.