# GATE-007 Certification: Observability Dashboards and Alerts Validated

Date: 2026-02-22  
Checklist Item: `GATE-007`  
Ticket: `ISIS-0257`

## Scope

Certify that Isis observability surfaces for lifecycle, provider reliability,
queue health, and cost anomaly detection are defined, wired, and validated via
deterministic checks.

## Evidence Matrix

| Requirement                                                                       | Evidence                                                                                                                                                                                                                                                                 | Status |
| --------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------ |
| Isis provider operations dashboard is defined for latency/failure/volume          | `infra/terraform/cloudwatch/dashboards.tf` resource `aws_cloudwatch_dashboard.isis_provider_operations` includes `ProviderExecutionLatencyMs`, `ProviderExecutionFailed`, and `ProviderExecutionTotal` visualizations.                                                   | Pass   |
| Isis cost dashboard is defined across endpoint/project/workflow/user dimensions   | `infra/terraform/cloudwatch/dashboards.tf` resource `aws_cloudwatch_dashboard.isis_cost` includes `ProviderExecutionCostByEndpointUsd`, `ProviderExecutionCostByProjectUsd`, `ProviderExecutionCostByWorkflowUsd`, and `ProviderExecutionCostByUserUsd`.                 | Pass   |
| Isis generation lifecycle dashboard is defined                                    | `infra/terraform/cloudwatch/dashboards.tf` resource `aws_cloudwatch_dashboard.isis_generation` includes lifecycle metrics `GenerationStarted`, `GenerationCompleted`, and `GenerationFailed`.                                                                            | Pass   |
| Provider anomaly alerts are defined for cost and failure-rate spikes              | `infra/terraform/cloudwatch/alarms.tf` resources `aws_cloudwatch_metric_alarm.isis_provider_cost_spike_anomaly` and `aws_cloudwatch_metric_alarm.isis_provider_failure_rate_spike_anomaly` use `ANOMALY_DETECTION_BAND` against provider execution cost/failure metrics. | Pass   |
| Queue/failure alerting path is defined                                            | `infra/terraform/cloudwatch/alarms.tf` includes `aws_cloudwatch_metric_alarm.runpod_queue_depth` and `aws_cloudwatch_metric_alarm.runpod_high_failure_rate`.                                                                                                             | Pass   |
| Metric ingestion for Isis provider telemetry is wired from canonical service logs | `infra/terraform/cloudwatch/logs.tf` scopes provider metric filters to Isis generation-api (`v.domain == "isis" && v.service == "generation-api"`) and emits `ProviderExecution*` + `ProviderExecutionCost*` metrics used by dashboards/alarms.                          | Pass   |
| Dashboard/alarm outputs are exported for operational consumption                  | `infra/terraform/cloudwatch/outputs.tf` exports `isis_provider_ops`, `isis_cost`, `isis_business`, and `provider_anomaly_alarm_arns`.                                                                                                                                    | Pass   |
| SLO metric contract remains aligned with observability gate                       | `docs/domains/isis/remediation/ISIS_SLO_DEFINITIONS.md` defines canonical metrics (`isis_generation_submit_latency_ms`, `isis_generation_queue_delay_seconds`, `isis_generation_terminal_total`, `isis_generation_e2e_completion_seconds`).                              | Pass   |
| Automated validation gate exists and passes                                       | `scripts/isis/validate_observability_stack.mjs` validates dashboards, alarms, metric filters, outputs, and SLO metric contracts; command succeeded with `Isis observability stack validation passed (67 checks)`.                                                        | Pass   |
| Alert dispatch behavior for readiness degradation remains tested                  | `apps/isis/generation-api/src/services/readiness-alert.service.spec.ts` passed to verify readiness alert dispatch + cooldown behavior for operational alert signal flow.                                                                                                 | Pass   |
| On-call runbook references the certified dashboard/alarm set                      | `docs/domains/isis/runbooks/runpod-oncall.md` lists primary CloudWatch dashboards and anomaly alert triggers used for incident response and operational validation.                                                                                                      | Pass   |

## Reproduction Commands

```bash
node scripts/isis/validate_observability_stack.mjs
pnpm --filter @isis/generation-api exec vitest run src/services/readiness-alert.service.spec.ts
```
