Status: Accepted
Date: 2026-02-23
Authors: OSHUN Studio Architecture, OSHUN Data Platform Engineering, OSHUN
Web Engineering, OSHUN BFF Engineering, Platform SRE
Reviewers: Domain Leads (Yemaya, Isis, Hathor, Aja, Bellona), Project
Obsidian Program Design, Security and Compliance
Context and Problem Statement#
Studio coordinates critical workflows across Yemaya, Isis, Hathor, Aja, Bellona, and Project Obsidian. Cross-domain data and operational states now require deterministic backup, restore, and disaster-recovery visibility for operators.
Recovery actions currently rely on fragmented operational tooling and domain-specific playbooks. Without a first-class backup and disaster-recovery UX, Studio risks:
- slow and inconsistent recovery execution during incidents
- unclear blast-radius and recovery-point visibility for operators
- weak governance evidence for backup/restore decisions
- higher chance of unsafe restores or incomplete state reconciliation
Decision Drivers#
- Operational resilience: recovery paths must be deterministic and fast.
- Safety: restore workflows must preserve state integrity and guardrails.
- Traceability: every backup/restore action requires auditable evidence.
- Consistency: one cross-domain UX model for recovery orchestration.
- Scalability: support increasing project count and recovery complexity.
Considered Options#
Option 1: Infrastructure-Only Recovery Tooling#
Keep recovery orchestration in infrastructure/operator tools without Studio UX.
Pros:
- Lower immediate Studio implementation overhead.
Cons:
- Poor workflow visibility for Studio operators.
- Weak cross-domain governance and evidence continuity.
Option 2: Domain-Specific Recovery Screens#
Each domain provides local backup and disaster-recovery interfaces.
Pros:
- Domain customization flexibility.
Cons:
- Fragmented operator experience and uneven safety guardrails.
- Harder to coordinate cross-domain recovery scenarios.
Option 3: Canonical Studio Backup and Disaster Recovery UX (Chosen)#
Adopt one Studio UX contract for backup plans, restore simulation, failover execution, and post-recovery verification with policy-aware governance.
Pros:
- Consistent and safer recovery operations across domains.
- Faster incident response through one control surface.
- Better auditability and compliance evidence for restore decisions.
Cons:
- Requires sustained governance across web, BFF, and platform operations.
Decision Outcome#
Chosen option: Option 3.
Studio backup/disaster-recovery UX requirements:
- Backup plan governance contract for scope, cadence, retention, and owner metadata.
- Restore simulation contract for dry-run validation and impact analysis.
- Failover and restore execution contract with explicit policy/approval controls.
- Recovery verification contract for integrity checks and closure evidence.
- Telemetry/audit continuity contract for disaster lifecycle operations.
Normative Rules#
Backup Plan Integrity Rule#
- Backup plans must include source scope, schedule, retention window, and owner attribution.
- Plan changes must be versioned and reason-coded.
- Missing required metadata must fail plan publish closed.
Restore Simulation Rule#
- Restore actions must support dry-run simulation before production execution.
- Simulation output must include affected entities, estimated duration, and integrity risk indicators.
- Simulation failures must block promote-to-execute transitions.
Failover and Execute Governance Rule#
- Production restore/failover actions require role/tier/approval validation.
- Restore execution must include deterministic checkpoint and rollback metadata.
- Unauthorized execution attempts must emit deny-path audit evidence.
Recovery Verification Rule#
- Recovery completion requires validation of cross-domain integrity checks.
- Open discrepancies must block incident closure.
- Verification outcomes must be attached to recovery evidence package.
Telemetry and Audit Continuity Rule#
- Backup, simulation, failover, and verification steps must emit telemetry and audit records.
- Missing telemetry/audit continuity must trigger degraded-observability mode.
- Governance exports must include actor, policy, action, outcome, and trace lineage.
Architecture Implications#
- Studio web introduces dedicated backup/disaster-recovery workspace flows.
- BFF exposes backup plans, simulation, restore, and verification APIs with policy enforcement.
- Platform services provide snapshot inventory, failover orchestration, and checkpoint evidence.
- Operational dashboards and runbooks require recovery SLOs and escalation mappings.
Acceptance Criteria (OST-00305)#
OST-00305 is complete only when:
- ADR exists at
docs/adr/ADR-0057-oshun-studio-backup-and-disaster-recovery-ux.md. - ADR defines options, trade-offs, and selected strategy.
- ADR defines backup plan, restore simulation, failover/execute governance, recovery verification, and telemetry/audit rules.
- ADR aligns with
ADR-0007throughADR-0050, especiallyADR-0012,ADR-0039,ADR-0041, andADR-0050. - ADR aligns with existing resilience, observability, and operational governance surfaces.
- ADR explicitly covers Yemaya, Isis, Hathor, Aja, Bellona, and Project Obsidian.
Consequences#
Positive#
- Stronger and faster disaster-recovery operations across Studio domains.
- Better recovery safety via simulation-first and policy-gated execution.
- Improved compliance/audit posture for backup and restore governance.
Negative#
- Increased governance and operational complexity for recovery workflows.
- Requires coordinated ownership between web, BFF, platform, and data teams.
Related Decisions#
docs/adr/ADR-0013-oshun-shell-architecture-and-domain-adapters.mddocs/adr/ADR-0018-analytics-taxonomy-and-event-naming.mddocs/adr/ADR-0045-oshun-studio-resilience-and-error-ux.mddocs/adr/ADR-0047-oshun-studio-observability-and-operational-dashboards.mddocs/adr/ADR-0056-oshun-studio-data-retention-and-lifecycle-controls.md
References#
docs/releases/v1/runbooks/studio-resilience-and-error-ux-runbook.mddocs/releases/v1/runbooks/studio-observability-and-operational-dashboards-runbook.mddocs/releases/v1/runbooks/studio-multi-project-operations-runbook.mddocs/releases/v1/runbooks/on-call-runbook.md