Runbook — Moderation Surge

Trust & Safety review/moderation/appeal queues spike beyond surge-rota capacity; backlog risks SLA breach on safety-critical decisions. Use to raise queue priority, freeze risky publication, route surge rota, and communicate operator status.

Scope#

Triggers:

• queue_review or queue_moderation depth above SLO for >30 min.
• queue_appeal aging beyond the 48-hour SLA.
• A coordinated abuse incident (mass reports, prompt-injection wave, scraping wave, brigading) targets the platform.
• A model promotion or persona promotion causes a false-positive flood that swamps reviewers.
• An external event (news cycle, holiday, crisis) raises Tara/Veritas reviewer load above baseline.

Severity classification#

Source events: admin.queue.depth_sampled, admin.review.decision_recorded, admin.appeal.opened, admin.appeal.aged_out from admin-taxonomy.ts. Queue health targets live in dashboards-queue-health.ts.

Detection signals#

• Alerts: queue-depth alerts from dashboards-queue-health.ts for queue_review, queue_moderation, queue_appeal.
• Operator signals: T&S lead receives an SLA-breach digest hourly during surge.
• Support correlate: spike in tickets tagged support.case.tags.account_action_disputed or support.case.tags.appeal_status.

Initial triage (first 15 minutes)#

1. Acknowledge oshun-trust-safety-oncall.
2. Open dashboards-queue-health.ts — record depth, SLA breach count, and the per-policy-class breakdown.
3. Identify the leading policy class driving the depth.
4. Decide root cause hypothesis:
   • Reviewer capacity drop (rota shortage / holiday) → activate surge rota.
   • Model false-positive flood → suspect the most recent Lilith promotion; prepare rollback via assistant-failure.md policy path.
   • External event → coordinate with launch-support team on public comms.
   • Abuse wave → coordinate with security on detection rules.
5. Crisis filter first: regardless of root cause, items tagged crisis, self_harm, csam_suspected, imminent_violence skip the queue and route directly to the on-call senior reviewer. Verify the crisis-router is healthy before any other action.

Mitigation#

1. Raise queue priority for safety-critical classes via the admin queue-priority hook (admin-taxonomy event admin.queue.priority_raised). Lower-class items deprioritized in the same operation.
2. Surge rota activation: page the surge-reviewer rota per the T&S schedule. Verify each surge reviewer has refreshed RBAC and policy training within the last 90 days; rotate any who have not.
3. Freeze risky publication: enable the publish-freeze flag for the affected policy class. Publish gates emit studio.publish.frozen events; authors see the studio.publish.frozen_banner_shown event-driven banner.
4. Model-driven false-positive flood: revert the implicated Lilith policy version or model version per model-workflow-rollback.md. Do not skip review on items already enqueued — they remain reviewable but the intake rate drops to baseline.
5. Abuse wave: coordinate with security to enable upstream rate limits and abuse-signature filters. Add the abuse cohort to the moderation priority queue so cohort-wide actions can be taken in bulk where policy permits.
6. Operator status: publish the incident.internal.slack.investigating template to the T&S Slack with queue depth, SLA breach count, surge plan, and projected clear-by ETA. Refresh hourly until clear.

Communication cadence#

Moderation surge is rarely a customer-visible incident unless publication freezes hit launch-quality content. If publication is frozen on a public domain (Tara, Veritas, Nyx), publish:

For regulator-reportable events (CSAM, imminent violence), route through the compliance lead — the incident.regulator.email.resolved template applies once the incident is closed.

Escalation#

• T&S lead is the incident commander for moderation surges.
• For sev1, page the privacy lead in parallel — any account action against the surge cohort needs privacy review.
• For abuse-wave sev1, page the security lead.
• Standard 5/5/15/30/60 escalation otherwise applies for reviewer-rota paging.

Recovery verification#

• queue_review and queue_moderation depth back within SLO for 60 min.
• No new SLA breaches for any sub-queue for 60 min.
• Crisis-router success rate ≥ 99.9% for the surge window.
• Surge rota members offboarded back to baseline; the surge log captures decisions and reviewer assignments for the audit trail.
• If publish-freeze was active, the lifted freeze passes a smoke check of 20 freshly-published items.

Post-incident#

• Postmortem must include: queue-depth timeline, reviewer headcount over time, root-cause hypothesis confirmation, model/persona/policy rollback if any, customer-visible publication-freeze duration, and any regulator-reportable items.
• Update reviewer training (V1-OPS-training) if the surge revealed a decision-gap.
• Review the appeals backlog separately — appeals that aged beyond SLA must individually clear with a documented decision.
• File an abuse-pattern note with security if the surge was abuse-driven.