Disciplines · Security

Pre-GA Penetration Test Scope

This document is the authoritative scope for the pre-GA penetration test required by §28.20.

9sections3 minread
Markdown source
On this page

Version: 1.0 (locked at signoff) Status: ready for engagement Last reviewed: 2026-05-13 Owner: head of security Spec: V1/TODOS.md §28.20

1. Purpose#

This document is the authoritative scope for the pre-GA penetration test required by §28.20. It is the contract between the engaging team (Oshun security) and the testing party (internal red-team plus an external firm). The scope, methodology, blackout windows, and signoff criteria are all defined here; deviations require written approval from the head of security AND the head of engineering.

2. In-scope assets#

2.1 Customer surfaces#

  • Customer web app (oshun.ai, *.staging.oshun.ai)
  • Customer mobile apps (iOS / Android beta builds)
  • Public REST APIs documented in openapi/

2.2 Admin surfaces#

  • Admin web app (admin.oshun.ai, admin-staging.oshun.ai)
  • Admin BFF (bff-admin-staging.oshun.ai)
  • Operator workspaces: support, review, moderation, privacy, model, persona, leadership (per the canonical role model in libs/shared/auth-primitives/src/platform-roles.ts)

2.3 Service-to-service#

  • Inter-service mTLS boundaries (@oshun/identity/mtls)
  • Tenant isolation guard (@oshun/auth-primitives/tenant-isolation)
  • Webhook ingress / outbound

2.4 Trust paths required to be exercised#

  1. Authentication – credential, OAuth, OIDC, step-up TOTP, recovery.
  2. RBAC – platform role + scope matrix (least-privilege escape attempts: SUPPORT → privacy, MODERATOR → moderation override, etc).
  3. Privacy – DSAR flow, retention, redaction, residency, cross-tenant PII leakage, audit-log immutability.
  4. Metis – institutional APIs, subject routing, standards interop.
  5. Grounded generation – prompt injection, tool-call abuse, sandboxing of plugin code, output filter bypass.
  6. Admin escalation paths – breakglass timing, scope-grant audit, audit-log tampering attempts.

3. Out-of-scope#

  • V2 (Unreal Engine racing module) – separate scope, separate vendor.
  • Third-party billing and IDP backends – we will share their SOC 2 attestations; testing must not target their endpoints directly.
  • Denial-of-service against shared infrastructure.
  • Social engineering / phishing against employees – red-team activity is covered by red-team/scenarios.md, not this engagement.

4. Methodology#

The engagement follows a hybrid grey-box model:

  • Day 1–2: orientation (architecture deep-dive with the on-call engineer pool, OpenAPI spec walkthrough, RBAC matrix walkthrough).
  • Day 3–14: active testing with provisioned credentials for each operator role. Two parallel cells: web-cell (customer + admin) and API-cell (REST + service-to-service).
  • Day 15: focused round-robin on findings discovered during day 3–14.
  • Day 16–17: report drafting, validation re-test.
  • Day 18: readout + signoff workshop.

5. Rules of engagement#

  • Testing windows: 08:00–22:00 UTC weekdays only.
  • A staging slot is reserved; production is not in scope unless a finding requires production confirmation, in which case the head of security must explicitly approve in writing.
  • All payloads must be tagged with the engagement header X-Pentest-Engagement: <ENG-ID> so SIEM can distinguish from real traffic.
  • No tooling that creates persistent backdoors, downloads customer data, modifies billing state, or sends customer notifications.
  • Findings must be filed in the shared tracker within 24h of discovery with severity, reproduction steps, and proposed remediation.

6. Severity classification#

Maps to CVSS 3.1 base score:

Severity Score SLA to fix
Critical 9.0–10 24 hours, release-blocking
High 7.0–8.9 7 days, release-blocking
Medium 4.0–6.9 30 days, release-warning
Low 0.1–3.9 90 days, tracked
Info 0.0 Tracked, no SLA

7. Release-blocking criteria#

Per §28.20, GA cannot proceed if any of the following is true:

  1. One or more open Critical findings.
  2. Two or more open High findings.
  3. Any unresolved finding in the authentication, RBAC, or privacy categories regardless of severity.

8. Sign-off#

Signoff is recorded in signoff.md and requires three signatures:

  • Head of security
  • Head of engineering
  • Head of privacy (if any privacy-tagged findings were filed)

Without all three, the release pipeline (controlled by the §28.8 canonical rollback plans) refuses to promote the build to GA.

9. Evidence retention#

All artefacts — scoped credentials, test plans, raw findings, the remediation diff, the retest result, the signoff — are retained for seven years in the secure evidence vault. The artefact set is referenced from compliance/pentest-artefacts.md and the path is audited under the canonical immutable audit log (@oshun/security/audit-logger).