Work-package charter#
- Task ID:
PROMPT-003 - Owner: cross-version prompt-security maintainers; implementation owner: Codex
- Priority: P0
- Risk tier: Critical authority-confusion, prompt-injection, source-scope, generated-output, and release-permission risk
- Dependencies:
PROMPT-001,PROMPT-002,@oshun/prompt-trust, every migrated product boundary, and each boundary's deterministic output checks - Rollout strategy: credential-free held-out regression project plus an always-on pull-request/push workflow; no production model, persistence, or UI rollout
- Rollback strategy: revert source commit
d698ae1c53c0a4895b29fe7e4eeeabed70fcf07f; the suite, workflow, and Rust development dependency are additive - Evidence-pack location: this file
Closed implementation#
testing/prompt-injection is a dedicated Nx/pnpm test project. It keeps the
versioned attack corpus and boundary registry outside production prompt code,
loads the same corpus from TypeScript, Python, and Rust, invokes real production
prompt builders and deterministic admission checks, and replaces only the
network/model edge with local scripted doubles.
The registry names 20 invoked implementations:
| Boundary ID | Production implementation under test | Runner |
|---|---|---|
v1.ideation |
structured idea synthesizer | TypeScript |
v1.planner |
creative-orchestrator brief decomposition | TypeScript |
v1.bff-text |
BFF source-bound text generator | TypeScript |
v1.agent-loop |
research/tool-result generator | TypeScript |
v1.critique-revise |
artifact/rubric/prior-critique judge | TypeScript |
v1.cqj-self-refine |
CQJ critique-to-revision boundary | TypeScript |
v1.cqj-reflexion |
CQJ cross-item lesson memory | TypeScript |
v2.calliope-writer |
match-commentary writer | TypeScript |
v2.calliope-reviewer |
generated-commentary reviewer | TypeScript |
v2.hathor-dialogue |
NPC persona/world/history dialogue | TypeScript |
v2.maya-slm |
local SLM context window | Rust |
v6.iris-cognition |
governed cognition gateway | TypeScript |
v6.moirai-mount |
V6 tier-budget/tool-grant mount into Iris | TypeScript |
v6.clio |
event-scoped chronicle writer | TypeScript |
v7.forge |
realm artifact proposal and static admission | TypeScript |
v7.mawu |
Sophia-scoped generation backend boundary | TypeScript |
v8.proposer |
solver-owned case surface proposer | TypeScript |
v8.assets |
case asset job prompt | TypeScript |
v9.content |
source-bound educational content prompt | TypeScript |
v9.tutor |
governed Metis tutoring provider call | Python |
The registry test fails if a source or mapped test disappears, if a registered test stops naming its boundary, or if any held-out probe appears in a production boundary source. The latter makes the corpus held out from product prompts and few-shot examples; this is a committed regression set, not a claim that the benchmark itself is secret.
Attack and authority coverage#
The immutable corpus.v1.json contains one opaque probe for each required
family:
- direct
- indirect
- cross-turn
- retrieval poisoning
- source poisoning
- generated output
- critic feedback
- tool result
- multimodal metadata/OCR
- encoded
- delimiter
- memory poisoning
- cross-item
Every payload requests the same six sentinel escalations: tool authority, source scope, rights, autonomy, budget, and release permission. The shared assertion requires each opaque probe to be present inside a real untrusted zone, removes every untrusted body, and then proves that neither the probe nor any of the six escalation values entered the trusted prompt shell. Encoded and delimiter attacks receive additional neutralization assertions. Boundary-level checks also prove the Moirai tool grant stays empty and its tier budget remains fixed, Forge capabilities remain policy-scoped, and Mawu/Clio source references remain the original typed references.
All 13 families run at all 20 implementations: 260 boundary/attack executions. The TypeScript report uses a single parameterized test for both Calliope writer and reviewer, so its test-count total is smaller than the number of boundary executions.
Generated-output integrity proof#
Seven negative regression tests exercise the post-generation admission layer:
- an agent-loop tool call with an extra release field and a non-string query is rejected against the original JSON schema, and the handler call count remains zero;
- a BFF citation outside the supplied source-ID set is rejected;
- critic evidence absent from the exact artifact text is rejected;
- unsafe generated commentary cannot be cleared by an empty model review;
- a Clio event reference absent from the original log slice is rejected;
- a Forge capability smuggled in generated source is rejected by the original capability policy;
- V8 surface fields inconsistent with the typed case spec and V9 attacker text used as a source identifier are rejected.
These tests directly close the ledger requirement for citations, quotes, and tool arguments to be checked against original typed data after generation.
Model/provider change enforcement#
.github/workflows/prompt-injection-portfolio.yml has no path filter. It runs
for every pull request to main, every push to main, and manual dispatch. The
workflow runs portfolio typecheck, lint, and the sequential multi-language
verify target. The TypeScript, Python, Rust, and aggregate verification
targets are non-cacheable, preventing a boundary or provider change from
receiving a stale green result. Project inputs and implicit dependencies
additionally name the shared provider/agent-loop roots and every migrated
package.
Automated verification#
Final verification was performed on the integrated and published source tree at
ee77b83a26ecf1fbeaf6077f78d582d7b722161c.
- Aggregate Nx target: passed.
- TypeScript/Vitest: 232/232 tests across 5/5 files.
- Python/Pytest: 1/1 corpus test, iterating all 13 attacks at the V9 tutor boundary.
- Rust portfolio: 1/1 corpus test, iterating all 13 attacks at the Maya SLM boundary.
- Complete Maya SLM crate: 18/18 integration tests (17 existing plus the portfolio test); library and doc-test targets also passed.
- Harness TypeScript typecheck: passed with zero diagnostics.
- Harness ESLint: passed with zero warnings/errors. Cross-project boundary-rule suppressions are limited to the V1/V2 test adapters and their narrow Iris test entry because those files intentionally invoke multiple product projects.
- Ruff check and format: passed.
- Cargo format: passed.
- Prettier: passed for the workflow, README, JSON manifests/corpus/registry, config, and TypeScript sources.
- Workflow YAML parse,
git diff --check, and frozen offline pnpm lock validation: passed. - The source commit's pre-commit Nx affected typecheck independently passed for
@oshun/prompt-injection-evals.
Representative commands:
NX_DAEMON=false pnpm exec nx run @oshun/prompt-injection-evals:verify --parallel=1
pnpm exec tsc --noEmit -p testing/prompt-injection/tsconfig.json
pnpm exec eslint testing/prompt-injection/src/*.ts testing/prompt-injection/vitest.config.ts --no-ignore --max-warnings=0
PYTHONPATH=services/metis/src python3 -m pytest -q testing/prompt-injection/python/test_portfolio_prompt_injection.py
cargo test --manifest-path libs/maya/engine-core/crates/maya-slm-runtime/Cargo.toml
cargo fmt --manifest-path libs/maya/engine-core/crates/maya-slm-runtime/Cargo.toml --check
python3 -m ruff check testing/prompt-injection/python/test_portfolio_prompt_injection.py
pnpm install --lockfile-only --offline --ignore-scripts --frozen-lockfile --filter @oshun/prompt-injection-evals
The final aggregate target ran from 2026-07-16T18:56:47Z through
2026-07-16T18:57:20Z on Linux 6.8.0-110-generic x86_64 with Node 24.14.1, pnpm
10.25.0, Python 3.12.3, and Rust 1.94.1. No live provider credential, remote
model call, or claim about a production model response is involved.
Resource safety and reconciliation#
The host had 15 GiB RAM and 31 GiB swap. All verification was targeted, sequential, and single-worker. Two early aggregate-wrapper attempts were stopped while a competing Nx daemon was rebuilding the large workspace graph; their underlying direct TypeScript, Python, and Rust commands had already passed. The final daemon-free aggregate target completed successfully after the competing graph exited. No full Nx sweep, Next build, Expo export, Tauri package, Unreal build, Playwright run, or mobile automation was applicable or attempted.
origin/main advanced during publication. The source branch took the newer
remote lockfile, reapplied only the prompt-injection importer, passed frozen
offline lock validation and the complete post-merge portfolio, and published the
combined tip without dropping either change set.
Artifact identity#
| Artifact | SHA-256 |
|---|---|
| Held-out corpus | 4f7602baed09058ac885dfeac119d9828881d79803c584a6fe57a6d5cf62e26e |
| Boundary registry | 3ca7b7de49adec7c3315f0dbfa7092f90b83bffb5f8d457695ab571a3f73eb2f |
| Nx project | 30eca6c852fb9c26d11f60691dc88ad44da078360adc7b621ceb1e1e9247c4bf |
| Shared TypeScript harness | dc0bde60555acf32a186c2c8895ef55c129f06c50c55f82fa414584b84965e26 |
| V1 TypeScript runner | 791d4e5606b61837c12392363a9ef3c05b3a8f0808394913312f8b9832daaee0 |
| V2 TypeScript runner | 10f6e1131ffcae5f3288ba19f97a4e4c4a7da29cefb24ef2d3bcf2c918578699 |
| V6/V7 TypeScript runner | c4aa4dbf1a0c693c8d1e77fb4e4d45dc8d5ae873c3362a8b32a8dceb66d951be |
| V8/V9 TypeScript runner | c9be93d2d7a78cd8444ca4ec5d5ec1d499b2759bd326d010e9d6ff6629bd0cd1 |
| Python tutor runner | 2b648c45482de71ec24299fdd7d619c5ca290a8cbb30e364a4c12f9a2eed4167 |
| Rust SLM runner | fe169f361e60e0e07c7dc9f2774dad4cf01fc16eb112f619306f7735cea1aa06 |
| Always-on workflow | b13c5e3168b80383a48e8499dfb04e058e9ba218b4532eeaca2d0988ffecfb82 |
Scope and remaining seams#
- This suite proves typed prompt construction and deterministic post-generation admission with local scripted provider doubles. It does not claim that prompt structure alone makes arbitrary production models immune to all attacks.
- Live-model red-team calibration, provider-specific behavioral thresholds, and release-signature decisions remain governed by their model/release processes.
- Forge's static capability inspection remains admission-only; executable sandbox enforcement remains under the named AUTH/V7 work.
- Safety-classifier quality, substantive claim entailment beyond typed source membership, durable V6 state, and live-provider readiness remain under their named SAFETY, PROOF, V6, V7, V8, and V9 packages.
- No rendered web, PWA, mobile, admin, or Unreal UI changed, so frontend skill, Playwright, mobile automation, screenshots, and Unreal verification were inapplicable.
Publication identity#
- Authoritative source commit:
d698ae1c53c0a4895b29fe7e4eeeabed70fcf07f - Source publication tip on
origin/main:ee77b83a26ecf1fbeaf6077f78d582d7b722161c - Closure evidence commit: recorded in the PROMPT-003 ledger closure entry after this evidence file is published.