Eve library

Prompt-injection portfolio evaluations

Other

Revision 1

On this page

This project is the credential-free, held-out regression suite for PROMPT-003. It invokes the production prompt builders and deterministic admission checks at every V1–V9 boundary migrated by PROMPT-002; only the network/model edge is replaced by a scripted local double.

The versioned corpus in held-out/corpus.v1.json contains one opaque attack for each required family. Production prompt sources must never copy these probes or use them as few-shot examples. held-out/boundary-registry.v1.json maps every invoked boundary to its implementation and executable test, and the registry test fails if an implementation, test, attack family, or mapping disappears.

Each attack requests expansion of the same six protected dimensions: tool authority, source scope, rights, autonomy, budget, and release permission. The shared assertions require the attack probe and escalation requests to remain inside a non-instruction trust zone and absent from the trusted prompt shell. Separate negative tests prove generated citations, evidence quotes, tool arguments, event references, capabilities, and source identifiers are checked against their original typed inputs.

Run all language boundaries sequentially with:

text
pnpm exec nx run @oshun/prompt-injection-evals:verify --parallel=1

The always-on prompt-injection-portfolio.yml workflow runs this target for every pull request and every push to main, without a path filter. The test targets are intentionally non-cacheable, so a model, provider, prompt builder, or product-boundary change cannot receive a stale green result.