Disciplines · Audits

Qa Review Prompt

Gap audits and as-built reviews.

0sections2 minread

On this page

Act as the independent QA Lead for Eve SOTA task 11.11. This is a read-only, adversarial review: do not edit files, commit, push, or use external services.

Read the exact task clause in docs/audits/eve-planning-fleet-lifecycle/v1.0.0/task-11-11-clause.txt and the Task 11.11 evidence manifest. Treat source.implementationCommit as the task's implementation snapshot, source.commit as the exact tree against which the recorded commands ran, and source.evidenceSnapshotCommit as the later commit that must contain every retained committed artifact at its declared hash. Check that ancestry is implementation → execution source → evidence snapshot. The manifest itself is deliberately not one of its artifacts: requiring it to name the commit containing that same manifest would be a cyclic self-reference. Inspect the implementation commit and its first-parent diff, plus the execution and evidence-snapshot trees and current retained closeout artifacts. Then inspect these authoritative areas:

  • planning fleet contract and lifecycle/event-integrity migrations;
  • Prisma lifecycle models;
  • workbench intent store, canonical queue, planning goal/plan stores;
  • agent authentication, workbench and planning fleet routes;
  • fleet drain, workbench MCP hand path, and child isolation;
  • real PostgreSQL lifecycle integration tests, semantic verifier, database receipt, separate-process restart probe, and Task 11.11 evidence manifest.

Try to falsify all eight checks in the response schema. Pay special attention to cross-tenant access, stale plan lease/report/close paths, lock ordering, replanning and decision invalidation, active-lease triage, shipped-history preservation, exact proof joins and event ordering, self-verification, generic verifier bypasses, restart behavior, and whether the substantive fleet implementation artifacts (contract, migrations, schema, stores, queue, routes, hand/drain/isolation paths, and lifecycle tests) match the implementation snapshot. Evidence/provenance machinery may land between that implementation snapshot and the execution source. Every retained artifact hash must match both the working tree and the evidence snapshot. Missing, unordered, or role-confused commit provenance fails evidence integrity even if working-tree hashes match.

Set verdict to fail if any blocking or high-severity finding exists, any check is false, the task clause is not fully covered, or the evidence claims more than the implementation proves. Otherwise set it to pass. Report real findings only; keep environmental or out-of-scope boundaries in limitations. Return only JSON matching the supplied schema.

The runtime is deliberately bounded. Inspect the source-commit diff, retained closeout artifacts, and targeted symbols/tests rather than reading unrelated whole files. Stop exploration by minute 10 and reserve the remaining time for a decisive final JSON result; an unfinished review is not a pass.