# Task 3.1 independent relevance-label review packet

> Status: **awaiting two independent human reviews**. This packet contains no
> proposed relevance labels and is not eligible human evidence.

- Set: `eve.relevance-set.v1`
- Set source SHA-256:
  `9863f64a266171ba76c5276c631d0c874496bc1aaf1f49d389de9320ccba295e`
- Inventory SHA-256:
  `7da1f77c35b847787f29ea34f0f1784e047421c5aee39c345a1dd87dbe65ccf5`
- Inventory content hash: `3d5f56907f27` (3256 pages)
- Member corpus: 0 pages / 0 chunks

## Protocol

1. Assign this packet to two distinct human reviewers. They must work
   independently.
2. Inspect the source documents directly. Do not use the BM25F system under
   evaluation, its result order, or model-generated relevance proposals.
3. For every query, record every relevant document as its exact `href` from the
   bound inventory. Lists must be non-empty, unique, and lexicographically
   sorted.
4. Each reviewer records their own rationale. The two href lists must match
   exactly; any mismatch requires independent human adjudication.
5. Create `human-labels.reviewed.json` beside this packet with schema
   `eve-sota-relevance-human-labels.v1`, the exact `corpusBinding` from the
   pending JSON, a canonical `reviewedAt`, and two reviewer objects containing
   the fields and labels shown by the pending template.
6. Run `node tools/eve-everywhere/verify-relevance-set-human-review.mjs`. A
   passing receipt can then be admitted into the versioned relevance set; it
   does not by itself close downstream retrieval-quality tasks.

## Queries

### concept-admin-intent-plane

- Class: `conceptual`
- Audience: `admin`
- Query: According to the documentation, how does the builder workbench intent
  plane work?

Reviewer 1 hrefs and rationale:

Reviewer 2 hrefs and rationale:

### concept-admin-reading-positions

- Class: `conceptual`
- Audience: `admin`
- Query: how do reading positions work?

Reviewer 1 hrefs and rationale:

Reviewer 2 hrefs and rationale:

### multihop-admin-lease-to-verify

- Class: `multi-hop`
- Audience: `admin`
- Query: What has to happen between an agent taking a lease and an item reaching
  verified?

Reviewer 1 hrefs and rationale:

Reviewer 2 hrefs and rationale:

### recency-admin-current-gates

- Class: `recency`
- Audience: `admin`
- Query: Which ADR-0076 implementation gates are still open?

Reviewer 1 hrefs and rationale:

Reviewer 2 hrefs and rationale:

### contradictory-admin-member-corpus

- Class: `contradictory`
- Audience: `admin`
- Query: Is the member documentation corpus the curated docs-center, or is it
  empty?

Reviewer 1 hrefs and rationale:

Reviewer 2 hrefs and rationale:

## Required attestation per reviewer

Identify yourself and your role, confirm that you manually inspected source
documents for all five queries, worked independently of the other reviewer, used
neither the evaluated BM25F ranker nor model-generated labels, and stand behind
every href and rationale in your label set.