Approved by @GreyChimp on 2026-08-14. The approval is the governing act; the writing is not.
The eleven subjects YSD-18001 names, each as a rule that refuses something. A rule phrased only as an intention cannot be violated and so cannot be complied with; every entry below therefore says what becomes impossible.
gold-set-versioning#
Every gold set is a named version whose manifest states work, edition and build identity, licences, allowed uses, source hashes, slices, annotation version, adjudication and known limitations.
Refuses. scoring against "the gold set" as though it were one thing. Two runs quoting one score must be able to prove they read the same version, and a set with no version cannot be re-read after it moves.
Accountable role. evaluation owner
Carried by. libs/yemaya/study-workspace/src/evaluation/dataset-manifest.ts, libs/yemaya/study-workspace/src/evaluation/gold-set-slices.ts
annotation-guides#
Every label cites the guide version it was made under, and a revision declares per task whether it changed a meaning or said the same thing better.
Refuses. pooling labels across a meaning change. No diff of prose distinguishes a clarification from a reversal, so the declaration is required and pooling across an undeclared revision is refused rather than averaged.
Accountable role. evaluation owner
Carried by. libs/yemaya/study-workspace/src/evaluation/annotation-guide.ts
adjudication#
An item is adjudicated to one label only when its task's answer model is single-answer; an interpretive item's disagreement is preserved rather than resolved.
Refuses. adjudicating an interpretive item to one label, which makes the score a measure of conformity to whichever reading won — it rises as a system gets narrower, and every system giving the other supported reading is counted wrong.
Accountable role. evaluation owner
Carried by. libs/yemaya/study-workspace/src/evaluation/annotation-guide.ts
legitimate-disagreement#
A bounded disagreement holds the readings the material supports and states what puts a reading outside them; any supported reading scores correct.
Refuses. treating a beginner who chose a different supported reading as having made an error. Inside the bounds it is a difference; only outside them is it teachable, and an evaluation without that distinction reports the corpus’s interpretive richness as learner failure.
Accountable role. domain expert panel
Carried by. libs/yemaya/study-workspace/src/evaluation/bounded-readings.ts
licenses#
No item enters a gold set before its licence is recorded and its allowed uses are stated; evaluation is an allowed use or the item is not evaluated against.
Refuses. acquiring evaluation material on the strength of it being available. Reachable is not licensed, and a set assembled that way cannot be published or shared without re-clearing every item.
Accountable role. rights owner
Carried by. libs/yemaya/study-workspace/src/evaluation/dataset-manifest.ts, libs/yemaya/study-workspace/src/corpus/open-corpus-catalog.ts
consent#
Material of a private performance, a minor, or an identifiable person enters evaluation only under recorded consent naming that use, and consent is withdrawable.
Refuses. reading consent for one purpose as consent for evaluation. A performance shared for study is not thereby a benchmark item, and withdrawal after the fact must remove the item rather than being recorded as regrettable.
Accountable role. rights owner
Carried by. libs/yemaya/study-workspace/src/policies/minor-consent.ts, libs/yemaya/study-workspace/src/policies/audience-consent.ts
privacy#
Evaluation records carry no identifier that is not needed to score, and protected attributes are slices for measuring fairness rather than inputs a product may infer.
Refuses. turning a fairness slice into a product feature. Skin tone, age, disability presentation and voice type exist in the manifest so performance can be measured across them, and a model reading them as inputs is the harm the slicing exists to detect.
Accountable role. privacy owner
Carried by. libs/yemaya/study-workspace/src/policies/telemetry-privacy-filter.ts
retention#
Evaluation material is retained under the class its rights basis implies, and an item whose basis lapses leaves the set rather than being kept for comparability.
Refuses. keeping a lapsed item so that a historical score stays reproducible. The score becomes irreproducible and that is the correct outcome; the alternative is holding material with no basis in order to protect a number.
Accountable role. rights owner
Carried by. libs/yemaya/study-workspace/src/policies/object-namespaces.ts
access#
Reading evaluation material is a named act with a stated purpose, and every read is re-decided against the live rights and consent state.
Refuses. a standing grant that outlives the reason it was given. A window opened for one review does not authorize the next one, and a read that does not say what it is for is refused before anything is returned.
Accountable role. evaluation owner
Carried by. libs/yemaya/study-workspace/src/policies/support-access.ts
publication#
A published evaluation report states failures, uncertainty, limitations, slice performance, reviewer disagreement and the release decision it supported.
Refuses. publishing a headline number without the slices under it. An aggregate that hides a slice it performs badly on is the specific dishonesty the slicing was built to prevent.
Accountable role. evaluation owner
Carried by. libs/yemaya/study-workspace/src/evaluation/model-card.ts
deletion#
Deletion of an item or a participant’s contribution propagates to every derived record, and a deleted item leaves a tombstone rather than a gap.
Refuses. a deletion that leaves derived scores, embeddings or projections behind. A gap with no tombstone is indistinguishable from an item that never existed, which makes an executed deletion unprovable.
Accountable role. privacy owner
Carried by. Nothing yet — declared and unenforced, which is recorded rather than hidden.
What this document does not yet hold#
1 rule(s) are declared and enforced by nothing: deletion. They govern acts that cannot yet occur or mechanisms not yet built, and are stated so the gap is visible rather than absent.