Evidence for 182.C.19.09: fairness slices for dark skin, varied lighting,
facial hair, head coverings, makeup, glasses, age, and disability-related facial
or motor differences.
Populating the slices needs subjects, which is 182.C.33, so this item stays
unchecked. What is built is the machinery that makes a corpus report anything
useful — and one measurement showing why the machinery has to exist.
Method#
Two subjects from the committed frame, differing by 63.7° of Individual Typology Angle, through nine exposure levels from −2 to +2 stops.
| stops | median lum | s0 detected | s0 shape | s0 complexion | s1 detected | s1 shape | s1 complexion |
|---|---|---|---|---|---|---|---|
| −2.0 | 11.4 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| −1.5 | 17.6 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| −1.0 | 25.9 | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ |
| −0.5 | 36.9 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| 0.0 | 51.8 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| +0.5 | 71.0 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| +1.0 | 94.9 | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ |
| +1.5 | 100.0 | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ |
| +2.0 | 100.0 | ✓ | ✓ | ✗ | ✓ | ✓ | ✗ |
| detected | face shape | complexion | |
|---|---|---|---|
| subject 0 (ITA −11.6) | 9/9 | 9/9 | 8/9 |
| subject 1 (ITA +52.1) | 9/9 | 9/9 | 5/9 |
The finding#
Both subjects were detected in every condition and both had their face geometry measured in every condition. What differed was the complexion measurement, which depends on the face-skin segment having enough pixels: the lighter-complexioned subject lost it in four conditions and the darker one in one.
Aggregate numbers hide this. Pooled, the complexion measurement succeeded 13
of 18 times — 72%, a number that describes neither subject (they were 89%
and 56%). An aggregate is the shape a fairness fault hides in, and
assertSliced refuses one that carries no slices.
Availability is the thing to slice, not just accuracy. Every gate in this phase distinguishes "measured and passed" from "not measurable", so a subject whose attribute is more often unmeasurable gets less scrutiny, not more. That is a fairness fault with no wrong answer in it anywhere, and it would be invisible to any report that only tracked pass rates.
The direction was not the expected one. The commonly-cited failure is dark skin under-detected in low light. Here the lighter subject lost more measurements, under brightening.
This is not a fairness evaluation#
Two subjects. This establishes that per-subject availability differs and
nothing whatsoever about a population. SEED_OBSERVATION.isFairnessEvaluation
is false in code, and MIN_SUBJECTS_PER_BUCKET is 30 — deliberately larger
than anything currently available, so no bucket is powered today and
assertBucketPowered refuses a rate from any of them.
What the two subjects do prove is that assuming the direction would have been wrong on this pair.
What the machinery does#
- Eight slices, each with buckets and a note on why it matters here. Skin tone
and lighting are measured; the rest are declared — a facial-hair or
makeup classifier would be a second uncalibrated model, and age is declared
and never estimated for the reason
182.C.19.02records. - Facial or motor differences are declared by the subject and by nobody else, and the note says why the slice matters: every artifact detector in this phase treats asymmetry and irregular motion as faults.
availabilityandpassRateWhereMeasuredare reported apart. A bucket where nothing could be measured has a pass rate of nothing-over-nothing, which prints as a blank or a 100% depending on who wrote the formatter.- A report is never
completewhile any slice is missing or has no powered bucket. A report of clean slices, none of them powered, is a report of nothing having been measured.
Per-item status this evidence supports#
| Item | Was | Is |
|---|---|---|
182.C.19.09 |
not-started |
blocked-corpus: slice machinery built and the availability gap measured on two subjects |