# CPU occlusion and body-visibility probe — 2026-09-10

Evidence for the two metrics `182.C.06.06` was still missing after the earlier
probe in `../2026-09-10-cpu-reference-quality/`: **occlusion** and **full-body
visibility**. Both are now measured on this machine's CPU. This folder records
the numbers, the control, and the one approach that was tried and failed.

## Machine and models

Linux dev server, 15 GiB RAM, 4 cores, no GPU. `mediapipe` 1.0.1 and
`opencv-python-headless` 5.0.0 in a venv at `~/.cache/oshun-human-video/venv` —
not the shared system interpreter, and Python packages do not touch the pnpm
lockfile.

| Model                               | Licence    | Size    | sha256      |
| ----------------------------------- | ---------- | ------- | ----------- |
| `pose_landmarker_lite.task`         | Apache-2.0 | 5.5 MB  | `59929e1d…` |
| `selfie_multiclass_256x256.tflite`  | Apache-2.0 | 16.4 MB | `c6748b12…` |
| `face_detection_yunet_2023mar.onnx` | Apache-2.0 | 227 KB  | `8f2383e4…` |

Both new models are Apache-2.0 and therefore commercially usable, the same test
`182.C.02.06` applied to the insightface weights it rejected.

## The approach that did not work

**MediaPipe Pose's own per-landmark `visibility` score does not respond to
occlusion.** It is the obvious candidate — the field is literally named
visibility — so it was tried first and measured before being discarded:

| Condition                          | mouth landmarks | wrist landmark |
| ---------------------------------- | --------------- | -------------- |
| baseline                           | 0.998 / 0.997   | 0.754          |
| opaque block drawn over the mouth  | 0.999 / 0.999   | —              |
| opaque block drawn over that wrist | —               | 0.758          |

Covering a landmark with a black rectangle moved its visibility score by +0.001
and +0.004. Whatever that field encodes, it is not "is this landmark covered",
and a metric named `occlusion` must not be backed by it. This is why
`tools/human-video/body-geometry.py` reports `presence` and in-frame
coordinates, and does not report `visibility` at all.

## Occlusion — segmentation, with a control

`tools/human-video/face-occlusion.py`: MediaPipe's multiclass selfie segmenter
labels every pixel background / hair / body-skin / face-skin / clothes /
accessory; within the YuNet face box, the class shares are the profile.

Four images, all derived from the committed probe photo
`../2026-09-10-openrouter-face-detection/two-foreground-subjects-plus-bystanders.png`,
each reproducible with one ffmpeg command (below). Raw output:
[`face-occlusion.json`](./face-occlusion.json).

| Image              | Subject face-skin | Obstruction (clothes+accessory) | Control face-skin |
| ------------------ | ----------------- | ------------------------------- | ----------------- |
| baseline           | **0.835**         | **0.000**                       | 0.736             |
| lower-face blocked | **0.434**         | **0.487**                       | 0.733             |
| eye band blocked   | **0.568**         | **0.266**                       | 0.736             |
| hand over face     | **0.248**         | **0.281**                       | 0.840             |

The control column is the second, untouched face in the same photograph. It
moves by at most 0.007 across every variant, so the changes in the first column
are the occlusion and not the segmenter drifting.

Thresholds follow directly: `minFaceSkinRatio` 0.65 sits between 0.568 and
0.733; `maxObstructionRatio` 0.15 separates 0.000 from 0.266. One probe is a
starting point, not a calibration — `182.C.34` re-fits on a corpus.

**Body-skin is reported but not thresholded.** A clear face carries 0.04–0.06 of
it (the neck), and the hand case moved it only to 0.095, because a hand arrives
with a sleeve that the segmenter reads as clothes. One reading is not enough to
threshold on, so the share is recorded for later fitting and nothing is claimed
from it now.

### Reproducing the four images

```sh
SRC=docs/domains/isis/human-video/evidence/2026-09-10-openrouter-face-detection/two-foreground-subjects-plus-bystanders.png
ffmpeg -i "$SRC" -vf "drawbox=x=815:y=189:w=89:h=69:color=black@1:t=fill"  lower-face-blocked.png
ffmpeg -i "$SRC" -vf "drawbox=x=815:y=155:w=89:h=34:color=black@1:t=fill"  eye-band-blocked.png
ffmpeg -i "$SRC" -i "$SRC" -filter_complex \
  "[1:v]crop=90:90:876:618,scale=86:78[hand];[0:v][hand]overlay=552:165"   hand-over-face.png
ffmpeg -i "$SRC" -vf "crop=352:346:704:77"                                 head-and-shoulders-crop.png
```

The hand fixture is real hand pixels — a 90×90 crop taken from one subject's own
wrist and composited over the other subject's face — rather than a synthetic
skin patch, so the segmenter is classifying a photograph of a hand.

The images themselves are not committed: they are 2.5 MB each and regenerate
byte-for-byte from the committed source with the commands above.

## Full-body visibility — pose landmarks, presence AND in-frame

`tools/human-video/body-geometry.py`. Raw output:
[`body-geometry.json`](./body-geometry.json).

| Image                          | People | Hips            | Knees         | Ankles             |
| ------------------------------ | ------ | --------------- | ------------- | ------------------ |
| baseline (waist-up, subject 1) | 2      | 0.992, in frame | 0.452, y≈1.03 | 0.045, y=1.21, out |
| baseline (subject 2)           |        | 0.997, in frame | 0.797, y≈0.99 | 0.074, y=1.19, out |
| head-and-shoulders crop        | 1      | 0.001, out      | 0.000, out    | 0.000, out         |

Two things this shows. First, the measurement tracks framing: cropping the same
subject to head-and-shoulders drops the hips from 0.997 to 0.001. Second,
**presence alone is not sufficient** — the model reports coordinates for
landmarks it places below the bottom edge (y = 1.21), so a region counts as
shown only when its presence clears the threshold _and_ its coordinates fall
inside [0,1]. The scorer applies both conditions.

The pose landmarker is also the person detector `182.C.06.06` asked for: it
found exactly the two foreground subjects in a photograph containing five
detectable faces, which is the same subject-versus-bystander separation the
face-geometry probe produced.

## What is delivered, and what is still an estimate

All nine metrics the item lists are now real measurements, with one honestly
labelled fallback:

- `fullBodyLikelyInFrame` — head-height arithmetic against the 7.5-head canon —
  is kept for deployments with no pose provider bound. The scored record carries
  `fullBody.source`, which is `measured` or `estimated`, and the estimate never
  presents itself as the measurement.
- The scorer only _warns_ about a body not being in frame when the answer is
  measured. Most identity references are portraits, so warning from the estimate
  would fire on nearly every good reference.

Occlusion of the BODY (a desk in front of a person) is still not measured, and
nothing claims it is: the segmenter's classes cover what is over a face, and the
scorer's occlusion issue is scoped to the face box.
