# CPU sync-evaluator and forced-alignment probe — 2026-09-10

Evidence for `182.C.17` (sync evaluators), `182.C.18.02` (word alignment), and
the phase-level hardware constraint. **Third correction of the day, same
cause**: all thirteen `182.C.17` rows, all eight `182.C.20` rows, and all eight
`182.C.32` rows carried one copied blocker, "No GPU host; needs a RunPod worker
per `182.C.17.13`", and `182.C.18.02` was blocked on "a forced aligner on the
hosted GPU worker". None of those items had been tried. Both premises were
tested here and both are false for the evaluator side.

## Machine

Linux, 15 GiB RAM, 4 cores, no GPU (`nvidia-smi` absent; `torch 2.14.0+cpu`,
`faster-whisper 1.2.1`, `transformers 5.17.0`, `onnxruntime 1.25.1` installed
system-wide). `free -m` before the runs: 10.9 GB available.

## 1. SyncNet v2 runs on this CPU

`joonson/syncnet_python` — the LSE-C / LSE-D baseline the phase names in
`182.C.17.01`. **MIT licence** (`LICENSE.md`, copyright Joon Son Chung). Weights
`syncnet_v2.model`, 54,573,114 bytes, sha256
`961e8696f888fce4f3f3a6c3d5b3267cf5b343100b238e79b2659bff2c605442`, fetched from
`https://www.robots.ox.ac.uk/~vgg/software/lipsync/data/syncnet_v2.model`. Loads
with `strict=True`: 13.63 M parameters.

Forward pass on CPU, 4 threads, `torch.no_grad()`, batch of 20 windows:

| Stream                                             | Input shape            | Output       | Time                                  |
| -------------------------------------------------- | ---------------------- | ------------ | ------------------------------------- |
| lip (`forward_lip`)                                | `[20, 3, 5, 224, 224]` | `[20, 1024]` | 0.80 s                                |
| audio (`forward_aud`)                              | `[20, 1, 13, 20]`      | `[20, 1024]` | 0.31 s                                |
| per second of 25 fps video (25 five-frame windows) | —                      | —            | **≈ 1.0 s CPU** (lip) + 0.4 s (audio) |

A 10 s candidate therefore costs about 15 s of CPU to score end to end, before
face tracking. That is a throughput number for `182.C.17.12`, not a blocker. The
hosted GPU worker in `182.C.17.13` is a production fan-out decision to be taken
from measured throughput, not a prerequisite for building the evaluator.

Not tested here: a real talking-face clip through the full pipeline (face crop →
MFCC → SyncNet → offset/confidence). No generated human video exists in the repo
yet, and producing one is provider spend under prohibition 2 (consent binding
before any reference upload). The forward pass on correctly shaped tensors
proves the compute premise only.

## 2. A CTC forced aligner runs on this CPU, and it is accurate

`facebook/wav2vec2-base-960h` through `transformers` (Apache-2.0; the
`masked_spec_embed` weight is reported missing at load, which only affects
training-time masking), with the standard CTC trellis and backtracking. 20.1 ms
frames.

### Ground truth

Timestamps from a chat model are generated, and Whisper's are inferred from
cross-attention; to measure either you need onsets known to the sample. So:
Kokoro-82M (`hexgrad/Kokoro-82M`, cached locally, voice `af_heart`) synthesized
six words **separately**, each trimmed to its first and last sample above
|0.01|, then concatenated with exactly 600 ms of digital silence between them.
Every onset is therefore known exactly (`synth-truth.json`;
`known-onset-clip-16k.wav`, 7.23 s).

### Results (`align-accuracy.json`)

| Aligner                                     | CPU time            | Onset error per word (ms)       | Max abs onset | Word-end error (ms)             |
| ------------------------------------------- | ------------------- | ------------------------------- | ------------- | ------------------------------- |
| **wav2vec2 CTC forced alignment**           | 1.29 s for 7.23 s   | +81, +65, +17, +17, +62, +35    | **81 ms**     | −96 … −141 (consistently early) |
| faster-whisper `tiny.en`, `word_timestamps` | 2.0 s (3.95 s clip) | −600, +102, +532, +151, +113    | 600 ms        | merged "forty dollars" → "$40"  |
| faster-whisper `base.en`                    | 5.7 s               | −600, −58, −128, −49, −87, −116 | 600 ms        | −218 … −352                     |
| faster-whisper `small.en`                   | 15.3 s              | −600, +182, +52, +171, +33      | 600 ms        | merged "forty dollars" → "$40"  |

Against the thresholds in `182.C.03.04` (ATSC IS-191 target +15/−45 ms; ITU-R
BT.1359-1 reject +45/−125 ms):

- **CTC** is inside the ITU reject boundary on every word, raw. Its error is a
  systematic _late_ bias (every onset positive, 17–81 ms), so a calibrated
  constant corrects most of it. That calibration was run — it is the first step
  of `182.C.18.02` — and it **corrects an earlier claim on this page**, which
  said the constant brings CTC inside the ATSC window. It does not.
  `aligner-calibration.json`
  (`tools/human-video/evaluator/calibrate_aligner.py`, reproducing the onsets
  above exactly): median bias **+48.288 ms**, so the correction is **−48.288
  ms**; the residual scatter that remains is **±32.87 ms** (worst lead −31.68,
  worst lag +32.87). ATSC IS-191 allows only 15 ms of _lead_, and the corrected
  worst lead is 31.68 — **outside ATSC**. It is comfortably **inside ITU-R
  BT.1359-1** (+45/−125) with ~30% margin on the tighter side. That ±33 ms is
  the aligner's own noise floor and therefore the floor on any timing claim
  built from it. Word ends are consistently ~100–140 ms early (CTC emits a token
  before the sound finishes), so **use onsets, not ends**, for offset work.
- **Whisper's timestamps are disqualified**, like the OpenRouter chat models' in
  `../2026-09-10-openrouter-asr/`: the first word is snapped to 0.0 (600 ms
  early, every model size), mid-clip onsets are off by up to 532 ms, and two of
  three sizes merged two spoken words into one orthographic token, which also
  breaks word counting. Whisper remains fine for _transcription_; it is not an
  aligner.
- On the real `probe.wav` (24 kHz TTS, `../2026-09-10-openrouter-asr/`), CTC
  aligned the spoken words `forty two dollars` as three words in 0.68 s
  (`fw-words.json` has the Whisper timings for the same clip); a chat model
  returns `$42` and the numbers-to-words expansion in `dialogue.ts` must run
  before alignment.

## Pronunciation, from the same posteriors (`pronunciation-gop.json`)

`182.C.18.03` asks a different question from `182.C.18.01`: not whether the
recognizer wrote the right letters, but whether the ACOUSTICS support them. A
recognizer's language model repairs a mangled brand name into the spelling it
expects, so a clean transcript is not evidence the name was said right.

The measure is Witt & Young goodness of pronunciation over the same CTC model,
`GOP = log P(canonical | emission frame) - max_s log P(s | emission frame)`. At
most 0; 0 means the acoustic model would itself have chosen that character.

| Clip                                               | Correct words              | One word substituted, same audio                                                           |
| -------------------------------------------------- | -------------------------- | ------------------------------------------------------------------------------------------ |
| `known-onset-clip-16k.wav`                         | all six at **exactly 0.0** | tuesday→thursday −15.32, costs→coasts −9.62, forty→fourteen −11.17, engine→enjoying −20.18 |
| `../2026-09-10-openrouter-asr/probe.wav` (24k→16k) | all ten at **exactly 0.0** | —                                                                                          |

In every substitution the other five words stayed at exactly 0.0, so the fault
localises to a word and to a character within it (`thursday` fails at `R`, where
the model preferred `E`).

**The finding, and it constrains how this can be used.** The separation is
total, so any floor in the gap separates these cases and the measurement does
not say which value is right. More importantly the measure SATURATES: two
different voices and two different sentences all sit at exactly 0.0, so there is
no headroom above zero in which to rank a perfect pronunciation against an
accented but acceptable one. It is a detector, not a grade. Both fixtures are
clean synthetic English, so they do not establish a threshold for real human
speech at all — that needs the `182.C.33` rated corpus, the same blocker as
`182.C.17.08`. The shipped floor is −3.0 and is labelled provisional in code,
placed nearer the correct end deliberately, because a false "mispronounced" on a
real accent rejects a take a human would accept.

**Locale is the other half.** `wav2vec2-base-960h` is LibriSpeech English. Given
a Yoruba name it has no symbols for those sounds, and a low GOP then means the
model does not know the language rather than that the speaker said it wrongly.
Those terms are reported UNMEASURABLE with no number attached and routed to
`182.C.18.07`; unmeasurable and failed must never look alike.

**A defect this work found in the aligner committed hours earlier.** `tokenize`
silently dropped any character outside the model vocabulary. Aligning
`"costs 40 dollars"` against audio saying "costs forty dollars" dropped both
digits, put `40` 463 ms late, and lost `dollars` from the output entirely. It
now refuses with `unalignable_character`, and `merge_words` groups by the labels
the tokenizer produced instead of counting transcript characters in parallel —
the two walks agreeing was the assumption that broke. The `182.C.18.02`
calibration onsets are byte-identical after the rewrite.

## Per-item status this evidence supports

| Item                | Was                                        | Is                                                                                                      |
| ------------------- | ------------------------------------------ | ------------------------------------------------------------------------------------------------------- |
| `182.C.17.01`–`.12` | `blocked-local`, "No GPU host"             | `not-started`; SyncNet v2 measured on this CPU; second evaluator still to be selected and licensed      |
| `182.C.17.13`       | `blocked-local`, "Mac cannot host SyncNet" | `not-started`; a throughput/hosting decision taken from `182.C.17.12` numbers                           |
| `182.C.18.02`       | `blocked-local`, aligner needs GPU worker  | done; CTC aligner on this CPU, calibrated −48.288 ms, ±32.87 ms residual (inside ITU, outside ATSC)     |
| `182.C.18.03`       | not covered                                | done; GOP from the same posteriors, saturated on clean speech, threshold provisional pending `182.C.33` |
| `182.C.20.01`–`.08` | `blocked-local`, "No GPU host"             | `not-started`; depends on `182.C.17` and on real generated active-scene clips (spend, consent-bound)    |
| `182.C.32.01`–`.02` | `blocked-local`, "GPU-bound research"      | `not-started`; a benchmark-harness definition and a licence review need no GPU                          |
| `182.C.32.03`–`.08` | `blocked-local`, "GPU-bound research"      | `blocked-local`, now justified: self-hosted diffusion video _generators_ need CUDA; none here           |

## Where the weights are

Staged outside every worktree, so a scratchpad clean-up cannot delete them:
`~/.cache/oshun-human-video/` — `yunet.onnx` (`8f2383e4…`), `sface.onnx`
(`0ba9fbfa…`), `face_landmarker.task` (`64184e22…`), `syncnet_v2.model`
(`961e8696…`), `SyncNetModel.py` (`41b85aed…`), `LICENSE.md`. 93 MB total.
Nothing under that directory is committed; `182.C.17.02` re-fetches by digest.

The evaluator interpreter is `~/.cache/oshun-human-video/venv` — CPU `torch`,
`transformers`, `scipy`, `python_speech_features`. It is the single interpreter
every evaluator script runs under, including the aligner, and the specs that
shell out to those scripts skip visibly when it is absent rather than passing
quietly.

## Reproducing

The alignment and calibration scripts are now committed and rerun the numbers
above from the committed fixture:

```
tools/human-video/evaluator/ctc_align.py        --audio known-onset-clip-16k.wav --transcript "engine ships tuesday costs forty dollars"
tools/human-video/evaluator/calibrate_aligner.py --audio known-onset-clip-16k.wav --truth synth-truth.json --out aligner-calibration.json
```

The rest lived in the session scratchpad and is reproduced from the numbers
above: SyncNet forward pass with random tensors of the shapes listed; Kokoro
per-word synthesis with 600 ms gaps; `faster-whisper` with
`word_timestamps=True`, `beam_size=5`, `compute_type=int8`. No provider spend.
