Eve library

Chapter 11 · Models, quality and cost · brief · 10 September 2026

Guide

Revision 1

On this page

The fifth chapter of the How Eve works track, rewritten in the Eve edition on 10 September 2026. It follows the plan agreed on 9 September and recorded in EVE-V1-REDESIGN-TODOS.md. The receipt for the delivery is under verification/; this file records what the chapter is for and why it is shaped this way.

The job#

Chapter ten ended with a verified change. This chapter is about the model underneath it: which one served the turn, what admitted it to that job, and what the whole thing cost once retries and human cleanup are counted.

The mistake it is written against is treating a model as generally good. A benchmark average says nothing about whether a model will reach for the right tool, keep an argument inside its schema, refuse when the evidence is absent, or park a mutation for approval.

Running example. The brief-revision family from chapter ten, now as a serving decision: which model may serve it, on what evidence, and at what cost per accepted revision.

Three learning promises (the cover)#

  1. A floor per family, not a benchmark — eligibility is decided per task family and model leg, against a defined floor and its hard invariants.
  2. Quality is measured around the model — routing, arguments, grounding and completion are independent checks, and every model-visible byte is under change control.
  3. Cost is per accepted outcome — retries, escalation, tool spend and human cleanup all belong in the denominator.

Three things to remember (the close)#

  1. Eligibility is per family and leg. A floor, its hard invariants, and evidence that expires.
  2. The checks run outside the model. And a changed verb in a tool description is a changed behaviour.
  3. The denominator is accepted outcomes. Not calls, and not responses.

Bridge to chapter twelve. Running it: what to watch, what an incident needs, and what a healthy dashboard does not prove.

Sections#

Three sections, opened by a section-divider that names all three.

Section Slides
Choosing a model is a family decision what a comparison must fix, the small-model doctrine, where intelligence is spent, the registry, the leg floor
What quality is measured by the checks around the model, the decks, the release record, the prompt ratchet, one changed verb, escalation
When it fails, and what it costs the ladder, the diagnosis, the four symptoms, the worked arithmetic, the cost measures, the denominator

Treatment decisions#

  • The chapter keeps one table, as the plan asked: eve-model-registry. Four legs, their pinned defaults and what each pin means are exactly the kind of thing a reader needs side by side, and nothing else in the chapter is.
  • The stat panel went to the capability floor, not the cost slide. The plan suggested a stat-panel for cost and latency, but the chapter's cost numbers are invented units chosen to show arithmetic — a panel citing a source for them would be a false citation. The registry's nine legs, on the other hand, are real counts read from the source: four bound, two operator-configured, three not admitted. That is where the panel went, and a test re-derives all four numbers from the pinned registry.
  • The worked comparison is a compare panel. Baseline against candidate, five rows, ending on the row that reverses the second — which is the whole point of the slide and is genuinely two-sided.
  • One changed verb is also a compare panel. "Propose a brief revision" against "update a brief", with an identical-code row on both sides, because the lesson is that nothing else changed.