--- artifact_kind: model-card artifact_name: utility-fitters version: 1.0.0 owner: Nous · preference-inference team last_reviewed: 2026-04-25 next_review: 2026-07-25 concordia_components: ['@nous/preference-inference', '@nous/agreement-search'] related_phase_179_tasks: [179.3.2.2, 179.3.2.5, 179.3.2.6, 179.3.3.1, 179.3.3.2, 179.3.3.3, 179.3.3.4] related_cards: system: ../systems/concordia-workbench.md evaluators: - ../evaluators/preference-stability.md - ../evaluators/fairness-suite.md --- # Model Card — utility-fitters v1.0.0 ## 1. Identity - **Model name:** utility-fitters (bundle: bradley-terry, thurstone-mosteller, plackett-luce, gaussian-process, neural-ranker) - **Provider / developer:** Concordia · preference-inference (this repo) - **Base model and version:** none — these are deterministic numerical estimators, not LLMs. Implementations live under `libs/nous/preference-inference/src/{bradley-terry,thurstone-mosteller,plackett-luce,gaussian-process,neural-ranker}.ts`. - **Purpose:** convert a stream of pairwise / ranking preference observations from the [pairwise-preference-ranker](./pairwise-preference-ranker.md) into a calibrated `UtilityModel` artifact (posterior mean, standard error, credible interval per alternative, plus comparison count) that agreement-search kernels can consume without reaching back into LLM scoring. - **Intended use cases (Concordia use-case classes):** all 14 `ConcordiaUseCaseClass` values. The fitters are LLM-free, deterministic, and side-effect-free; the only routing constraint is that observations feeding the fit must themselves come from a model card permitted for the use-case class. - **Out-of-scope uses:** never used as standalone preference inference without the §179.3.2.6 abstention gate (`abstention-gate.ts`). Abstention, refusal, redline-blocked, and non-tradeable observations are excluded from the fit by the gate so optimizer selection cannot launder a refusal into a ranking. ## 2. Development context - **Training data summary:** none — these are estimators, not learned models. Per-fit inputs are `PairwiseObservation` / `RankingObservation` records with a non-negative weight; the calibration is purely on the observations the fit sees. - **Fine-tuning:** not applicable. - **Prompt family:** not applicable. - **Concordia-specific customization:** - **Bradley-Terry** (`bradley-terry.ts`): MAP under isotropic Gaussian prior `θ ∼ N(0, I/λ)`; Newton-Raphson with backtracking line search; Davidson-style tie handling; observation weights scale per-pair log-likelihood. - **Thurstone-Mosteller** (`thurstone-mosteller.ts`): probit link; Newton on the log-posterior with the same prior and tie treatment. - **Plackett-Luce** (`plackett-luce.ts`): rank-form likelihood for ranking observations of length ≥ 2; same Gaussian prior. - **Gaussian-process preference** (`gaussian-process.ts`): RBF kernel over alternative-feature embeddings supplied by the caller. - **Neural utility ranker** (`neural-ranker.ts`): single-layer MLP with pairwise log-loss; deterministic seed, fixed batch size, no GPU. ## 3. Inputs and outputs - **Input schema:** `PairwiseObservationSchema` and `RankingObservationSchema` from `libs/nous/preference-inference/src/utility-model.ts`. Fit options schema: `FitOptionsSchema` (tol 1e-7, maxIterations 500, l2 1e-3, seed 0xC0_11_AB_01, credibleMass 0.95). - **Output schema:** `UtilityModelFit` mapped to the contract surface via `toContractUtilityModel`. Per alternative: posterior mean, posterior standard error, central credible interval at the configured mass, comparison count. - **Deterministic settings used in production:** all five fitters are deterministic: fixed seed for initialization and tie-breaking, fixed Hessian ordering, and (for line search) a fixed step-size sequence. Reproducibility manifest pins the fit options and observation set hash. ## 4. Evaluation | Metric | Value | Evaluator card | Date | | ------------------------------------------------- | ----------------------------- | ------------------------------------------------------------- | ---------- | | Run-to-run determinism (same input → same output) | 1.0 (asserted by unit tests) | n/a — `bradley-terry.test.ts` etc. | 2026-04-25 | | Newton convergence on log-likelihood Δ | ≤ 1e-7 (default tol) | n/a — `FitOptionsSchema` default | 2026-04-25 | | Iteration cap | 500 (default) | n/a — `FitOptionsSchema` default | 2026-04-25 | | Credible-interval coverage (calibration) | 95% nominal, validated by | [preference-stability](../evaluators/preference-stability.md) | 2026-04-25 | | | `calibration.test.ts` | | | | Abstention-gate exclusion correctness | 100% (asserted by unit tests) | n/a — `abstention-gate.test.ts` | 2026-04-25 | | Hard-constraint vs soft-preference separation | enforced by | [fairness-suite](../evaluators/fairness-suite.md) | 2026-04-25 | | | `hard-constraints.ts` | | | | BATNA-plausibility refusal of unsupported BATNA | enforced by | [fairness-suite](../evaluators/fairness-suite.md) | 2026-04-25 | | | `batna-plausibility.ts` | | | | Median fit latency (BT, 50 alternatives, 1k obs) | < 100 ms (synchronous) | n/a — `bradley-terry.test.ts` performance assertion | 2026-04-25 | ## 5. Privacy posture - **Data visibility at the model:** alternative ids and the pairwise / ranking outcomes the abstention gate has admitted. Snippet content, party-private fields, and reviewer annotations are _not_ visible to the fitter; only the structured outcomes are. - **Per-party isolation:** §179.5.1.1 isolation enforced upstream; the fitter is per-party by construction (one fit per party id). - **Training-data reuse eligibility:** fits are not "training data" — the fit artifact is consumed by optimizers in the same case. If a fit is exported as part of a derived dataset, it follows §179.5.5.5 training-data governance. - **Zero-retention mode:** the fitter is in-process and emits no telemetry beyond the orchestrator audit event; eligible for zero-retention routing. - **Secure-enclave / confidential-compute posture:** §179.5.1.4 enclave-eligible. Today the fit runs in the same process as the orchestrator; enclave isolation is a roadmap item. ## 6. Safety posture - **Hard-boundary coverage:** observations originating from a hard-boundary-violating candidate are refused upstream by the candidate-filter gate. The fitter never sees them. - **Adversarial robustness:** weight scaling for unstable pairs (down-weighted by `1 - stabilityFlagFraction`) reduces the influence of preferences flagged unstable in §179.3.2.4. - **Coercive-choice refusal:** not applicable at this layer. - **Prompt-injection resistance:** not applicable — no LLM call. ## 7. Fairness and bias posture - **Protected-class disparate treatment:** the fitter is identity-agnostic; the upstream `fairness-profiles.ts` ensures profiles do not encode protected-class disparate treatment. - **Power-asymmetry handling:** BATNA enters only via the fairness-metric layer (`fairness-metrics.ts` Nash product, KS distance, egalitarian welfare) and is sanity-checked by `batna-plausibility.ts`. - **Language / locale coverage:** language-agnostic at this layer. ## 8. Known limitations - Bradley-Terry / Thurstone-Mosteller require non-zero L2 prior for identifiability without anchor constraints; the default `l2 = 1e-3` is a small bias toward zero. - GP-preference and neural-ranker need caller-supplied feature embeddings. Misconfigured embeddings (e.g., zero-vector) will produce uninformative posteriors; the unit tests enforce a non-degenerate input. - Tie handling uses Davidson-style half-credit; this is a modeling choice consistent with the literature but may be wrong for cases where ties signal genuine indifference rather than equal preference. ## 9. Human-in-the-loop expectations - **Required reviewers when this model is used:** none triggered by the fitter itself. Reviewer routing is determined by the use-case class and the abstention / redline observations present in the input. - **Escalation triggers specific to this model:** Newton non-convergence after `maxIterations` is reported as a fit-quality incident; the workbench surfaces it and routes the case to the operator queue. - **Mediator-assist vs autonomous use:** the fitter is always part of the mediator-assist pipeline; the routing decision (assist vs autonomous) happens at the case level in §179.7.6.3. ## 10. Incident history - 2026-04-25 (v1.0.0 release review): no incidents on file. ## 11. Regulatory posture - **EU AI Act Article 11 / Annex IV alignment:** the fitter is deterministic and reproducible; the manifest pins seed, options, and observation hash. Article 11 documentation requirements satisfied at this layer. - **JAMS AI Rules expert-inspection readiness:** all numerical settings pinned; expert can re-run the fit byte-identically given the recorded options and observations. - **NIST AI RMF harm coverage:** H3 (interpretability) and H4 (fairness) primarily. H1/H2 inherit from the upstream LLM ranker. ## 12. References - `libs/nous/preference-inference/src/utility-model.ts` — observation / options / fit-result schemas - `libs/nous/preference-inference/src/bradley-terry.ts` - `libs/nous/preference-inference/src/thurstone-mosteller.ts` - `libs/nous/preference-inference/src/plackett-luce.ts` - `libs/nous/preference-inference/src/gaussian-process.ts` - `libs/nous/preference-inference/src/neural-ranker.ts` - `libs/nous/preference-inference/src/calibration.ts` — credible-interval calibration - `libs/nous/preference-inference/src/abstention-gate.ts` — exclusion rules consumed before fit - `docs/research/bibliography.md` §preference-models