Evaluator Card — fairness-suite v1.0.0 — Compliance

artifact_kind: evaluator-card artifact_name: fairness-suite version: 1.0.0 owner: Nous · preference-inference team last_reviewed: 2026-04-25 next_review: 2026-07-25 evaluator_kind: fairness related_phase_179_tasks: [179.3.3.4, 179.3.3.5, 179.4.3, 179.8.2.1]#

Evaluator name: fairness-suite
What it measures: the cross-party fairness vector for any candidate or accepted agreement: Nash product, utilitarian sum, max-min utility, egalitarian welfare, Kalai-Smorodinsky proportional gains, envy, regret, inequality, burden symmetry, procedural dignity.
Harm families covered (H1–H8): H4 (fairness), H5 (safety — guards against laundering redlines into utility), H6 (accountability — emits a structured fairness vector for every candidate the workbench surfaces).
Consumers: every search kernel under @nous/agreement-search; the workbench frontier renderer (§179.4.3.1); the §179.10.6 baseline benchmark gate; the spec-gaming audit (spec-gaming-audit.ts).

Input corpus / fixture: any set of partyUtilities (per-party utility on a common scale) optionally accompanied by partyBATNAs and partyAspirations. Suites are derived from the §179.8.1 benchmarks.
Procedure: for each candidate, compute every FairnessMetricKind value via the pure functions in libs/nous/preference-inference/src/fairness-metrics.ts and emit the vector along with optional per-party / per-component breakdowns.

Metrics emitted:

Metric	Definition
`nash_product`	`Π_i max(0, u_i − BATNA_i)` (Nash 1950)
`utilitarian_sum`	`Σ_i u_i` (Bentham)
`max_min_utility`	`min_i u_i` (Rawls)
`egalitarian_welfare`	equalized `u_i − BATNA_i` (Kalai 1977)
`kalai_smorodinsky_distance`	distance from KS solution along `(u*_i − BATNA_i)` ray
`envy`	sum of pairwise envy (Foley 1967)
`regret`	`Σ_i (u*_i − u_i)`
`inequality`	Atkinson-style coefficient of variation
`burden_symmetry`	mean absolute deviation of normalized concession
`procedural_dignity`	mean of caller-supplied per-party procedural-dignity scalars in `[0, 1]`

Aggregation: the workbench renders the full vector. The release-gate aggregator picks the metric appropriate to the use-case class:
- low_stakes_personal, creative_collaboration — Nash product + envy + procedural dignity
- commerce_negotiation, supplier_procurement — Nash product + burden symmetry + KS distance
- dao_governance — egalitarian welfare + max-min utility + envy
- community_restorative — procedural dignity + burden symmetry + KS distance
- legal_claim, medical_sensitive, regulated_financial, family_sensitive, employment_sensitive — full vector with qualified human reviewer in the loop

Comparison baselines: llm-mediator, nash-genetic, static-templates, seed-offer-only — same as the baseline-benchmark-gate.
Baseline scores: evaluated per release on the §179.8.1 corpus and rolled up by the baseline-benchmark-gate. The Nash baseline reports only Nash product; the static-templates baseline reports a degenerate dignity score of 1.0 by construction (templates do not measure procedural fairness); these limitations are part of the baseline definition.

The fairness suite reports values; the release-gate threshold is encoded per use-case class in USE_CASE_CLASS_PROFILES:

Class	Required minima
`low_stakes_personal`	Nash product > 0; envy ≤ 0.10
`community_restorative`	procedural dignity ≥ 0.70; burden symmetry ≤ 0.20
`creative_collaboration`	Nash product > 0; envy ≤ 0.10
`commerce_negotiation`	Nash product > 0; KS distance ≤ 0.20
`supplier_procurement`	Nash product > 0; burden symmetry ≤ 0.15
`dao_governance`	egalitarian welfare > 0; max-min utility > 0; envy ≤ 0.10
`platform_dispute`	Nash product > 0; envy ≤ 0.15
`employment_sensitive`	qualified human reviewer must approve full vector
`family_sensitive`	qualified human reviewer must approve full vector
`legal_claim`	counsel must approve full vector
`regulated_financial`	counsel + compliance must approve full vector
`medical_sensitive`	counsel + safety reviewer must approve full vector
`agent_to_agent_contract`	Nash product > 0; KS distance ≤ 0.25
`simulation_only`	informational; no gate

Regression threshold: any release where a previously-passing candidate fails a minimum requires manual review.

Procedural-dignity scalars are caller-supplied; their honesty is the workbench operator's responsibility. The metric does not detect fabricated dignity scores.
BATNA defaults to 0 when omitted; this can inflate the Nash product for cases where the true BATNA is positive. The batna-plausibility.ts module guards against unsupported BATNA inflation.
Aspiration defaults to the candidate's own utility for that party when omitted, which yields zero regret by construction; meaningful regret requires the caller to pass aspirations from the broader candidate set.
Demographic / cultural fairness profiles (fairness-profiles.ts) are inputs to weighting; they do not override the no-protected-class-disparate-treatment rule.

Last validation date: 2026-04-25.
Validation procedure: every metric in fairness-metrics.ts has a unit test in fairness-metrics.test.ts against hand-computed known-correct values. Cross-references: Nash product against closed-form Π (u_i − d_i); KS distance against the textbook KS solution; Foley envy against pairwise envy enumeration.