# @isis/3d-generation-benchmarks

Benchmarking domain library for Phase 71 3D generation quality validation.

## Purpose

This package establishes the dedicated home for Isis 3D benchmark
infrastructure, quality metrics, and competitive comparison workflows. The
initial foundation ships with a standalone seed dataset and threshold-gated
runner so the package is immediately executable while later Phase 71 tasks add
larger prompt/image corpora, richer metrics, and external provider comparisons.
It now also includes an automated provider/model execution runner that can drive
the standardized prompt and reference-image suites across existing
`@isis/3d-generation` providers and collect per-run output reports. Those
reports can also be stored in the package's versioned benchmark history store so
quality trends remain queryable as providers and model revisions change over
time.

## Modules

- `./infrastructure`: benchmark suite orchestration, seed dataset coverage, and
  the report contract plus standardized prompt/reference-image corpora and the
  automated provider/model generation matrix runner plus versioned benchmark
  result storage/history.
- `./metrics`: blueprint surface for geometric, topology, texture, adherence,
  and consistency scoring work planned in `71.7.2.x`.
- `./comparison`: blueprint surface for external benchmark adapters, rendering,
  reports, and leaderboard work planned in `71.7.3.x`.

## Verification

```bash
NX_DAEMON=false pnpm nx test @isis/3d-generation-benchmarks
NX_DAEMON=false pnpm nx lint @isis/3d-generation-benchmarks
NX_DAEMON=false pnpm nx build @isis/3d-generation-benchmarks
pnpm tsc -p libs/isis/3d-generation-benchmarks/tsconfig.json --noEmit
```