Benchmarking domain library for Phase 71 3D generation quality validation.
Purpose#
This package establishes the dedicated home for Isis 3D benchmark
infrastructure, quality metrics, and competitive comparison workflows. The
initial foundation ships with a standalone seed dataset and threshold-gated
runner so the package is immediately executable while later Phase 71 tasks add
larger prompt/image corpora, richer metrics, and external provider comparisons.
It now also includes an automated provider/model execution runner that can drive
the standardized prompt and reference-image suites across existing
@isis/3d-generation providers and collect per-run output reports. Those
reports can also be stored in the package's versioned benchmark history store so
quality trends remain queryable as providers and model revisions change over
time.
Modules#
./infrastructure: benchmark suite orchestration, seed dataset coverage, and the report contract plus standardized prompt/reference-image corpora and the automated provider/model generation matrix runner plus versioned benchmark result storage/history../metrics: blueprint surface for geometric, topology, texture, adherence, and consistency scoring work planned in71.7.2.x../comparison: blueprint surface for external benchmark adapters, rendering, reports, and leaderboard work planned in71.7.3.x.
Verification#
NX_DAEMON=false pnpm nx test @isis/3d-generation-benchmarks
NX_DAEMON=false pnpm nx lint @isis/3d-generation-benchmarks
NX_DAEMON=false pnpm nx build @isis/3d-generation-benchmarks
pnpm tsc -p libs/isis/3d-generation-benchmarks/tsconfig.json --noEmit