# @gaia/weather-data

Sovereign weather & climate **data foundation** for Phase 175. Provides the
ingestion clients, canonical sample schema, normalisation statistics, temporal
splitting, and stratified curriculum sampling that feed every downstream model
(GraphCast, GenCast, cyclone, nowcast, climate emulator, downscaling).

## Modules

| Module                       | Phase       | Contents                                                                                                                                                                                                                  |
| ---------------------------- | ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `variables`                  | —           | Canonical 37-level ERA5 variable set + published GraphCast per-variable/per-level loss weights + flat state-vector layout.                                                                                                |
| `schema`                     | 175.1.3.1   | `WeatherSample`, lat/lon & icosahedral grid specs, blake2b provenance hashing, Zarr chunk alignment.                                                                                                                      |
| `statistics`                 | 175.1.3.4   | Numerically-stable (Chan parallel) per-variable/per-level mean/std/min/max, diff (tendency) stats, versioned save/load, exact normalise/denormalise.                                                                      |
| `splits`                     | 175.1.3.3   | Strictly temporal train (≤2018) / val (2019–2020) / test (2021+) split with an embargo band that prevents rollout leakage across boundaries.                                                                              |
| `sampling`                   | 175.1.3.2   | Extreme-event labelling + inverse-frequency stratified sampler + curriculum scheduler that up-weights cyclones/atmospheric-rivers/blocking over training.                                                                 |
| `ingest.reanalysis`          | 175.1.1.1–5 | ERA5, ERA5-Land, IFS HRES (MARS), GFS (NOMADS), MERRA-2 (GES DISC) request builders + MinIO hot/warm/cold tiering.                                                                                                        |
| `ingest.ibtracs`             | 175.1.1.6   | IBTrACS v04 best-track CSV parser → cyclone tracks with Saffir-Simpson classification and R34/R50/R64 wind radii.                                                                                                         |
| `ingest.nexrad`              | 175.1.2.1   | Sovereign NEXRAD Level-II (Archive II) decoder: bzip2 LDM blocks → Message-31 radials → REF/VEL/SW moments via `(raw−offset)/scale`, encoder for round-trip tests + AWS Open Data key builder.                            |
| `ingest.observation`         | 175.1.2.2–6 | Radar Marshall-Palmer Z-R, bilinear grid remap, GOES ABI Planck brightness temperature, IMERG grid, METAR/SYNOP decoders, lightning flash gridding.                                                                       |
| `ingest.observation_clients` | 175.1.2.2–6 | Real provider requests for public EUMETNET ORD OPERA composites; GOES-R, MTG and Himawari imagery; GPM IMERG; METAR/SYNOP/BUFR/ASOS; and GOES GLM/EUCLID lightning, with cadence/band validation and tiered archive keys. |

## Live-fetch seam

Request construction, tiering, retry policy, and all format parsing/calibration
run and are tested **offline**. The live network fetch is gated behind a
credential/dependency seam (`CredentialsMissing` / `DependencyMissing`) — the
optional `ingest` extra pulls `cdsapi`, `xarray`, `dask`, and `kerchunk`. No
call fabricates a response.

## Test

```bash
cd libs/gaia/weather-data && python3 -m pytest tests/ -q
```