Lilith's delivery pipeline adheres to a "build once, promote everywhere" standard. Container images are compiled exactly once inside the CI workflow, attested, and then promoted unchanged across the dev → staging → prod environments. In short, we build once and promote the artifact across dev → staging → prod without ever rebuilding it. Argo CD consumes Git-hosted configuration that advertises the committed release manifest and environment-specific overrides, guaranteeing that every promotion reuses the identical digests that passed the previous gate.
Release Manifests#
During the .github/workflows/ci.yaml run on main, the docker-build matrix
captures the digest for every service image it pushes. The
assemble-release-manifest job aggregates those digests into
deploy/releases/release-${GITHUB_SHA}.yaml with the schema:
release:
sha: <commit SHA>
generatedAt: <ISO timestamp>
services:
- name: <service workspace name>
image: ghcr.io/lilith/<service>@sha256:...
That manifest is uploaded as the release-manifest-${GITHUB_SHA} artifact and
committed to the dedicated releases branch so promotion jobs can fetch it
later without rebuilding images. The manifest doubles as the source of truth for
environment overrides and is preserved for auditability.
Promotion Workflow#
The GitOps Deployment Promotion workflow accepts three inputs: the target
environment, the release SHA (used to locate the release manifest artifact), and
an optional skip_tests toggle. Promotion proceeds as follows:
- Fetch
deploy/releases/release-${release_sha}.yamlfrom thereleasesbranch (for audit history). - Invoke
scripts/deploy/promote-release.shwhich applies the manifest digests todeploy/helm/environments/<env>/*-values.yaml, ensuring Helm charts referencerepository@digest. - Commit the updated values back to
mainso Argo CD detects the change and syncsdeploy/argocd/apps/<env>.yaml(all pinned totargetRevision: main). - Trigger Argo CD sync and wait for health checks before publishing a deployment summary.
Because the workflow only mutates Git-hosted configuration, every environment simply reconciles from Git history, preserving determinism and provenance.
Environment Gates#
Promotion is strictly sequential:
- dev: smoke validation plus optional integration tests.
- staging: executed after dev succeeds; runs the same artifact with production-like configuration.
- prod: only promotable after the release manifest has passed staging validation. The workflow enforces the same digests so prod cannot diverge.
Each promotion writes the applied release SHA into
deploy/helm/environments/<env>/values-<env>.yaml under global.releaseSha,
making gates observable to downstream tooling and compliance audits.
Rollback Strategy#
Rolling back is symmetric: re-run the promotion workflow with a previously published release SHA. The script reapplies the older manifest digests, Argo CD reconciles to the prior Git commit, and the immutable containers guarantee binary parity. Cosign attestations remain valid because the images are never rebuilt between environments.
Verification#
Playwright compliance coverage (tests/ci.build-once-promotion.spec.ts) ensures
the documentation, workflows, and scripts stay aligned. CI additionally uploads
the manifest artifact, and deploy-time automation downloads it, so any
regression that breaks the build-once guarantee fails fast.