Eve library

`@oshun/live-media`

Other

Revision 1

On this page

@oshun/live-media is the canonical shared contract for tenant-owned live media. It defines runtime-validated identifiers, request scopes, authorization, and the deterministic lifecycle of a live stream.

The root package deliberately exposes no media transport implementation. It does not open sockets, relay packets, write recordings, or claim production ingest. Server-only execution adapters live behind the contract and must preserve its tenant, authorization, versioning, and event invariants.

Boundaries#

  • @oshun/live-media owns live-stream domain contracts and lifecycle rules.
  • Media edge paths use the canonical live-media.v1/<tenant>/<stream UUID> codec. A stream UUID is unique only inside its tenant, so the MediaMTX provider defaults to this tenant/resource path and filters sibling tenants even when their stream UUIDs are equal. Bare UUID paths require the explicit legacy-stream-id provider mode; production Aphrodite and Veritas compositions use the canonical mode.
  • @oshun/streaming remains the Kafka and data-streaming package; live media must not be added there.
  • libs/neith/stream-* contains media execution kernels such as FFmpeg, WebRTC, SRT, and QUIC. Those kernels are adapter dependencies, not the owner of tenant policy.
  • @oshun/live-media/server exports NeithFfmpegTranscodeProvider, which invokes the validated neith-stream-transcoder executable, constrains each output to <operator root>/<tenant ID>/<stream ID>/<execution ID>.ts, rejects symlink aliases, canonical path escapes, and pre-existing output, atomically reserves the execution path, removes partial media on failure/cancellation, owns process cancellation, and accepts success only after ffprobe measures non-empty H.264/AAC media.
  • The same server entry point exports FfmpegHlsDashPackager, VerifiedMediaObjectPublisher, create-only local and AWS S3/MinIO object stores, and PlaybackGrantService. Packaging stream-copies the measured transcode into HLS MPEG-TS and DASH fragmented MP4, independently probes both manifests, publishes segments before entry manifests, verifies every stored checksum, and removes its private workspace. The create-only store contract also supports bounded, checksum-verified byte writes and reads for downstream publication adapters; S3 reads fence the observed object version and recheck length plus SHA-256. S3 writes use conditional If-None-Match: *; the local adapter provides the same immutable-key contract for development and transport verification.
  • createLiveMediaPipelineRuntime is the server-only assembly boundary for one tenant's durable job-to-transcode-to-package-to-publish worker. Product hosts provide fixed tenant configuration and deployment adapters; they do not rebuild the pipeline services or choose tenant authority per request.
  • The root also exports the transport-neutral tenant playback-grant control so products can fix tenant and service authority without importing server stores or signing implementations. Durable stores, grant signing, and object authorization remain injected server responsibilities.
  • The root contract also defines a bounded, test-only MPEG-TS ingest port. @oshun/live-media/server implements it with an exact tenant/resource, declared length and SHA-256, one-MiB chunks, a 64-MiB ceiling, and immutable content-addressed local input. It drives one isolated tenant queue through createLiveMediaPipelineRuntime and returns only the exact successful durable ready publication. This synchronous rehearsal bridge is not a deployed publisher service or a general production upload endpoint.
  • Aphrodite services may orchestrate ingest, relay, playback, recording, chat, and analytics through adapters that consume this package.

Optional media and client adapters#

The root contract defines a fail-closed registry for immersive rendering, segment encryption, DRM, watermarking, content analysis, recording, publisher and player clients, remote guests, and composition. An operation carries an idempotency key and exact tenant/stream plus publication, viewer-session, playback-grant, publisher-grant, or publisher-connection bindings as applicable. An injected authority verifier must prove exactly those bindings before the provider readiness probe or execution runs.

LiveMediaTenantContentAnalysisControl is the canonical product-runtime entry for advisory analysis. A composition fixes one validated tenant and one content-analysis adapter ID; callers supply only a validated service principal and stream-local operation input. The control creates the operate scope and exact tenant/resource/adapter/kind request internally for probe, analysis, and teardown, so a product cannot introduce a parallel authority while retaining its own provider, binding, storage, and policy composition.

LiveMediaTenantRecordingControl and LiveMediaTenantRecordingWorker provide the corresponding durable recording boundary. A composition fixes one tenant and recording adapter before accepting work; the control owns strict input, binding verification, readiness, the semantic request fingerprint, queue reservation, and tenant-local reads. The worker reparses every durable request and refuses a tenant or adapter mismatch before provider execution. Queue, record validation, source authority, retention policy, object storage, and capture execution remain injected ports, so product runtimes can compose real dependencies without recovering caller-selectable authority.

Readiness is explicit and dependency-complete; not-configured and unavailable cannot execute. Successful results must preserve operation and resource identity and supply kind-specific conformance proofs, measured artifacts, and bounded teardown state. Encryption and watermark requests carry only key references, DRM carries only a challenge checksum, and client provisioning carries grant identities rather than raw credentials.

These contracts do not make the existing Aphrodite simulation packages into providers. A concrete adapter is production-capable only after its real runtime, key custody/vendor dependency, authoritative binding verifier, durable idempotency, media-byte evidence, provider-failure behavior, and teardown pass the applicable integration and end-to-end gates.

All shared browser and native client contracts are tenant-neutral. Canonical browser entry points use joinLiveMediaBrowserPlayback, provisionLiveMediaBrowserPublisher, mountLiveMediaBrowserPlayer, mountLiveMediaBrowserImmersive, provisionLiveMediaBrowserRemoteGuestHost, exchangeLiveMediaBrowserRemoteGuestInvite, and provisionLiveMediaBrowserComposition plus their LiveMediaBrowser*Surface types. Operation descriptors accept only a kind-correct tenant-qualified adapter identity: {tenant}.player-client.vN, {tenant}.publisher-client.vN, {tenant}.immersive.vN, {tenant}.remote-guest.vN, or {tenant}.composition.vN. Each client binds the adapter returned at registration or provisioning and rejects an adapter change during evidence, heartbeat, or release. The prior Aphrodite* names remain deprecated exact aliases so tenant #1 consumers retain the same implementation and lifecycle behavior; they are not a second client stack. Shared surface roots use live-media-* classes.

@oshun/live-media/browser/player is the narrow browser entry for protected HLS/DASH playback and its accessible surface; the broader @oshun/live-media/browser barrel retains the complete browser client API. The player uses hls.js or dash.js with Media Source Extensions, opens only URLs inside the exact origin and publication prefix named by the grant, and attaches the memory-only Bearer credential only after that check. DASH requests also omit ambient cookies, and its response interceptor turns 401/403 into explicit authorization loss before the runtime reduces it to a generic download error. The implementation deliberately has no native-media fallback because the browser cannot reliably constrain native media requests to that credential boundary. Playback becomes ready only after the video element reports real dimensions and a positive decoded-frame count and the same-origin, no-store heartbeat returns the matching durable player operation. Later heartbeats preserve that first proof; failure destroys the selected media runtime, clears the credential, and releases the viewer session.

@oshun/live-media/browser/contracts exposes only browser-safe identity, resource, edge-path, and publication-path schemas/helpers for product previews that must not evaluate server controls or Node-only modules.

The browser entry point also exports a restrained, accessible full-stage player surface with explicit loading, authorization-loss, blocked-publication, paused, and released states. The Playwright gate generates real H.264/AAC HLS and DASH with FFmpeg and covers both protocols' decoded playback, repeated heartbeats, exact-prefix Bearer containment, malicious cross-publication references, authorization loss, release, axe checks, mobile layout, and reduced motion. This entry point claims only the browser runtime; the separately bounded Linux-native runtime is described below.

The browser entry point additionally exports an immersive renderer and media-dominant accessible surface over that same protected player. Its WebGL shader supports equirectangular 180° or 360° video with mono, side-by-side, or top/bottom source sampling; cardboard mode renders two eye viewports, while flat mode is restricted to mono. Pointer, keyboard, and permission-backed device orientation update the view. A Web Audio graph measures decoded mono or stereo samples and applies HRTF or stereo panning without uploading raw pose values. Success requires positive decoded and rendered frames, non-black readback, distinct initial and oriented SHA-256 samples, positive bounded audio evidence, and bounded orientation counts. The control API receives only that summary and never the media Bearer, URLs, raw pose, gaze, location, or headset identity.

aphrodite.immersive.v1 binds this evidence to the exact succeeded browser player, publication, active viewer lease, and client instance. Its forced-RLS PostgreSQL operation uses database-clock lifecycle/worker fences, stable replay, expired-lease recovery, and explicit teardown. The Chromium gate generates real 1920×960 H.264/AAC stereo HLS and DASH, verifies both eye buffers on a paused source before and after sensor/pointer orientation, runs axe, checks mobile and reduced-motion behavior, and proves ordered immersive/player release. The real PostgreSQL gate covers migration, replay after reconstruction, stale-worker fencing, tenant denial, credential/tracking non-disclosure, revocation-driven release, and teardown.

This is deliberately not a headset-runtime claim. Cubemap/EAC, ambisonics, and native openxr/visionos clients are rejected. webxr is also rejected until a real immersive XR session and XR-frame render loop can produce nonzero XR frames; support detection or an inline canvas cannot be recorded as WebXR success.

The same browser entry point supplies a credential-contained WHIP publisher. It provisions one short-lived grant from the same-origin application API, restricts that Bearer to the exact secure live-media.v1/<tenant>/<stream UUID>/whip endpoint (with loopback HTTP admitted only for local testing), negotiates a send-only WebRTC session, and reports only monotonic outbound byte, packet, and encoded-frame counters. Publishing is ready only after the application heartbeat returns the exact durable operation with both those client counters and independently observed positive edge ingress for the authenticated MediaMTX connection. Counter regression, authorization loss, malformed session locations, and missing edge proof fail closed.

Teardown deletes the WHIP resource before revoking and durably releasing the grant operation, and it attempts every cleanup step even when an earlier step fails. The Chromium Playwright gate publishes real encoded video through MediaMTX, proves edge corroboration and Bearer containment, and verifies both normal and failed-start cleanup. This entry point claims browser WHIP only; the native SRT runtime is separately bounded below, and RTMP remains unimplemented.

@oshun/live-media/browser/remote-guest supplies the bounded two-person browser remote-guest runtime and surface. One authenticated host exchanges one-use invitation and signaling capabilities with one invited guest; neither credential is placed in a URL. The signaling client accepts only the exact same-operation WebSocket path, sends the one-use ticket in Sec-WebSocket-Protocol, validates every bounded message, enforces monotonic sequences and participant roles, and keeps SDP/ICE transient. The browser peer connection captures one local audio/video stream and proves reciprocal media with monotonic WebRTC statistics plus a SHA-256 link over the guest outbound and host inbound video SSRC. Success requires both sides' independently submitted evidence, not merely signaling or an open peer connection.

The companion full-bleed studio surface keeps the two video feeds dominant and exposes only identity/connection state, invite copy, microphone, camera, and leave controls. It preserves the initiating media/signaling error when cleanup also fails, labels every control, supports keyboard use, fits a mobile viewport, passes axe automation, and removes transition/animation under reduced motion. Transient HTMLMediaElement.play() interruption is retried only after both required remote tracks exist; a genuine playback denial remains fail-closed.

This is deliberately a direct one-host/one-guest WebRTC claim. It does not claim SFU or MCU relaying, TURN availability, multi-party rooms, screen share, end-to-end media encryption, a native/mobile client, or multi-node signaling. Those capabilities require separate contracts, deployments, and conformance evidence rather than inference from the browser peer path.

The browser entry point also supplies a bounded live compositor and accessible full-stage surface. It accepts one to four caller-owned live MediaStream sources, renders deterministic single, grid, or two-source picture-in-picture layouts through Canvas 2D, mixes optional source audio through Web Audio, and captures one 16:9 output stream at a declared 640–1920 pixel width and 15–60 frames per second. The derived stream is published through the existing credential-contained WHIP client. The application control plane receives only source references, layout/output declarations, changed/non-black frame counts, bounded source-region hashes, and aggregate audio evidence—never raw pixels, raw audio, source media, a Bearer, SDP, or ICE.

aphrodite.composition.v1 succeeds only when that render evidence is joined to the exact succeeded browser/WebRTC publisher operation and an independently connected edge observation with positive ingress. Its forced-RLS PostgreSQL operation uses database-clock lifecycle, semantic replay, expired-lease recovery, worker fencing, immutable bounded evidence, and repeat-safe release. The Chromium gate composes two changing Canvas sources plus a real oscillator, publishes the result through MediaMTX, verifies distinct non-black source regions and positive audio/RTP evidence, runs axe, checks keyboard, mobile, and reduced-motion behavior, and proves denial and ownership-correct teardown. The real PostgreSQL/MediaMTX gate covers migration, exact publisher/edge joins, restart replay, abandoned-lease recovery, stale-worker rejection, tenant isolation, privacy, release, and edge cleanup.

This is deliberately a bounded browser-composition claim. Custom layouts, scene switching and transitions, GPU/OBS/native composition, SFU/MCU mixing, and multi-node orchestration remain unimplemented rather than inferred from the Canvas path. Releasing the compositor destroys its derived output and publisher state but does not stop caller-owned source tracks.

@oshun/live-media/native supplies Linux-native execution components for authenticated HLS/DASH playback and SRT publishing. It is intended for a native application shell or supervised Node process; it is not, by itself, a complete desktop or mobile UI. The player holds the Bearer in a memory-only, exact-publication-prefix loopback proxy. FFmpeg and ffplay receive only a one-time local capability in their argument lists. Manifest redirects and every fetch-bearing HLS or DASH reference remain inside that publication boundary, and the credential buffer is cleared during teardown. The runtime proves real FFmpeg decode, supports ffplay rendering, and reports bounded measured QoE to the durable player heartbeat.

The native publisher wrapper passes its operation over standard input to an isolated Rust executable. Its FFmpeg child never receives the publisher credential or SRT passphrase: encoded MPEG-TS enters the Rust process over a pipe and is sent by an in-process pure-Rust SRT 1.4.4 implementation. Remote SRT publishing requires an AES-256 passphrase, while unencrypted publishing is limited to loopback integration tests. Readiness still requires both measured native send statistics and MediaMTX ingress observed for the exact durable publisher operation. The current native production claim is SRT; RTMP remains an extension-contract option without an implementation. Native provisioning passes the canonical tenant-scoped media path separately from the compatibility resource UUID; the Rust boundary rejects bare UUID paths and tenant identifiers that cannot be represented unambiguously in MediaMTX's colon-delimited SRT stream identifier.

Readiness and binding timestamps are compared with the service's current clock, not the original operation timestamp. This permits an exact durable replay to retain its caller-stable request while preventing stale dependency or authority evidence from being reused as fresh readiness.

Security model#

Every lifecycle command includes both a request scope and a tenant-owned stream resource. Runtime parsing rejects inconsistent scopes, and authorization denies a resource whose tenant differs from the authenticated principal even when that principal holds the requested capability. There is no global or system bypass capability.

Lifecycle updates use an explicit expected version and monotonically nondecreasing timestamps. A storage adapter must commit the new state and event atomically with compare-and-swap semantics and enforce command and event identifier uniqueness.

Rendition publications reserve an immutable live-media/<tenant>/<stream>/<publication> prefix before packaging. A ready record carries the exact verified object ledger; failed and revoked state is mutually exclusive with active playback. Playback grants are HMAC-SHA256 signed, expire within bounded lifetimes, carry the entitlement-decision and viewer session IDs, and authorize only objects currently listed beneath that exact publication prefix. Every origin read rechecks ready state and the active viewer lease, so publication revocation, an explicit leave, or lease expiry invalidates an outstanding token immediately. Signing keys and entitlement decisions remain operator inputs; this package does not invent viewer authorization.

SqlLiveMediaPublicationStore is the canonical server-only PostgreSQL publication state machine. It fixes one validated tenant, establishes transaction-local RLS context, serializes each publication with an advisory lock, and preserves idempotent reservation and terminal-transition replay. Prefix and tenant mismatches are rejected before a transaction. Product adapters may translate those errors through thin wrappers, but do not own a second publication state machine.

Viewer sessions are entitlement-bounded, idempotent leases rather than direct counter mutations. Join, heartbeat, reconnect, expiry, and leave are represented by a durable lifecycle ledger; stores persist only a SHA-256 digest of each rotating lease token and must serialize updates per tenant/stream/viewer/client resource. Presence is derived atomically from active leases so multi-node active session and unique-viewer counts share one authority. Heartbeats accept only the strict bounded QoE contract, whose event identity and sample remain immutable.

SqlLiveMediaViewerSessionStore is the canonical server-only PostgreSQL implementation. It fixes one validated tenant, uses transaction-local RLS and a resource advisory lock, takes lifecycle time from PostgreSQL, fences every mutation with the lease digest, and commits session events, monotonic QoE, and derived presence atomically. A deployment may provide an optional transactional presence-projection port for product-owned counters; the canonical store never assumes a product stream table. Aphrodite retains that legacy projection only in its thin tenant wrapper.

Durable live chat#

Live chat is a tenant/resource authority, not a product room or process-local socket concern. Membership is bound to an active viewer session and its exact entitlement decisions; the durable store persists only a SHA-256 digest of the rotating membership lease. Message creation combines idempotency, bounded rate limits, the caller-supplied moderation decision, message persistence, and an ordered event in one transaction. History, presence, deletion, room clearing, participant revocation, lease expiry, and reconnects use the same database clock and authority.

SqlLiveMediaChatStore is the canonical server-only PostgreSQL implementation. It fixes one validated tenant, establishes transaction-local RLS context, and serializes resource and ordered-event mutations with advisory locks. Product adapters may retain policy/filter configuration and map errors through thin wrappers, but do not own a second membership, message, rate-limit, presence, or event state machine. Aphrodite's existing moderation policy remains an input to this shared persistence boundary.

LiveMediaTenantLiveChatControl is the matching product-runtime boundary. It fixes one validated tenant and its system, payment, presence, and fan-out principals, creates participant/moderator scopes internally from a narrow product identity policy, and binds join, lease lifecycle, messaging, history, moderation, presence, server events, and event polling to that tenant. Its optional operational projection preserves stable body-free chat telemetry; product adapters retain only identity, entitlement, moderation/filter policy, configuration, and error translation.

Durable operational telemetry and hooks#

Operational telemetry is a tenant/resource-scoped event ledger and durable hook outbox, not a product-local analytics buffer. One append transaction serializes the semantic idempotency key, records the event with a monotonic database sequence and database time, and creates the selected configured hook deliveries. Duplicate replays return the original event without duplicating deliveries; conflicting payloads fail deterministically.

SqlLiveMediaOperationalTelemetryStore is the canonical server-only PostgreSQL implementation. It fixes one validated tenant, establishes transaction-local RLS context, and claims due deliveries with the database clock, SKIP LOCKED, and expiring lease tokens. Completion, retry, and dead-letter transitions are lease-fenced, including final-attempt expiry. Product adapters retain hook registrations, provider mappings, and legacy product analytics where needed; they do not own another SQL event-ledger or delivery state machine.

Tenant-fixed runtime controls#

LiveMediaTenantViewerSessionControl and LiveMediaTenantOperationalTelemetryControl are the canonical product-runtime facades for leased viewer authority and operational writes. Each validates and fixes one tenant and principal at construction, mints request scopes internally, and accepts only stream-local command data; callers cannot inject or replace a tenant or capability. Viewer lifecycle/QoE telemetry may be wired through the same tenant-fixed writer without introducing a product-owned session service.

LiveMediaTenantPlaybackGrantControl provides the matching server-only grant facade. It fixes the tenant and view principal, validates exact stream and publication identities, and delegates only the canonical entitlement/session request to PlaybackGrantService. Product classes may preserve constructors, default stores, hook registrations, and provider configuration through thin wrappers, but do not mint a second authority path.

LiveMediaTenantMediaPublicationControl is the matching publication command/query boundary. It fixes and validates one tenant plus its worker, reader, and manager principals at construction, generates publication and request identities internally, and binds publish, get, and revoke to the exact tenant-qualified stream. Callers provide only rendition output; they cannot replace the resource, timestamp, tenant, or capability.

LiveMediaTenantPublisherGrantControl and LiveMediaTenantTranscodeExecutionControl provide the server-only publisher and worker entry points. Publisher composition fixes the tenant, owning product, resource mapping, and grant service once; issuance registers the immutable mapping before minting a secret, while rotation, revocation, and authentication remain bound to that tenant. Transcode composition fixes the tenant, worker principal, operations port, and absolute operator-owned output root, then mints the operate scope and tenant-namespaced output URI internally. Neither control accepts a runtime tenant or caller-authored capability.

Durable media-pipeline jobs#

The root contract also owns idempotent transcode/publication jobs, immutable attempt identities, fenced expiring leases, bounded exponential retry state, cancellation, and terminal publication linkage. MediaPipelineWorker recovers expired leases before claiming due work, renews an active lease while the real executor runs, aborts on cancellation or lease loss, and settles only the publication ID reserved for that attempt. Every retry receives fresh execution, publication, attempt, and lease IDs; a stale worker token cannot settle newer work.

LiveMediaTenantPipelineJobControl is the product-composition boundary for those jobs. It fixes one validated tenant and its operate/manage principals at construction, then exposes enqueue, get, list, and cancel without accepting caller-authored tenant or authority fields. Multiple tenant controls may share one canonical durable job service and use the same stream UUID and idempotency key without observing or replaying one another's jobs. Product-specific classes remain thin configurations over this control rather than copies of pipeline behavior.

SqlLiveMediaPipelineJobStore is the server-only durable implementation. It accepts the canonical database transaction port and one immutable tenant at construction, establishes transaction-local RLS context, and uses advisory idempotency locks, SKIP LOCKED claims, database-clock leases, lease-token fencing, and bounded expired-lease recovery. Product adapters may supply their own configuration and tenant-mismatch error mapping through thin wrappers, but they do not duplicate the persistence state machine.

TranscodePublicationAttemptExecutor is the server-only composition from a leased job through the existing Neith transcode service and publication service. It abandons partial transcode/package state on recovery, failure, cancellation, or lease loss. MemoryMediaPipelineJobStore is exported only from @oshun/live-media/testing; deployables must use durable, tenant-isolated storage.

Publisher grants#

Server-only publisher grant orchestration and durable adapters are exported from @oshun/live-media/server. The injected tenant-fixed control contract is also available from the canonical root so product packages can declare their composition without importing server/private implementation entry points. A 256-bit sk_live_ secret is disclosed only when it is issued or rotated; stores receive only its SHA-256 digest. Grant-store adapters must serialize rotation per tenant/resource, revoke the previous generation atomically, and make explicit revocation immediately visible to authentication reads.

Deployable products enter that authority through LiveMediaTenantPublisherGrantControl, not through raw caller-authored scopes. The control composes registry registration with grant issuance and keeps authenticate/rotate/revoke resource identity fixed to the configured tenant. Product-specific classes may retain default databases and constructor seams as thin wrappers over the canonical control.

SqlLiveMediaResourceRegistryStore and SqlLiveMediaPublisherGrantStore are the canonical server-only PostgreSQL authority for substrate identity and publisher credentials. Each accepts a database transaction port and fixes one validated tenant at construction. The registry serializes immutable mapping registration and uses database time; the grant store serializes per-resource rotation and generation. Both establish transaction-local RLS context and reject cross-tenant mutations before opening a transaction. Product adapters may retain only thin default-database, tenant, and error-translation wrappers.

SqlLiveMediaPublisherEdgeSessionStore is the matching server-only durable edge authority. It serializes each tenant/resource, refreshes only an exact active edge/connection identity without increasing its revision, replaces a different connection atomically, ignores stale disconnects, and lists connected sessions inside the configured tenant's RLS context. Product adapters may provide only their database, tenant default, and error translation.

Deployable edge callbacks and reconciliation workers enter that store through LiveMediaTenantPublisherEdgeControl. The control fixes one validated tenant and edge principal at construction, builds every operate scope internally, and binds connect, disconnect, and provider reconciliation to that tenant. Optional operational telemetry receives stable publisher/live and disconnected/ended identities from the durable edge revision; product adapters may supply defaults only through thin wrappers.

@oshun/live-media/testing contains the process-local grant, publication, publisher-edge, media-pipeline job, and viewer-session stores used by contract tests. They are intentionally absent from the production root and server exports. Deployable services must supply durable adapters.