The
apps/oya/area: fifteen Fastify microservices that make up the control plane of Oya, a multi-embodiment household-robotics "hive" (drones, floor-care units, companion/manipulation robots) — each a deterministic domain core fronted by HTTP, with honest fail-loud seams where the real neural models and radio transport plug in.
What this area is#
Oya is a home-robotics platform: a "hive" of heterogeneous embodiments — flying
camera drones, vacuum/mop floor-care units, companion and manipulation robots —
sharing one world model and one safety envelope across a home. The apps/oya/
directory holds the services of that platform. Every one is an Nx
application tagged scope:oya, type:app (see each project.json), and every
one is a Fastify HTTP service built the same way: a main.ts that calls
listen(), an app.ts that calls createOyaServer(...) from
@oya/fastify-core and registers a src/routes/ tree, and a set of pure,
clock-free domain modules under src/ that hold the actual logic. The naming is
uniform: the Nx project and npm package names are all @oya/svc-*.
The defining design rule across the area is deterministic core + injected
seam. The hard logic — auctions, finite-state machines, sensor fusion,
geometry, safety gates — is written as pure functions and classes that take a
caller-supplied timestamp (atEpochMs / atTick) instead of reading a clock,
so the same code runs identically under fastify.inject() in tests and in a
live control loop. Where a capability genuinely needs a large neural model (a
VLA grasp policy, a YOLO/RT-DETR detector, a VLM, a Moshi-style duplex speech
model, a RAG conversation LLM) or a hardware byte-pipe (a drone radio), the
service does not fabricate a result: it defines a typed interface and ships
an Unconfigured* default that throws a 503 not_configured. These fail-loud
seams (svc-assistant/src/llm.ts, svc-manipulation/src/vla-policy.ts,
svc-perception/src/model.ts, svc-hri/src/speech.ts,
svc-eldercare/src/conversation.ts) are the honest representation of a
real-but-absent integration, not stubs.
The services lean on a shared foundation under libs/oya/: @oya/common
(physical constants like EARTH_RADIUS_METERS / JOULES_PER_WATT_HOUR, geo
helpers, ids), @oya/fastify-core (createOyaServer plus the typed error
handler and HTTP status map that turns a thrown { statusCode, code } into a
scrubbed envelope), @oya/database (a QueryExecutor seam + repositories over
pg, used only by svc-mission), @oya/sdk (a typed client), and the wire
contracts at @oshun/contracts/oya (primitives, safety, fleet, dock,
mission, sensor, privacy, capability). Several TS cores are explicitly
deterministic ports of the Rust kernels in libs/oya/engine/crates
(oya-fleet, oya-perception, oya-cinematography) and are written to match
the Rust arithmetic field-for-field so an on-robot kernel and a cloud service
compute the same answer.
How it fits the wider system#
These are the server-side brains and routers of the hive; the embodiments
themselves run the on-robot kernels. The split is deliberate and load-bearing:
the assistant (svc-assistant) is a planner that is categorically
forbidden from emitting a motor command — it decomposes goals and delegates, and
its SafetyGate enforces that mechanically. The fleet orchestrator
(svc-fleet-orchestrator) is a router that allocates tasks to agents and
reserves shared choke-points but never steers a robot. The flight gateway
(svc-flight-gateway) is the only thing that signs and emits drone commands,
and only after arming/command interlocks pass. Perception and home-map maintain
the shared world model that the planners read; energy/dock/mission keep the
fleet powered and on-task; and the application services (floor-care, eldercare,
pet, manipulation, ambient, director, HRI) deliver the actual household
behaviors. Boundaries are enforced by the contracts and by the fail-closed
gates, so a payload validates identically on both sides and an unsafe or
unconfigured request is refused loudly rather than acted on. Walk the dependency
edges from any node below into libs/oya/* and libs/contracts/src/oya to see
exactly what it composes with.
Entity catalog (15)#
The 15 tracked Nx projects in oya, each a code-linked entity node — package, type, source path, declared targets, and its internal dependency graph (depends-on / used-by, resolved from the package manifests, §6/§8), read from the project graph. Grouped by architectural layer; walk the dependency links to travel the system. 15 of these carry an authored deep-dive (what / why / how it fits); the rest are generated scaffolds awaiting one.
unclassified (15)#
Oya ambient service — Matter controller / Thread border-router: exposes hive nodes as Matter Occupancy/Camera, controls 3rd-party lights/locks/shades, and runs the sensing-escalation policy that decides when to dispatch a mobile eye
The Matter controller / Thread border-router service (apps/oya/svc-ambient).
Its real deliverable is src/sensing-escalation.ts — a pure, total decision
function decideEscalation() that arbitrates whether to dispatch an expensive
"mobile eye" (a robot) to physically resolve an ambiguous, low-confidence, or
unconfirmed-anomaly reading from the home's cheap always-on fixed sensors, gated
by QueryUrgency and a tunable EscalationConfig, returning an ordered,
auditable reason set. It also carries a device-registry.ts and a
matter-driver.ts for exposing hive nodes as Matter occupancy/camera endpoints
and controlling third-party lights/locks/shades; the escalation policy itself
performs no dispatch — it returns a decision for the fleet allocator to act on.
Oya assistant service (§4.1) — System-2 brain: tool registry, mission decomposition, spatial reasoning, fail-loud LLM seam, and the mandatory fail-closed safety gate (never emits motor commands; always delegates through oya-safety)
The System-2 "brain" service (apps/oya/svc-assistant). It is a planner, never
an actuator: src/safety-gate.ts's SafetyGate is a fail-closed checkpoint
that categorically refuses motor/actuation action classes
(motor_command_forbidden) and validates every delegated action against an
ISO/TS 15066 SafetyEnvelope (e-stop, speed-and-separation, power-and-force
limits, all fail-closed on malformed telemetry). src/mission-decomposition.ts
turns a GoalTemplate into a dependency-ordered task graph via Kahn's algorithm
with deterministic id-tie-breaking and real cycle detection. The free-form
reasoning step is the one genuine model dependency, represented honestly as the
LlmClient seam in src/llm.ts whose UnconfiguredLlmClient throws
503 not_configured rather than fabricating a plan; the tool registry and
spatial reasoning cores run with no model.
Oya director service — DroneDirector agent: live composition scoring, a preference-learning loop, and the Yemaya shot-list → oya-cinematography camera-path bridge
The DroneDirector service (apps/oya/svc-director). Its core
src/composition-feedback.ts scores a live frame against four classical
cinematography rules — rule of thirds (Gaussian falloff to power points),
headroom, lead room (sigmoid-ramped by subject speed), and shot-size adherence —
blends them with fixed RULE_WEIGHTS, and emits a concrete world-space camera
correction (lateral/vertical translation + dolly distance) to raise the score.
The module is a faithful TypeScript re-expression of the oya-cinematography
Rust crate's shots.rs/continuity.rs, so a TS director and the on-board
kernel score a frame identically. It also carries a preference-loop.ts
learning loop and shot-bridge.ts, the Yemaya shot-list → camera-path bridge.
Oya dock service — dock-network controller, perch routines, per-slot charging FSM, and battery-swap orchestration
The dock controller service (apps/oya/svc-dock). src/charging-fsm.ts models
one drone-on-one-slot contact charging as a deterministic finite-state machine
(Idle → Approaching → Aligning → Contact → Charging → Complete → Released,
plus a Fault terminal reachable from any non-terminal state); the
LEGAL_TRANSITIONS table is the single source of truth, illegal transitions
throw a typed 409, SOC must climb monotonically while charging, and reaching the
target SOC auto-promotes Charging → Complete. Alongside it the service carries
dock-network.ts, perch.ts, and battery-swap.ts for the dock-network
controller, perch routines, and battery-swap orchestration. Everything is
clock-free — every advance takes a controller-supplied atEpochMs.
Oya eldercare service — multimodal fall-detection fusion (radar/CSI/skeleton) with a trigger→confirm→escalate FSM, closed-loop medication adherence, proactive engagement, and a fail-loud RAG conversation seam fronted by a fail-closed refusal-to-advise router
The eldercare service (apps/oya/svc-eldercare). Its centerpiece
src/fall-fusion.ts fuses three confounded fall-sensing modalities (mmWave
radar, Wi-Fi CSI, vision skeleton) with a weighted Bayesian log-odds pool
(naive-Bayes combination under a uniform prior — summing logits so independent
corroboration amplifies) and gates the alarm behind a deliberate
Monitoring → Triggered → Confirmed → Escalated FSM with corroboration and
self-clear windows. src/conversation.ts pairs a fail-loud RAG
ConversationPolicy seam with a deterministic refusal-to-advise router that
fails closed on any medical-advice or emergency phrasing (curated lexicons,
emergency patterns taking precedence) and never consults the model for those.
med-adherence.ts and engagement.ts round out the domain.
Oya Fleet Energy Manager - dock registry/broker, SOC/SOH aggregation, return-to-dock decisions (§4.2)
The Fleet Energy Manager (apps/oya/svc-energy, §4.2). src/return-to-dock.ts
computes a return-to-dock trigger from real flight physics rather than a fixed
SOC threshold: energyToReturnFrac = powerW·(distanceM/speedMps)/3600/batteryWh
(using JOULES_PER_WATT_HOUR from @oya/common), firing when soc reaches
that cost plus a safety reserveFrac, and failing loud on out-of-range input.
The service composes a dock registry/broker, SOC/SOH aggregation, predictive
maintenance, mission checkpointing, and an on-robot failsafe across
dock-registry.ts, soc-aggregation.ts, battery-swap-orchestration.ts,
predictive-maintenance.ts, mission-checkpoint.ts, and
on-robot-failsafe.ts, with the route tree under src/routes/.
buildServer11BuildServerOptions11EnergyServer11DockRegistry13DockAlreadyRegisteredError13ChargeRequest13ChargeDecision13ChargeDenialReason13aggregateFleetEnergy21CRITICAL_SOC21BatteryStateInput21FleetEnergySummary21decideReturnToDock28ReturnToDockInput28 +25 moreOya Fleet Orchestrator (§4.2) - Open-RMF-pattern router: pose/battery/capability ingest, CBBA auction allocator, and space-time traffic reservation for shared resources. Routes assignments only - never a motor controller.
The Open-RMF-pattern fleet router (apps/oya/svc-fleet-orchestrator, §4.2). Its
src/auction.ts is a full CBBA market allocator (the SGA score-improvement
variant of Choi/Brunet/How 2009), mirroring oya-fleet::auction: a bundle phase
that greedily grows each agent's route by discounted marginal score under a hard
capability gate and cumulative battery feasibility, then a consensus phase where
the highest bid wins (ties to lower agent id), iterated to a fixed point and
validated against AllocationSchema. src/traffic-schedule.ts is the
space-time reservation book for shared choke-points (doorways, lifts) using
half-open interval overlap with deny/queue conflict policies. It routes
assignments and time windows only — agent-registry.ts and fleet-adapter.ts
ingest pose/ battery/capability — and is never a motor controller.
Oya flight-gateway service — the secure drone link: arming/safety interlocks, fail-closed command gate, and signed MAVLink2-style command frames
The secure drone link (apps/oya/svc-flight-gateway). src/mavlink-frame.ts
builds and signs MAVLink2-style command frames: a FrameBuilder owns a
monotonic wrapping sequence counter and a strictly-increasing 48-bit signature
timestamp, and signs the canonical little-endian frame bytes with a real
node:crypto HMAC-SHA256 (full 256-bit tag, not the 48-bit truncation),
verified in constant time via timingSafeEqual, with strict-hex decoding to
defeat silent-truncation tampering. The byte transport (serial/SiK/LoRa/4G) is
deliberately out of scope — the module's responsibility ends at a fully-formed
signed frame. arming-interlock.ts and command-gate.ts provide the
fail-closed arming/safety interlocks a command must clear before framing.
Oya floor-care service — zone scheduling, coverage-job lifecycle with recharge-and-resume, and dirt-heatmap persistence
The floor-care service (apps/oya/svc-floorcare). src/coverage-job.ts is a
deterministic coverage-job FSM with recharge-and-resume built in:
Queued → Running ⇄ Paused → … → Complete (plus Aborted), where a Running
step covers up to cellsPerStep cells from a cursor and the FSM derives the
next state — auto-pausing and freezing a checkpoint (covered set + cursor) the
instant SOC drops to the recharge threshold with cells remaining, and resuming
from exactly that checkpoint only once the unit has recharged to the resume
target, so no cell is cleaned twice or skipped. It is supported by
zone-scheduler.ts and dirt-heatmap.ts for zone scheduling and dirt-heatmap
persistence; SOC must drain monotonically during a sweep (a rise is a fault).
Oya home-map service — scene-graph layers, structured spatial-query API (where is X / what changed), privacy-zone annotations, and freshness-tracked change detection
The shared-world-model service (apps/oya/svc-home-map). src/spatial-query.ts
is the structured "where is X / what changed" API over a SceneStore, built
around an honest freshness seam: query() returns a WorldModelResult
discriminated union (fresh / stale / unknown) and never fabricates a
confident answer for a decayed instance — a below-threshold instance reports
stale with its lastSeen and honest low confidence. whereIs() ranks label
matches by current decayed confidence, and changesSince() produces a
structured added/moved/removed diff annotated with freshness and
positional uncertainty. scene-store.ts, change-tracker.ts, and privacy.ts
back the store, change tracking, and privacy-zone annotations; the model mirrors
oya_scenegraph::memory::QueryResult.
Oya HRI service — full-duplex speech-to-speech core (Moshi-style seam), HIVE-wide turn-taking / floor arbitration, mic-array speaker localization fusion, per-embodiment affect rendering, and per-resident persona/trust
The human-robot-interaction core (apps/oya/svc-hri).
src/floor-arbitration.ts is the deterministic HIVE-wide turn-taking referee: a
FloorArbiter that enforces a one-holder invariant across all speakers with
priority ordering (emergency > resident > agent), strict barge-in/preemption
that re-queues an interrupted holder ahead of later same-class arrivals, and
FIFO fairness within a class (timestamp then insertion sequence for a total
order). The defining full-duplex speech-to-speech capability is an honest
fail-loud seam in src/speech.ts — UnconfiguredSpeechPolicy.converse()
rejects with a 503 not_configured rather than fabricate audio — while the
surrounding deterministic cores (affect.ts, persona-trust.ts,
speaker-localization.ts) that drive the model run regardless.
Oya manipulation service — VLA policy serving per-embodiment adapters, grasp-confidence gating with fail-loud escalation, and object-handoff coordination
The manipulation service (apps/oya/svc-manipulation). src/vla-policy.ts is
the per-embodiment VLA (vision-language-action) policy-serving seam: a typed
VlaPolicy interface, an UnconfiguredVlaPolicy whose infer() throws
503 not_configured (it ships no fake model and will not fabricate a grasp from
Math.random()), and an EmbodimentPolicyRegistry that resolves an
unregistered embodiment to the fail-loud default and refuses to register a
policy that reports configured === false. The proposed VlaAction (target,
approach, confidence, aperture) is shaped to feed straight into the
grasp-gate.ts confidence gate with fail-loud escalation, and handoff.ts
coordinates object hand-offs.
Oya mission service — mission CRUD, follow-me policy, and energy-aware flight planning
The mission service (apps/oya/svc-mission) — the one service in the area
backed by a real database. src/app.ts wires createOyaServer to a
MissionRepository over an injected @oya/database QueryExecutor, so the
same handlers run against a pg.Pool in production and an in-memory executor
under inject() with no test-only path. src/energy-aware-planning.ts gives
the fail-loud energy-feasibility gate: it sums 3D leg lengths (haversine
horizontal
- altitude-delta slant) over a
FlightPlan, converts cruise + hover time to required watt-hours, andassertMissionFeasible()throws a typed 422 when the plan overdraws the usable pack energy.follow-me.tsadds the follow-me target geometry, surfaced over/v1/follow-me/target.
Oya perception service (§4.1) — vision/VLM "see what you see": deterministic detection post-processing (NMS, thresholding, bbox→world), multi-object tracking, and confirmed-track publication to the shared world model (svc-home-map). The neural detector/VLM is an honest fail-loud seam.
The vision/VLM perception service (apps/oya/svc-perception, §4.1).
src/detection-postprocess.ts is the deterministic front-end that conditions a
neural detector's raw output: per-class greedy NMS with a faithful
bbox_iou port (clamped intersection, area+area−inter union, strict->
suppression), confidence thresholding, a class allow-list, and a real pinhole
back-projection of surviving box centres onto the ground plane — a
field-for-field port of the oya-perception::geometry Rust kernel. The actual
neural inference is a fail-loud seam in src/model.ts: UnconfiguredDetector
and UnconfiguredVlm throw 503 not_configured rather than invent a detection,
label, or confidence. tracking.ts and map-publish.ts add multi-object
tracking and confirmed-track publication to the shared world model.
Oya pet service — per-individual pet identification (weight-based, plus an honest fail-loud face/gait biometric seam), a behavior/affect state machine with a graded anxiety-mitigation ladder, a calorie-budgeted treat dispenser with consumption verification, and a pet-aware navigation policy
The pet service (apps/oya/svc-pet). src/calorie-budget.ts is a two-phase,
daily-rollover calorie ledger that is honest about dispense-vs-eat: a
dispensed treat is recorded Pending (holding a budget reservation but counting
as not yet eaten), moves to Consumed only on a real verification signal, and
is swept to Wasted if no verify arrives within the grace window — with day
buckets keyed to the home's local UTC offset. src/behavior.ts classifies a
per-individual affect state (Calm / Playful / Anxious / Distressed) from fused
behavioral signals via a documented weighted anxiety score, then drives a
monotonic, graded anxiety-mitigation ladder (slow/widen → halt/calming-tone
→ retreat/alert-caretaker). identification.ts (weight-based plus a fail-loud
biometric seam) and nav-policy.ts complete the service.