# VAC: The Communication Pipeline
In V6 — **Egbe**, the agentic-companion universe — you do not pick options from
a wheel; you _talk to_ an autonomous being (an **Ori**), and it talks back.
**Vac** is the substrate that makes that exchange trustworthy in both
directions. On the inbound side it owns the whole path from a player's spoken or
typed words to a _structured agent objective_ the world can act on: microphone
capture, transport through the realtime gateway, speech recognition, and the
parse that turns natural language into a constrained intent grammar — `verb`,
`target`, `constraints`, `priority`, `deadline`, and `forbidden lines`. On the
outbound side it owns the negotiation and conversation routing that carries an
Ori's reply — an acceptance, a question, a counter-offer, a deferral, or a
_refusal_ — back to the player as a spoken, lip-synced dialogue turn, plus the
persistent squad-comms channel a household of agents talks over. The
load-bearing design commitment is that **a misheard word never becomes a
misunderstood life**: every parsed intent is a _draft_ that the steward must
confirm or correct before it can bind, the acknowledgement is held to a hard ≤
400 ms budget, and every Vac capability has a complete non-voice equivalent so
voice is never required. The recognizer, the speech synthesizer, and the
language model that powers free conversation are all **injected, governed seams
— not hard-coded dependencies** — and this page is honest about exactly where
each one sits. It belongs to the **Communication, Story, and Lifecycle** group
and is the deep companion to the "The Vac Communication Pipeline" section of the
hub, [../V6_ARCHITECTURE.md](../V6_ARCHITECTURE.md).
## What ships, honestly
**Implemented and tested (real today).** The Vac intent pipeline is
dual-authored in two places that must agree, exactly mirroring V6's split
between Mind-layer logic and the engine runtime. The TypeScript canon is
`@oshun/vac-intent` (`libs/v6/vac-intent/src/index.ts`, ~1,750 lines, 11 unit
tests): a five-stage voice→intent pipeline, a strict
constrained-function-calling grammar with a full domain validator, a
natural-language objective parser, the steward-confirmation gate, and the
agent-side negotiation router — all pure functions. The C++ mirror is `V6Voice`
(`V6/ue/Source/V6Voice`): `FV6VacVoiceIntentPipelineRuntime` re-implements the
same five stages on the engine side, and `FV6SquadCommsRuntime` owns the squad
channel. Both are covered by real UE automation —
`V6.Vac.VoiceIntentPipeline.MicGatewayAsrParserBudget`,
`V6.Vac.ConversationRouting.PsycheTtsLipSyncBudget`, and the
`V6VacSquadCommsTests` realm-parity scenario (all under
`V6/ue/Source/V6Tests/Private`). The parsed objective is validated against the
shared `ObjectiveSchema` from `@oshun/contracts`, so the same
`v6.ori.objective.1` shape round-trips between TS, the contract package, and the
engine struct.
**The injected seams (honest, not faked).** Three boundaries in this pipeline
are deliberately _not_ implemented here, and the code says so out loud. (1)
**ASR is an injected adapter.** `VacAsrAdapter` (`vac-intent/src/index.ts:266`)
is a one-method interface (`transcribe(packet)`); when none is wired,
`transcribeVacGatewayPacket` (`:736`) returns a _deterministic echo_ of the
captured utterance at a fixed 0.94 confidence — a test/wiring stand-in, **not**
a speech recognizer. The real recognizer is passed in via `options.asrAdapter`,
and the same is true engine-side, where `TranscribeGatewayPacket` takes the
recognizer's confidence as a parameter. (2) **TTS and the conversation LLM are
seams owned by Psyche.** The spoken reply (provider `v6-local-tts`, Opus 48 kHz
mono) and the model that authors free dialogue are the `@oshun/psyche-agent`
gateway boundary documented in
[the cognition stack](./cognition-stack-and-agent-behavior.md); Vac routes to
them, it does not contain them. (3) **The transport is the Egbe Realtime
Gateway** (`apps/v6/egbe-realtime-gateway/`, Rust WebTransport/WebRTC/WS + voice
SFU) — this library owns the _gateway packet shape and route_
(`egbe-vac:voice -intent`), not the deployed SFU.
**Honest scope limits.** The stage-latency numbers Vac percentiles against the
400 ms budget are a deterministic **budget model** (the per-stage millisecond
table in `VAC_DEFAULT_STAGE_LATENCIES_MS`), not a live network measurement — the
same honest pattern V3's voice gateway uses. And the `V6Voice` module object
itself is a thin `IModuleInterface` (`V6Voice.cpp` has empty startup/shutdown);
the logic lives in the runtime classes, not the module shell. Nothing in this
pipeline is `.uasset` content — it is all logic and contract.
## The communication pipeline — voice to a confirmable objective
A single voice command for one Ori runs an ordered, five-stage pipeline. Each
stage is a pure function taking the previous stage's typed frame, so the whole
path is testable end-to-end with the real recognizer wired only at the boundary.
`runVacVoiceIntentPipeline` (`vac-intent/src/index.ts:830`) threads them; the
engine's `FV6VacVoiceIntentPipelineRuntime::RunVoiceIntentPipeline`
(`V6VoiceIntentTypes.cpp:738`) does the same, validating each frame's
`Validate(OutReason)` and failing loud on the first bad stage.
```mermaid
flowchart TD
PLY["player: spoken or typed words"]
PLY --> CAP["captureVacMicrophoneFrame()
push-to-talk · 48kHz mono · end-of-utterance"]
CAP --> GW["routeVacCaptureThroughGateway()
egbe-vac:voice-intent · streamId · traceId"]
GW --> ASR["transcribeVacGatewayPacket()"]
SEAM(["injected VacAsrAdapter
(real ASR engine)"]) -. "transcribe(packet)" .-> ASR
ASR --> PARSE["parseVacTranscriptToObjective()
verb · target · constraints · forbidden lines"]
PARSE -->|"confidence < 0.7 or no verb"| CLAR[/"status: needs-clarification"/]
PARSE -->|"parsed"| BUDGET{"total ≤ 400 ms?"}
BUDGET -->|no| OVER[/"status: budget-exceeded
(never promoted to success)"/]
BUDGET -->|yes| DRAFT["draft VacStructuredObjective
+ confirmationPreview"]
DRAFT --> GATE{{"steward confirms or edits
(V6UI)"}}
GATE -->|confirm| OBJ[/"confirmed objective → agent"/]
OBJ --> NEG["routeVacNegotiation()
accept · clarify · counter · defer · refuse"]
NEG --> REPLY["Psyche TTS + lip-sync
spoken reply on agent pawn"]
```
**1 — Capture.** `captureVacMicrophoneFrame` (`:697`) normalizes a raw mic input
into a `VacMicrophoneCaptureFrame`: it sanitizes the session and device ids,
collapses whitespace in the utterance, and clamps the audio metadata to
supported bounds (1–30 s duration, 8–96 kHz, 1–2 channels), defaulting to
push-to-talk, 48 kHz, mono. The engine struct
`FV6VacMicrophoneCaptureFrame::Validate` (`V6VoiceIntentTypes.cpp:512`) is the
fail-loud twin: it rejects a frame missing session or device identity, an empty
utterance, or a capture that is not `bEndOfUtterance`.
**2 — Gateway routing.** `routeVacCaptureThroughGateway` (`:713`) wraps the
frame in a `VacGatewayPacket` bound to the fixed route `egbe-vac:voice-intent`,
minting a deterministic `streamId` and `traceId` (FNV-1a over session +
utterance) that become the spine of the trace. The engine twin pins the same
route string and rejects any packet not on it — `egbe-vac:voice-intent` is a
hard invariant on both sides, so a stray transport can't smuggle audio into the
parser.
**3 — ASR (the seam).** `transcribeVacGatewayPacket` (`:736`) is where the
recognizer plugs in. With a real `VacAsrAdapter` it returns that engine's
`VacAsrTranscript` (text, confidence, language tag, `isFinal`); with none, it
returns the deterministic echo described above. Either way the _next_ stage
trusts nothing: it re-checks `isFinal` and the confidence floor.
**4 — Parse.** `parseVacTranscriptToObjective` (`:755`) is the heart of the
pipeline (its own section below). It gates on `minimumAsrConfidence` (default
0.7) and a resolvable command verb; failing either returns `null`, which
surfaces as `status: 'needs-clarification'` rather than a guessed objective. The
test `'holds low-confidence ASR and unsupported commands for clarification'`
(`index.spec.ts:196`) pins this: a 0.41-confidence transcript and an off-topic
"maybe the weather is nice" both refuse to produce an objective.
**5 — Budget + status.** `runVacVoiceIntentPipeline` assembles a
`VacLatencyBudget` from `VAC_DEFAULT_STAGE_LATENCIES_MS` (`:24` — capture 24,
gateway 46, asr 152, parser 68, ack 34 = **324 ms**, inside the
`VAC_PARSED_INTENT_ACK_BUDGET_MS` of 400) and resolves a status of `'parsed'`,
`'needs-clarification'`, or `'budget-exceeded'`. The crucial honesty: a parsed
objective whose stages blow the budget is reported as `'budget-exceeded'`,
**not** silently promoted to success — the test
`'reports a budget miss without promoting the parsed intent to a successful acknowledgement'`
(`:243`) forces ASR to 360 ms and asserts the demotion. Engine-side,
`IsSuccessfulWithinBudget` (`V6VoiceIntentTypes.cpp:613`) requires `Parsed`
_and_ a parsed objective _and_ within-budget _and_ an empty failure reason, all
four.
### The steward-confirmation gate — a draft, never a command
A parsed objective is born a _draft_. `parseVacTranscriptToObjective` always
sets `status: 'draft'`, `confirmedBySteward: false`, `acceptedByAgent: false`,
and attaches a `VacIntentConfirmationPreview` whose `state` is
`'awaiting-steward-confirmation'` and whose `canBecomeStandingObjective` is the
literal `false`. `canVacObjectiveBecomeStandingObjective` (`:680`) returns true
_only_ once `confirmVacParsedObjective` (`:684`) has flipped both
`confirmedBySteward` and `status: 'confirmed'`. The test
`'keeps parsed standing-intent output behind steward confirmation'` (`:268`)
proves even an explicit "always protect the grove path" parse stays a draft
until the steward acts. The engine enforces the same wall:
`IsSchemaReadyForDraft` (`V6VoiceIntentTypes.cpp:565`) _rejects_ any objective
where `bConfirmedBySteward` or `bAcceptedByAgent` is already set — a parsed
intent that arrives pre-confirmed is treated as malformed. This is "a misheard
word never becomes a misunderstood life" encoded as a state machine, and it is
rendered for the player in the `V6UI` intent-grammar builder (see
[the UE5 client modules and embodiment](./ue5-client-modules-and-embodiment.md)).
## Intent parsing and grounding
### The constrained intent grammar
Vac never lets a model emit free-form JSON.
`buildVacIntentGrammarFunctionSchema` (`:598`) publishes a single strict
function schema named `vac_parse_objective _intent`
(`VAC_INTENT_GRAMMAR_FUNCTION_NAME`) with `strict: true` and
`additionalProperties: false` at every level — the contract a constrained
function-calling model is held to. Its required fields are exactly the six the
hub names: `verb`, `target`, `constraints`, `priority`, `deadline`,
`forbiddenLines`. The grammar is opinionated where it matters:
`target.targetKind` and `constraint.kind` are _enums_ (the nine target kinds and
eight constraint kinds declared as `const … satisfies` arrays at `:403`/`:415`),
`priority` is an integer bounded 0–100, `deadline` is an ISO-8601 string or
`null`, and `forbiddenLines` has `minItems: 1` — **an objective with no boundary
is structurally unrepresentable.** The release gate also caps a parse at
`maxTokensPerParse: 2000`
(`V6/release/vac-communication-readiness.v6release.json`).
That schema is enforced twice. `validateVacObjectiveIntent` (`:623`) is the
runtime validator: it checks the verb and target refs against a V6 reference-id
regex, label/description lengths, the priority bound, ISO timestamps, and runs
`validateConstraints`/`validateForbiddenLines` for per-entry kind membership,
uniqueness, and the 1–24 count window. The test
`'publishes a strict constrained function-calling schema'` (`:100`) drives a bad
call — `priority: 101`, `forbiddenLines: []` — and asserts both specific error
strings come back, so the validator fails on the right reasons, not just
truthiness.
### From utterance to a structured objective
The parser is a real cascade of domain-specific extractors, not a regex that
relabels anything. `extractCommandClause` strips the addressing frame ("ask /
tell / have / direct / get _Abeni_ to …"); `resolveVerb` matches one of eight
authored verbs (Repair, Map, Investigate, Protect, Build, Support, Befriend,
Report) by token set; `extractTargetLabel` slices the clause _before_ the first
modifier ("without", "by", "urgently", …) so boundaries don't leak into the
target; `resolveTargetKind` classifies the target into `agent`/`ground`/`prop`/
`relationship`/`activity`/`custom` by lexical cues. `buildConstraints` lifts a
deadline, a careful-pace request, or a report-blockers safety constraint;
`resolvePriority` maps urgency words to 90/78/55/25; `resolveAutonomyMode` reads
"exactly as I say" → `direct-tether`, "always" → `standing-intent`, "use your
judgment" → `free`, default `brief`; and `extractDeadline` resolves "by tomorrow
/ this week / today" or a literal ISO date against the creation time.
The most safety-relevant extractor is `buildForbiddenLines` (`:1479`). It scans
for `without` / `do not` / `don't` / `never` / `avoid`, normalizes the captured
phrase, and maps it to a _value reference_ — "lie/deceive/mislead" →
`value:honesty`, "private/secret/consent" → `value:privacy`,
"harm/unsafe/danger" → `value:safety` — and when the player marks no boundary,
it still injects a default `forbidden-line:respect-agent-autonomy` so the
agent's own refusals stay binding. The end-to-end test (`index.spec.ts:42`) runs
_"Ask Abeni to repair the garden gate by tomorrow, without lying to visitors and
keep it gentle"_ and asserts the verb `Repair`, target kind `prop` / label
"Garden gate", a `2026-05-02` deadline, constraints `[time, pace]`, and a
forbidden line "Do not lie to visitors" tagged `value:honesty`. The C++ parser
(`ParseTranscriptToObjective`, `V6VoiceIntentTypes.cpp:680`) reproduces the same
verb tokens, forbidden-line markers, priority tiers, and ref formats, and its UE
test parses _"map the grove path and report blockers without crossing private
gardens by tomorrow"_ to the identical structured shape — the proof the two
authorings agree.
### Negotiation — the agent's reply is not "yes"
Because an Ori is an agent and not a tool, a confirmed objective is _offered_,
not imposed. `routeVacNegotiation` (`:860`) is the outbound counterpart of the
parser: given the objective and the agent's ranked values, needs, and
relationship, it returns one of five decisions — `accept`, `clarify`,
`counter-offer`, `defer`, `refuse` — each as a `VacNegotiationRouteResult`
carrying a player-facing reply, the agent's reason, and an evidence list. The
decisions are computed, not random: `detectObjectiveValueConflict` (`:1028`)
refuses only when the objective text trips one of four value rules (honesty,
privacy, safety, autonomy) _and_ the agent actually holds that value at priority
≥ 50; `shouldCounterOffer` proposes a safer rest-first version when a late-night
task meets low energy and an existing trust bond; `shouldDefer` pushes back when
commitments stack or energy is depleted and priority is below 85. Crucially,
**coercion is a first-class, logged event**: when a steward pushes past a prior
refusal, the router returns `refuse` _and_ emits a `VacNegotiationOriEventDraft`
of type `ObjectiveRefused` with audit tags
`['vac-negotiation', 'coercion', 'objective-refusal']`, written to the Ori and
the audit log. The test
`'refuses coercion and drafts an ObjectiveRefused Ori event'` (`:393`) pins the
refusal, the `value:honesty` / `forbidden-line:negotiation :honesty` evidence,
and the event draft — pressure cannot fabricate a yes. This is where Vac hands
off to the agent-behavior negotiation layer documented in
[the cognition stack and agent behavior](./cognition-stack-and-agent-behavior.md).
### Squad comms — one channel, two realms
A household acts as a squad, so Vac runs a persistent group channel.
`FV6SquadCommsRuntime` (`V6/ue/Source/V6Voice`) routes five message kinds —
**status-report**, **callout**, **permission-request**, **inter-agent-banter**,
**directed-order** — over the gateway SFU, resolving recipients per kind (a
permission-request goes to the steward; banter to the other agents). The
behavior is honestly domain-shaped: a callout and a permission-request are
marked `bTimeSensitive`, a permission-request requires a steward response, and
inter-agent banter is `bDuckable` _and_ throttled by a 45 s window
(`BanterThrottleWindowMs = 45000`) so chatter never buries a real callout. The
load-bearing guarantee is **realm parity**: `CompareRealmParity` runs the same
script in Orun and during an Aye incarnation and asserts the routed message
_shapes_ match exactly — `BuildParityScenario` is what the
`V6VacSquadCommsTests` automation drives to zero mismatches, so comms behave
identically whether the squad is at home or incarnated in a tactical world.
## The voice seams — ASR, TTS, and the conversation LLM
The discipline at every model/audio boundary is _fail loud or report absence,
never fake_. The **ASR seam** (`VacAsrAdapter`) and its engine equivalent take
the recognizer's transcript and confidence as inputs; the parser then
_independently_ enforces the 0.7 confidence floor and a resolvable verb, so a
low-quality recognition degrades to `needs-clarification` rather than a
confident wrong objective. The default in-process transcriber is explicitly a
deterministic echo for wiring and tests — it claims a fixed 0.94 and never
pretends to have heard anything the packet did not carry.
The **conversation seam** is the other direction. Free conversational turns (as
opposed to objective commands) route to `@oshun/psyche-agent`'s
`routePsycheConversationTurn`, which assembles a real `PsycheAgentTtsPlan`
(provider `v6-local-tts`, codec `opus`, 48 kHz mono), a viseme
`PsycheLipSyncTrack` bound to the agent pawn, and a spoken-reply latency budget
capped at `PSYCHE_SPOKEN_REPLY_BUDGET_MS = 1_000` (its default stage table sums
to 646 ms). The engine twin is `V6Audio`'s `FV6ConversationAudioRuntime`,
exercised by `V6.Vac.ConversationRouting.PsycheTtsLipSyncBudget`: it routes a
grounded turn, produces a `v6-local-tts`/`opus` plan and a >8-cue lip-sync track
including the open-vowel `A` viseme, demotes a slow budget to `BudgetExceeded`,
and — tellingly — **rejects a turn whose Ori context is missing its `MemoryRefs`
with an `InvalidContext` failure**, so the agent cannot speak from a hollow
context. The actual utterance text and the model that authors it come from the
governed `PsycheCognitionGateway` seam — which _fails loud_ on a gateway error
and never substitutes a template — documented in full on
[the cognition stack page](./cognition-stack-and-agent-behavior.md). Vac owns
the routing, the budgets, and the lip-sync contract; the LLM and the TTS engine
are injected behind it.
## Where this connects
- **The mind that replies** —
[the cognition stack and agent behavior](./cognition-stack-and-agent-behavior.md)
— owns the LLM gateway, the Sophia grounding and Isis policy gates a reply
passes, and the value-based negotiation Vac's `routeVacNegotiation` is the
player-facing surface of.
- **The story that remembers** — [the Clio story engine](./clio-story-engine.md)
— reads the Ori event log Vac writes to, including the `ObjectiveRefused`
coercion events, turning the consequences of what was said into a Chronicle a
human wants to read.
- **The client that hears and speaks** —
[the UE5 client modules and embodiment](./ue5-client-modules-and-embodiment.md)
— is where `V6Voice` mic capture, the `V6UI` intent-grammar confirmation
builder, the squad-comms HUD, and TTS playback on the agent pawn actually
live.
- The hub overview and the readiness gate
(`verify:v6 vac-communication-readiness`, backed by
`V6/release/vac-communication-readiness.v6release.json`) live in
[../V6_ARCHITECTURE.md](../V6_ARCHITECTURE.md).