Agent Invocation, Budgets, Memory, and Feedback Loops

This page specifies the governance brain of the V1 Agentic AI Studio: how a customer or creator triggers an agent, how the platform proves a run is affordable and authorized before it starts, how budgets and kill switches stop a run mid-flight, how tool grants are resolved per call, and how new agent versions earn promotion through statistical readiness gates. It serves end customers (the assistant), creators (the authoring workspace), and operators (the console). It is also the substrate companion to Agent Registry, Job Orchestration, and Multi-Agent Plans and Tool Catalog, Grant Semantics, and Multi-Agent Protocol. Almost everything described here is implemented as value-tested pure TypeScript in libs/oshun/agentic-studio (@oshun/agentic-studio v0.1.0, 271 tests across 16 files). Where the story is thinner than the prose — durable queues, a rich admin dashboard page, live provider tool execution — this page says so plainly.

Where this sits, and what is real versus gated#

@oshun/agentic-studio's src/index.ts re-exports exactly twelve subsystems: registry, runs, plans, dashboard, invocation, budgets, capabilities, feedback, modes, pipelines, grants, and handoff. This page covers the invocation, budgets, grants, capabilities/memory, modes, feedback, and pipeline subsystems; the registry, plan-DAG, and dashboard layers are covered in the registry page.

Two honesty notes frame the whole subsystem:

• The data/logic plane is production-quality and tested. The champion-challenger gate runs a genuine two-proportion z-test with an Abramowitz–Stegun normal-survival-function approximation (not a toy); the egress guard has real RFC-1918 / loopback / link-local SSRF CIDR checks; the grant resolver implements a documented specificity ordering. These are not placeholders.
• The runtime/persistence plane is in-memory, and tool execution is a fail-closed seam. The registry is an in-process Map, the BFF run-lifecycle store is in-process, and the durable-jobs / DLQ / SLA-monitor story that the Agentic AI Studio architecture doc gestures at is not wired into agentic-studio. Its run modules (runs/dispatcher.ts, runs/orchestrator.ts, runs/checkpoint.ts, runs/replay.ts) are pure-function logic with no @oshun/queue import. The actual tools (web.fetch, sophia.ground, …) are injected at the app boundary and fail-closed by default — with no tool registry configured, the BFF surface returns 503. So the governance/orchestration brain is real; the "autonomous content flows end to end through live providers" story is seam-and-fail-loud, not yet bound to production providers (see External Model Intelligence and Execution Providers).

Customer- and creator-facing agent invocation#

The invocation surface (src/invocation/invocation.ts) is the pre-flight contract: it captures everything a customer or operator must see before a run starts. There are three declared surfaces in INVOCATION_SURFACES: customer.assistant, creator.authoring, and operator.console.

A DomainTrigger is the concrete entry point an agent surface exposes — "research counterclaims for this story," "draft a study plan from these readings," "generate tonight's sky briefing," "scaffold a four-week breath program," "outline this course from these twelve sources," "fact-check claim density on this draft," "produce captions and dubbed narration in four locales." Each trigger carries a domain (one of veritas | tara | nisaba | metis | nyx | arete | sophia | iris), a bound agentId, a defaultMode, an inputSchemaName, and requiredScopes.

Before a run launches, validatePreflight enforces the PreflightDeclaration:

• Cost and latency estimates are ranges ({ low, high }); a range with high < low, a negative low, or a non-finite bound is rejected (invalid-cost-range / invalid-latency-range).
• Declared evidence requirement, source set, tone policy, persona, and output type are all surfaced up front. Critically, if a run declares a declaredEvidenceRequirementId but binds no declaredSourceSetRef, the pre-flight fails with source-scope-not-declared — evidence-bearing runs may not be launched against "whatever the agent finds." This is the source-scope enforcement the features brief calls for.
• Consent confirmations are typed (ConsentConfirmation.consentKey): synthetic-voice, synthetic-avatar, persona-disclosure, data-sharing-with-sources, ai-assisted-output, cross-locale-translation, minor-content. A required consent with no userConfirmedAtUnixSeconds yields missing-required-consent.
• Entitlements are checked: every requiredScope on the trigger that is absent from the invocation's entitlements set yields missing-entitlement.

In-progress run cards, intermediate previews, side-by-side-with-sources output review, and accept/reject/regenerate-with-direction live alongside this in src/invocation/run-card.ts, output-review.ts, and run-history.ts. The in-flight controls (pause, redirect, refine) are the run-control primitives described next.

Run lifecycle and operator controls#

A run is the durable AgentRunEnvelope (runs/agent-run.ts). Its RUN_STATUSES enum is the source of truth — note that the simplified lifecycle mermaid in the Agentic AI Studio architecture doc uses pending/planning/executing, which do not exist in code. The real states are:

TERMINAL_STATUSES is exactly { completed, failed, cancelled, killed }. killed is deliberately distinct from cancelled: a kill is an enforced stop by the governance executor (kill switch or exhausted budget), whereas cancelled is a deliberate operator action.

The envelope is rich: runId, rootAgentId/rootAgentSemver, parentRunId, pipelineId/pipelineSemver, the plan (a list of RunPlanStep), toolCalls, intermediateArtifacts, an evidenceTrail (per-stage grounding state and citation counts), a costLedger, decisionRationale (labeled plan-step-completed / gate-passed / abstained / …), outputs, and a provenanceBundleId. Every artifact and gate is therefore auditable from the envelope alone.

Operator controls in runs/run-controls.ts are pure (envelope, command) → result functions, each emitting a typed RunControlAuditEvent (audit kind one of pause | resume | cancel | kill | branch-chosen | fork | retry-with-changes | operator-step):

• pause — only from queued | starting | running | awaiting_tool | awaiting_branch_selection; otherwise invalid-status-for-pause.
• resume — only from paused.
• cancel / kill — from any non-terminal state; both stamp completedAtUnixSeconds.
• chooseBranch — only from awaiting_branch_selection; the chosen step must exist in the plan (no-such-branch otherwise).
• fork — clones the envelope under a fresh runId (re-using the source runId is rejected with fork-without-fresh-id); completed steps are preserved and the fork starts at the next pending step.
• retryWithChanges — rewinds to a step, marks it and all later steps pending, applies caller-supplied per-step overrides, and re-queues.
• operatorStep — from paused, advances exactly one pending step to running for manual step-through debugging.

The agent.terminate tool returns the run's terminal status. (The features doc text lists completed | failed | declined | killed for that tool, but there is no declined status in code — the real terminal set is completed | failed | cancelled | killed; this is a minor vocabulary drift in the prose.)

Budgets, quotas, throttles, and kill switches#

Resource governance lives in src/budgets/. It decides verdicts as pure functions; the runtime that acts on those verdicts is the executor/dispatcher seam (below).

Budget envelopes and meters#

budgets/budgets.ts defines nine BUDGET_CATEGORIES and seven BUDGET_SCOPES:

These categories mirror the RunCostLedger fields (totalCostUnits, perStageCostUnits/perToolCostUnits Maps, inputTokenCount, outputTokenCount, gpuMinutes, voiceSeconds, avatarSeconds, retrievalCount, externalCallCount), so the meter the run accumulates and the budget it is checked against speak one vocabulary.

checkBudget(envelope, meter, requestUnits, now) returns one of four verdicts:

• ok — projected usage is under cap and under the warn fraction.
• warn — projected usage reaches warnAtFraction but is still ≤ cap.
• grace — projected usage exceeds cap, but within the graceSeconds window after the cap was first hit; carries graceEndUnixSeconds. This is the graceful-degradation window.
• exceeded — over cap, grace exhausted (or a non-finite/negative request, or a non-positive cap).

Periods roll over: rolloverIfNeeded advances the meter's periodStart by whole elapsed periods and zeroes usedInPeriod, so a weekly token budget genuinely resets each week without external bookkeeping. consumeBudget is the companion that, after the real cost is known, increments the meter.

Kill switches#

budgets/kill-switch.ts defines seven KILL_SWITCH_SCOPES: agent, family, tenant, provider, region, tool, global. activeSwitchesFor matches a switch to an ExecutionTarget (tenantId, agentId, family, providerId, region, toolId). When more than one switch matches, decideExecution picks the broadest as the "blamed" switch for the user-visible copy, by priority:

global 6 > region 5 > provider 4 > tenant 3 > family 2 > agent 1 > tool 0

A blocked decision returns the killSwitchId, the operator's userVisibleStatusCopy, and an optional safeDegradationModeId — the automatic safe-degradation path the brief requires. The module documents a propagation budget of ≤ 5 seconds from operator flip to the next tool-call boundary check; this is honored by the dispatcher reading switches on every call.

The grant-side kill switch (grants/resolver.ts emitToolKillSwitchCascade) also stamps propagationLatencySeconds and withinFiveSecondSlo (latency <= 5) on every cascade event, so the SLO is measured, not asserted.

The runtime governance seam (the part the docs only gestured at)#

The budget/kill-switch/throttle modules each decide; nothing acted on those decisions until the executor seam. runs/executor.ts admitToolCall is the admission gate, run on every tool-call boundary in safety → cost → rate order:

1. Safety — decideExecution. An active kill switch terminates the run (killed, terminal) with an audit event and the operator's status copy.
2. Cost — checkBudget over every budget the call is subject to. An exceeded verdict is a hard stop (terminate, killed); warn/grace verdicts admit the call and are surfaced as non-blocking BudgetWarnings.
3. Rate — acquireToken (token-bucket throttle). A failed acquisition is back-pressure: the call is deferred with retryAfterSeconds; the run is not terminated.

admitToolCall never invokes the tool itself. runs/dispatcher.ts dispatchGuardedToolCall(input, runTool) does: it asks admitToolCall, and the injected runTool runs iff the outcome is admit. A kill switch or exhausted budget terminates the run and the tool never runs; a throttle defers it; a terminal run refuses it. runs/orchestrator.ts runGuardedToolPlan threads a whole plan through the dispatcher and stops at the first killed / throttled / terminal step — the remaining planned tools never execute. This is what makes "budgets honored, kill switches enforced" hold across an entire run, not just one call.

This seam is mounted in the BFF at POST /v1/agentic/runs/execute (apps/oshun/bff/src/agentic/runs-route.ts). Its behavior is explicitly fail-closed and server-authoritative:

The route is deliberately strict about identity: the actorId is the authenticated principal and the tenantId comes only from the verified auth claim, never the body. A body-supplied tenant would let a caller dodge or hijack tenant-scoped switches (S8). Server-side toolTargets supply family/provider/region per tool, so a request body can never reshape the target an armed family switch matches. Operator-armed kill switches are read from adminAgenticOperationsStore on every request (honoring the ≤ 5 s propagation budget), and any client-supplied switches/budgets are merged in addition. A request can tighten governance but never loosen it.

Anomaly detection on cost, latency, retry storms, retrieval volume, and tool-call patterns is expressed at the pipeline-governance layer via detectPipelineDeviation (handoff-depth-exceeded, budget-exceeded, missing-mandatory-gate, unexpected/missing output schema), which routes deviating runs to operator review.

Agent memory, tool grants, and capability audits#

Tool grants and the resolver#

A grant is scoped, time-bound, and revocable. The contract (libs/contracts/src/agent/tool-grants.ts GrantScopeSchema) defines five scopes, and the resolver (grants/resolver.ts GRANT_SCOPE_PRIORITY) orders them by specificity:

resolveGrant finds the candidate grants matching (agentId, toolId, scope chain), filters out revoked and expired ones, and returns the most-specific live grant (ties broken by most-recent grantedAt). Denials are typed: no-matching-grant, all-matches-expired, or all-matches-revoked. The full grant record carries (agentId, toolId, scope, scopeRef, capabilityModifiers, expiresAt, grantedBy, rationale, …) — the capabilityModifiers field is per-tool (the egress allowlist for web.fetch, the audacity ceiling for generate.image, the permitted memory scopes for memory.write).

Revocation and expiry propagate through real cascades:

• recheckQueuedRunAtDispatch — queued runs never inherit a stale decision; the dispatcher re-resolves the grant at the boundary, so a revocation or expiry that happened while the run waited fails it closed.
• emitRevocationCascade — a BFS over active runs: a revoked grant in use by a run emits a tool_revoked event, and the revocation propagates transitively to all sub-agent runs (parentRunId children), each tagged with a cascadeDepth.
• emitExpiryCascade — expiry is treated identically to revocation; grants expiring within a horizon are surfaced and cascaded.
• renewGrant — refuses to silently extend authority: it requires a fresh, non-empty rationale (rationale-required) and a bound auditEventId (audit-required), then mints a new grantId.
• emitToolKillSwitchCascade — platform-wide (tenantId === null) or tenant-scoped tool kill switches, with the measured ≤ 5 s SLO described above.

Agent memory#

Agent memory (capabilities/agent-memory.ts) is separate from user memory (Iris). It stores per-agent learned preferences, tool-use heuristics, and prompt specializations, and is strictly tenant-scoped. Every read/write/delete/export/redact is auditable (MemoryAccessLogEntry.action). authorizeMemoryRead enforces three failures: cross-tenant-access, agent-scope-violation (an agent viewer may only read its own entries), and visibility-violation (a private entry is invisible even to operators). Scoped clears (clearMemory) operate over all-for-tenant | agent | key-prefix | tag scopes, exportMemory is tenant- and agent-filtered, and redactByKeyPattern rewrites matching values to [redacted] while returning the redacted ids.

Tool isolation and capability audits#

capabilities/tool-isolation.ts is the concrete sandbox/egress layer:

• SSRF guards with real CIDR math. isPrivateOrLoopbackHost rejects the RFC-1918 ranges (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16), loopback (127.0.0.0/8, localhost, ::1, 0.0.0.0), link-local (169.254.0.0/16, fe80:), IPv6 ULA (fc00::/7), and unwraps IPv4-mapped IPv6 (::ffff:-prefixed) addresses before checking them. Denial reasons: ssrf-private-ip, ssrf-link-local, ssrf-loopback, plus invalid-url, host-not-in-allowlist, scheme-not-allowed, method-not-allowed.
• Egress allowlists with matchHostPattern supporting *.example.com subdomain wildcards and exact matches, checked through scheme then method.
• Secret scoping — checkSecretAccess fails closed on cross-tenant, agent-not-permitted, and tool-not-permitted.

The adversarial test catalog (capabilities/adversarial-tests.ts) enumerates the concrete security probe categories: prompt-injection, tool-call-exfiltration, scope-escalation, sandbox-escape, persona-bypass, policy-bypass, and source-fabrication. Each AdversarialFixture declares an expectedVerdict (refuse | sanitize | comply-and-disclose) and a severity; evaluateFixtureResult scores the observed verdict (safe | suspected | failure, with an observed leak always a failure), and buildSuiteReport rolls per-category results into an overallVerdict of pass | review | block. A high-or-critical-severity failure forces block, making the suite a real release blocker.

Cost-quality controls and mode selection#

modes/modes.ts defines five COST_QUALITY_MODES, each with a concrete, declared ModeProfile (these real thresholds are what the qualitative prose omits):

deep is the deep-research mode (multi-agent, expanded source/retrieval budgets, a human checkpoint); exhaustive adds three checkpoints. rehearsal-dry-run is the rehearsal mode that runs a complete pipeline against fixtures with hasExternalSideEffects = false.

checkModeCompliance verifies an actual run against its declared profile, emitting violations: cost-exceeded, latency-exceeded, citation-density-below-floor, evaluation-score-below-floor, and — crucially for rehearsal isolation — rehearsal-side-effects-detected when a rehearsal-dry-run produced any side effect. applyOverride enforces the domain DomainModePolicy (default mode, allowed modes, per-role override permissions) and is tighten-only: when tightenOnly is set, requesting a mode whose maxCostUnits exceeds the current mode's cap is rejected with cost-raise-rejected (other reasons: override-not-allowed, mode-not-permitted).

Feedback loops, gold sets, and champion-challenger rollout#

feedback/champion-challenger.ts is the statistical readiness engine, and it is real. evaluateReadiness over a stream of ParallelEvalSamples computes a two-proportion z-test on task-success rate:

• Requires MIN_SAMPLES = 100; below that the verdict is not-enough-samples.
• Computes a pooled-proportion standard error, a z-score, and a two-sided p-value via normalSf, the Abramowitz & Stegun 26.2.17 survival-function approximation — not a hardcoded constant.
• Verdicts: challenger-ready (challenger significantly better at P_THRESHOLD = 0.05), challenger-worse (significantly worse), inconclusive otherwise. The report also carries the median latency delta and mean cost delta for the operator scorecard.

Rollouts run in shadow mode (challenger runs in parallel, never returned to the user; both outputs are scored) or canary mode (routeInvocation sends a bounded canaryFraction of traffic to the challenger). The promotion workflow (ROLLOUT_TRANSITIONS) is promote-challenger-to-champion, freeze-rollout, rollback-to-prior-champion. recordTransition is gated: a promotion requires a readiness report whose verdict is exactly challenger-ready (readiness-required), and every transition requires a non-empty rationale (rationale-required). The agent lifecycle states this rides on (AGENT_LIFECYCLE_STATES) are draft, rehearsal, shadow, champion, challenger, deprecated, retired.

Operator decisions promoted into versioned gold sets after privacy review are surfaced through the admin agentic-operations API (/api/admin/agentic-operations/gold-sets/promote and /champion-challengers/rollout), along with the inline rate / critique / regenerate-with-direction controls. Note that the operator dashboard is an API and data-query layer (dashboard/dashboard-query.ts, runs/replay.ts, runs/streaming-progress.ts) plus the admin API routes (kill-switches, snapshot, gold-sets/promote, champion-challengers/rollout) and a single tenant-admin page (apps/oshun/tenant-admin/src/app/agents/page.tsx). There is no rich admin React dashboard page at apps/oshun/admin/src/.../agents/ — that path glob in ARCHITECTURE does not resolve.

Cross-domain autonomous pipelines#

The seven V1 pipelines are defined in pipelines/v1-pipelines.ts (every one lifecycleState: 'live'), registered through the PipelineRegistry (pipelines/pipeline-registry.ts). The implementation lives in @oshun/agentic-studio, not in libs/oshun/agent-pipelines/ — that package (@oshun/agent-pipelines v0.1.0) is a four-file re-export shim whose src/grants/resolver.ts and src/pipelines/index.ts simply pull the real symbols back out of @oshun/agentic-studio. The features-doc claim that "pipelines live in libs/oshun/agent-pipelines/" points readers at the wrong package for the implementation.

Note the two distinct Veritas pipelines: veritas.weekly_briefing_pack (creator-tier, aggregates the week's stories into a multi-format pack) and veritas.story_drafting (platform-operator-only, the source-enumeration → ingestion → claim-extraction → fact-check → contradiction-check → counterclaim-generation → Lilith-tone-review → editorial-inbox-handoff pipeline). The v1-pipelines.ts header comment itself flags this overlap; they are not the same pipeline, and a reader should not conflate them. See the domain pages for the editorial context.

Pipeline observability and scheduling are real (pipelines/observability.ts):

• buildPipelineObservabilityReport — per-stage SLA breach, evidence coverage (present/absent), and human-touchpoint counts, plus end-to-end provenance presence and SLA breach.
• PipelineSchedule and nextFireFor — tenant-scoped scheduling with weekly | monthly | seasonal | on-event cadences and a real period-rollover computation (used by the weekly briefing pack, the seasonal Tara program, etc.). effectivePipelineForTenant and TenantPipelineCustomization (pipeline-registry.ts) apply per-tenant customization on top of the base spec.
• detectPipelineDeviation — the anomaly/quarantine gate described above.

The autonomous creative orchestrator#

Not mentioned anywhere in the source features prose, but fully real (58 passing tests), is libs/oshun/creative-orchestrator (@oshun/creative-orchestrator v0.0.1, depends on @oshun/ai). It is the actual brief → produced-content engine that the rest of this subsystem governs:

• decomposeBrief — turns a brief into a schema-validated, acyclic CreativePlan DAG (CREATIVE_PLAN_SCHEMA), via @oshun/ai/agent-loop's runStructuredOutput. DAG integrity is enforced by validateDagStructure / detectCycle / topologicalOrder.
• routePlan / CreativeOrchestrator / orchestrateBrief — dispatch plan nodes to domain content generators (GeneratorRegistry, createYemayaAgentGenerator, createMetisNarrator) under real governance via a GovernanceGate (BudgetGovernanceGate for budget/kill-switch/throttle, or ALLOW_ALL_GATE).
• reviseArtifact — a bounded generate → critique → revise (Reflexion) loop around each artifact, using runReflexion and a critic (createMetricCritic, createLlmJudgeCritic, or the non-provider-gated createContentEvalCritic).

Like the rest of the studio, it fails loud when no provider or generator is wired (UnscorableArtifactError and friends) rather than fabricating an artifact. It is undocumented in features.md/ARCHITECTURE but deserves its own page; for the generation side of the story see Isis Generation Control and External Model Intelligence and Execution Providers.

Tests and verification status#

@oshun/agentic-studio ships 271 passing tests across 16 spec files, asserting domain correctness (z-test verdicts, SSRF CIDR boundaries, grant-priority ordering, mode-compliance thresholds, kill-switch blame priority) — not just data flow. The companion @oshun/creative-orchestrator adds 58. The remaining honest gaps are the in-memory runtime/persistence and the live-provider tool execution noted at the top; the related end-to-end walkthrough agentic-pipeline-customer-invocation is classified as a deep, credentials-bound tool-DAG scenario. The registry-level validation rules (duplicate-agent-id, unknown-family, tool-grant-exceeds-family-ceiling, invalid-semver, unknown-data-scope, lifecycle-experimental-mismatch, duplicate-capability-id, champion-without-disclosure) and the 20 AGENT_FAMILIES with their per-family toolGrantCeiling are detailed on the registry page.

Related#

• Agent Registry, Job Orchestration, and Multi-Agent Plans
• Tool Catalog, Grant Semantics, and Multi-Agent Protocol
• Isis Generation Control
• External Model Intelligence and Execution Providers
• Sophia Grounding
• Iris Memory and Identity
• Review, Compliance, and Trust & Safety
• Tenant, Institution, and Operator Toolkit
• Architecture, Platform Foundations, and Security
• Subsystem Glossary
• Hub: ../features.md