Search, Discovery, and Knowledge Graph

Search and discovery are the cross-domain "find anything, then see what to do next" surfaces of V1. They serve every member who opens the universal search sheet or scrolls a mixed-domain recommendation rail, and they sit one layer above the six customer domains (Tara, Veritas, Nyx, Arete, Nisaba, Metis) — fanning queries out to each domain's data and blending the results back into one ranked list inside the BFF. This page is candid about a split that the rest of the V1 docs glossed over: there are two search stacks in this repo, and only the simpler one ships. The live /v1/search and /v1/recommendations paths use deterministic, lexical, domain-fan-out ranking over data the system actually holds; a much richer signal/candidate/ranker stack (libs/oshun/search-discovery) exists, is fully tested, and is explicitly retired from V1 scope. Read this page as the description of what runs, with the aspirational library called out honestly wherever it appears.

Where this sits in V1#

member query / home rail
        │
        ▼
  apps/oshun/bff   ── GET /v1/search ──────────► search/ranking.ts (lexical)
   (Fastify)       ── GET /v1/recommendations ─► recommendations/scoring.ts
                   ── POST /v1/recommendations/feedback
                   ── POST /v1/search/offline-eval ─► @oshun/search-discovery (the one adopted piece)
        │
        ▼  domain adapters (app.domainAdapters.*)
  tara · veritas · nyx · arete · nisaba · metis

Every route is scope-gated: a query only ever fans out to the domains the caller's token authorizes (resolveAuthorizedShellDomains / resolveAuthorizedDomains over authContext.scopes). The canonical domain set is OSHUN_DOMAIN_IDS = ['tara','veritas','nyx','arete','nisaba','metis'] (libs/oshun/auth/src/types.ts), re-exported as DOMAIN_IDS through apps/oshun/bff/src/routes/feed-helpers.ts.

The honest split. libs/oshun/search-discovery/src/index.ts:1-15 opens with a banner: the library is "RETIRED FROM V1 SCOPE (audit E5, 2026-06-11)." Its ranker scores DiscoveryObject features the live candidates do not carry — persona-tone fit, evidence integrity, grounding state, per-object entitlement/region — and the repo's honesty rules forbid inventing those signals just to feed it. So the live paths "keep their own real ranking over the data they actually have." The library stays tested and intact as the V1.x target: adoption becomes real only when search candidates begin carrying genuine DiscoveryObject metadata. Wherever this page describes the rich stack (signals, collaborative filtering, concept-graph candidates, the feature ranker, A/B experimentation), assume not shipped unless it says otherwise.

Live Universal Search#

Universal search is registered at both GET /search and GET /v1/search (apps/oshun/bff/src/routes/search.ts:230-243), behind the abuse-protection and auth pre-handlers. A member types into the search sheet (web surface at apps/oshun/web/src/app/search/page.tsx) and the BFF returns one flat, ranked, cross-domain result list.

Query parsing and guards#

• The query is read from either q or query and trimmed; an empty query is allowed (it returns curated/feed/user candidates ranked by kind and intent rather than by lexical match).
• An optional domain filter narrows results to a single domain; the value is validated by isSearchDomainFilter and a bad value returns 400 (invalid_domain_filter).
• Pagination defaults to limit 20, max 100 (parsePagination).
• Non-empty queries are recorded per-user via recentSearchStore.addQuery(...), which is what backs the "recent searches" affordance.
• Missing auth → 401 (missing_auth_context); a token with no domain scope → 403 (domain_scope_missing).

Three candidate pools merge before ranking#

A point the prior architecture text omitted entirely: live search does not query a single index. It assembles three distinct candidate pools and ranks them together (search.ts:140-179):

1. Domain feed highlights and continue items. For each authorized domain, the route calls fetchDomainContinue(app, domainId, userId, failures) and fetchDomainHighlights(app, domainId, failures) in parallel, then flattens and applies the domain filter. These are the same "continue where you left off" and "domain highlight" objects the home feed uses.

2. Curated universal seeds (selectUniversalSearchSeeds, apps/oshun/bff/src/routes/universal-search-seeds.ts). A static UNIVERSAL_SEARCH_SEEDS catalog of representative objects — one or more per object class, filtered to the member's authorized domains and the active domain filter. The seed kind union is exactly the object taxonomy the docs promise: ritual, practice, concept, passage, claim, source, notebook, collection, program, sky-event, course, lesson, learning-artifact. Each seed carries a real targetPath (e.g. /library/passage/nisaba-passage-speech, /events/nyx-quadrantids-2026, /courses/crs-002).

3. The member's own objects (collectUserObjectCandidates, apps/oshun/bff/src/search/user-object-candidates.ts). This is the C7 audit fix that made search find things the member actually owns. It pulls from four real per-user stores:

   • saved library items (savedLibraryItemsStore, durable/LWW-synced) → kind saved-item, one per saved object with its own domain;
   • library collections (domainStubsStore.collections, owner-scoped, surfaced under Nisaba) → kind collection;
   • Nisaba notebooks (nisabaConsumerStateStore.listNotebooks) → kind notebook;
   • Arete habits (domainStubsStore.habits.listForOwner, strictly the member's own) → kind practice.

   The module is honest about emptiness: "a member with nothing saved contributes no candidates." There is no fabricated personalization — if you own nothing, this pool is empty.

The three pools are concatenated, mapped onto a uniform SearchRankingCandidate ({ id, domain, kind, title, summary, ...passthrough }), and handed to rankSearchCandidates(candidates, query).

The live ranker: lexical match + kind boost + domain intent#

The shipping ranker (apps/oshun/bff/src/search/ranking.ts) is deterministic and purely lexical. It is not the feature-rich libs/oshun/search-discovery ranker — there is no persona, evidence-integrity, grounding-state, or entitlement-class scoring on the live path. A candidate's score is the sum of three components (rankCandidate, lines 93-116):

score = lexical(title/summary match) + kindBoost(per kind) + domainIntentBoost

Lexical weights (DEFAULT_WEIGHTS, ranking.ts:38-65):

An empty query short-circuits scoreLexical to a flat 1, so kind boost and intent become the ordering signal.

Per-kind boosts (kindBoosts) reward the most actionable result types — a "continue" item ranks above a generic highlight, and learning/practice objects above passive references:

Unknown kinds fall back to the highlight boost (7).

Domain intent routing (resolveDomainIntent, lines 165-217) is a concrete, undocumented relevance feature. It scans the query tokens and infers a target domain from vocabulary, then boosts that domain by 10 and penalizes the others by 4:

So "morning ritual" tilts toward Tara, "calculus lesson" toward Metis, "verify this claim" toward Veritas — even when those tokens appear in other domains' titles.

Sort and tie-breaking (rankSearchCandidates, lines 75-90): candidates with a non-positive score are dropped, then results sort by descending score; ties break by canonical domain order (DOMAIN_IDS.indexOf), then alphabetically by title. The result is stable and reproducible for the same inputs — important for caching and for the offline-eval gate below. The route strips the internal breakdown before returning, exposing only the final score per item.

Response envelope, resilience, and caching#

Search responses carry a partial-failure envelope the prior docs omitted (SearchRoutePayload, search.ts:48-64). Because the route fans out to up to six domains, a single domain failing must not blank the page:

• domainStatus: Record<DomainId, 'ok' | 'degraded' | 'forbidden'> — per-domain health, built by buildAggregationFailureState from the collected failures.
• errors[] — { domain, stage, message } for each failed domain call.
• partial / partialFailure — set when any authorized domain degraded, so the client can show a "some results may be missing" banner instead of treating an empty section as "nothing found."
• attachPartialResponseTrace attaches a debug trace for the partial response.

Responses are memoized in SEARCH_ROUTE_CACHE, a createRouteResponseCache with a 20 000 ms TTL, keyed by user, lowercased query, domain filter, and pagination (buildAuthenticatedCacheKey). Cache hits and misses both set applyResponseCacheHeaders.

Finally, each request fires trackSearchTelemetry({ userId, query, resultCount, domainFilter, partialFailure, surface: 'results' }) (apps/oshun/bff/src/telemetry/search-telemetry.ts); the emit is fire-and-forget and a failure only logs a warning — telemetry never blocks the response.

Cross-Domain Recommendations#

Recommendations are the "what's worth your attention right now" rail, blended across every authorized domain with transparent reason labels. Routes: GET /recommendations, GET /v1/recommendations, and POST /v1/recommendations/feedback (apps/oshun/bff/src/routes/recommendations.ts:439-581). Pagination defaults to limit 12, max 50.

Candidate generation is domain-adapter fan-out only#

This corrects a real inaccuracy in V1/features.md: the live path does not do collaborative filtering, content-similarity, embedding, concept-graph, or cross-domain-bridge candidate generation. Those generators live only in the retired libs/oshun/search-discovery/src/candidates/. Live candidate generation is exactly six adapter calls, each shaping its own domain's objects into RawCandidates. The six-call fan-out lives in apps/oshun/bff/src/routes/recommendations.ts (lines 64/95/135/186/226/287); the RawCandidate shape is defined at recommendations/scoring.ts:20-32:

Each fetcher assigns a reason, a human explanation, a baseScore, and a typed meta blob for rich-card rendering. The reason often depends on real state — Arete tags a goal streak_support only when goal.streakDays > 0 ("Keep your N-day streak alive"); Nyx tags a sky event time_based only when its windowStart is today ("Visible tonight — don't miss it"); Veritas chooses personalized / trending / new_content from hasInteracted and score.

The reason taxonomy is a real nine-value enum#

The shipping reason taxonomy is the machine-readable RecommendationReason union (recommendations/types.ts:21-30), not the free-text "because you saved X … fresh in your concept graph" taxonomy the features doc describes:

RecommendationReason =
  'trending' | 'personalized' | 'time_based' | 'goal_based' | 'popular'
| 'new_content' | 'streak_support' | 'cross_domain' | 'editorial'

Each item still carries a human explanation string for the card, but clients can also filter or group by the stable category. The RecommendationItem contract (types.ts:42-61) exposes { id, domain, itemType, title, subtitle, explanation, reason, score (0–100), meta }.

Scoring: a five-dimension weighted composite#

scoreAndRankCandidates(candidates, limit, options) (recommendations/scoring.ts:112-184) blends five normalized dimensions into a single 0–100 score:

The two reason→dimension maps are concrete (scoring.ts:69-92):

So a sky event visible tonight rides recency; a streak nudge or goal rides engagement. The diversity term is piecewise: a domain at or below the 1/6 target gets up to 100; an overrepresented domain is penalized at roughly 3× the overshoot. Ties break by preferring the underrepresented domain (scoring.ts:175-181), and the final list is capped at limit.

Preference alignment is driven by the member's profile. buildOshunRecommendationRankingProfile(profileRecord.preferences, { currentDaypart, availableDomains }) and resolveOshunPersonalizationDaypart(new Date()) come from @oshun/auth-client (defined in libs/oshun/auth/src/personalization.ts). The daypart is one of morning / midday / evening, and it produces a preferredDomainOrder. resolvePreferenceScore (scoring.ts:186-216) maps that order onto a score curve — first preferred domain 100, then 80 / 60 / 40, and 20 for anything past fourth or unranked. A member with no preferences gets a flat 50, so personalization gracefully degrades to neutral.

Response envelope and caching#

The recommendations payload (RecommendationsRoutePayload, types.ts:95-115) mirrors search's resilience model: items / results, pagination, domainBreakdown (count per domain via computeDomainBreakdown), errors[], and partial / partialFailure. A failed domain fetch pushes a { domain, message } and is excluded — the rail still renders from the domains that answered. Responses cache in RECOMMENDATIONS_CACHE at 30 000 ms TTL, keyed by user, profile revision, and pagination — so a preference change (which bumps the revision) busts the cache immediately.

Recommendation feedback#

POST /v1/recommendations/feedback (recommendations.ts:454-581) is a real, shipping endpoint the prior search/discovery docs never mentioned. The body is validated strictly:

• itemId (or legacy recommendationId), domain, and signal are required — otherwise 400 (invalid_body).
• signal must be one of hide | less | more (RecommendationFeedbackSignal), else 400 (invalid_signal).
• domain must be in DOMAIN_IDS, else 400 (invalid_domain).

The handler builds a RecommendationFeedbackPayload (userId, itemId, domain, itemType, signal, reason, timestamp, plus optional source, surface, sourceDomain, targetPath, attributionId) and logs it for downstream processing. When the feedback is genuinely cross-domain — surface === 'cross-domain' from a home or hub source — it also fires trackCrossDomainRecommendationFeedback(...) (apps/oshun/bff/src/telemetry/cross-domain-recommendation-telemetry.ts), recording the source→target domain hop, signal, reason, and attribution. The endpoint is candid in its own comments that persisting feedback to a queue / event stream for model training is a downstream concern — today it validates, logs, emits telemetry, and returns { ok: true, feedback: {...} }.

The One Adopted Piece of the Retired Library: the Offline-Eval Gate#

Exactly one module of @oshun/search-discovery is wired into the running product: the offline evaluation release gate, mounted at POST /v1/search/offline-eval (apps/oshun/bff/src/search/offline-eval-route.ts, registered in apps/oshun/bff/src/server.ts:961). It exists because the V1 exit criterion asks for "offline evaluation … operational," and the rest of the library — which computes real per-slice ranking metrics — was otherwise a dead island. This route mounts the real gate so an operator or CI job can score a candidate ranker/recommender against a baseline before it ships.

The gate imports buildSearchReleaseGateSummary, OfflineEvalSlice, and DriftReading from @oshun/search-discovery and validates the request body strictly. Each slice must carry a sliceId, domain, locale, and all six metrics in SLICE_METRICS:

Optional driftReadings must name a known detector (ranker-quality, candidate-generator, or signal-pipeline) with baseline, observed, and an mdeRatio. The gate (buildSearchReleaseGateSummary, in libs/oshun/search-discovery/src/evals/offline-evals.ts:158) compares each candidate slice to its baseline and fails any metric whose observed drop exceeds |baseline| × mdeRatio (the minimum-detectable-effect threshold; mdeRatio defaults to 0.05). The returned summary reports ok, missingCandidateSliceIds, unbaselinedCandidateSliceIds, sliceFailures, sliceMetricFailures, and driftFailures — and ok is true only when all of those are empty:

{
  "summary": {
    "releaseId": "rank-v3",
    "ok": false,
    "sliceFailures": [{ "sliceId": "tara-en", "ndcgAt10": 0.71, "...": "..." }],
    "sliceMetricFailures": [
      {
        "sliceId": "tara-en",
        "metric": "ndcgAt10",
        "baseline": 0.78,
        "candidate": 0.71,
        "observedDrop": 0.07,
        "allowedDrop": 0.039
      }
    ],
    "driftFailures": []
  }
}

A malformed body returns 400; ok: false means the candidate must not ship. This is a real, non-fabricated gate — it would fail a regressing ranker. What is not mounted is the online side: the experiment metrics, significance machinery, guardrails, and canary/kill-switch logic in libs/oshun/search-discovery/src/experiments/ (ab-framework.ts, canary.ts) are not imported by any app. So where V1/features.md presents offline and online evaluation as one shipping suite, only the offline gate is live.

Knowledge Graph: Spec, Not Live Path#

The prior architecture text described search/discovery as running over a "live Neo4j concept-graph with Sophia-evaluated promotion." That is aspirational. Neither /v1/search nor /v1/recommendations touches a concept graph or Neo4j: the search ranker is lexical over feed/seed/user-object candidates, and recommendations are domain-adapter fan-out. The only concept-graph code in the repo is the retired libs/oshun/search-discovery/src/concept-graph/ (schema, queries, curation) — its candidate generator does not feed the live paths. The BFF's Sophia integration (apps/oshun/bff/src/adapters/ sophia-read-adapters.ts) is an evidence read adapter over @oshun/evidence-sophia — grounding, review, and admin read roles for claims and notebooks — not a concept-graph substrate for search. See Sophia — Grounding Substrate.

When the Nisaba concept-graph / Metis knowledge-graph and Sophia-evaluated promotion become live discovery inputs, the adoption path is the one the retired library's banner names: search candidates start carrying real DiscoveryObject metadata (signal collection and catalog enrichment). At that point, the signals / aggregation / candidates / ranker modules wire into the live routes, and this section can drop the "spec, not live path" caveat.

The retired @oshun/search-discovery library (V1.x target)#

For completeness, the unadopted-but-tested modules (libs/oshun/search-discovery/src/):

Treat everything in that table except evals as planned, not shipped. The V1/features.md sections on the Searchable Object Catalog (per-class lexical+embedding+facet indices), Ranker Features (persona/evidence/grounding/entitlement scoring), and Online Experimentation describe this library, not the running BFF.

Edge Cases and Rationale#

• Why lexical and not embeddings? The honesty rule. An embedding/feature ranker needs DiscoveryObject signals (persona fit, evidence integrity, grounding state) that the live feed/seed/user-object candidates simply do not carry. Shipping the rich ranker would mean fabricating those inputs, so V1 ships the ranker whose every input is real.
• Empty query is valid. scoreLexical returns a flat 1, so an empty query surfaces a kind/intent-ordered browse list (continue items first) rather than erroring or returning nothing.
• Authorization shapes the candidate pool, not just the response. Unscoped domains never enter the fan-out, so a member without Veritas access never sees a Veritas seed, feed item, or recommendation — and search over their own objects skips other members' data entirely (the user-object stores are owner-scoped).
• Partial failure is first-class. One domain timing out degrades that domain's slice (domainStatus: 'degraded', partialFailure: true) without blanking the page — the resilience pattern shared by feed, search, and recommendations.
• Caching respects identity and freshness. Search caches per user, query, filter, and page (20 s); recommendations cache per user, profile revision, and page (30 s), so a preference change invalidates immediately while repeated scrolls stay cheap.
• Honest staleness. Per the V1 completeness audit, the search-explore-deep-read-library-save journey is partial: the web page.tsx is thin (≈901 bytes) and the rich object-class universal search the docs list is served by the lexical BFF route, not a dedicated per-class index.

Backlog and product scope: search/discovery/recommendations/knowledge-graph is tracked under § 15 in ../TODOS.md, with product detail in V1/features.md § Search, Discovery, Recommendations, and Knowledge Graph. Search and recommendations consume member objects from Customer Curation (notebooks, collections, saved-searches, saved items, habits).

Related#

• Customer Curation
• Customer-Facing Domains
• High-Level Architecture
• Sophia — Grounding Substrate
• Observability, Design System, Testing, and Performance
• Subsystem Glossary
• V1/features.md § Search, Discovery, Recommendations, and Knowledge Graph
• Hub: V1 Architecture