Eve library

Veritas (Yaa) Package Inventory

Guide

Revision 1

On this page

This document provides a comprehensive inventory of all packages, services, and applications in the Yaa codebase, which will be integrated into the Oshun monorepo as the Veritas domain.

Domain Assignment: Yaa → Veritas (named after the Egyptian goddess of writing, wisdom, knowledge, libraries, and record-keeping)

Library prefix: @veritas/* App prefix: veritas-*


Table of Contents#

  1. Overview
  2. Technology Stack
  3. Package Directory
  4. Applications
  5. Services
  6. Cross-Package Dependencies
  7. External API Integrations
  8. Database Schema Areas
  9. Infrastructure
  10. Summary Statistics

Overview#

Yaa is an autonomous AI-powered news agency for Ghana featuring:

  • Multi-source news aggregation and original journalism
  • AI-powered content generation (text, audio, video)
  • Multi-language support (English, Twi, Ewe, Ga, Hausa, Dagbani)
  • Multimedia production (AI video anchors, podcasts, TikTok/Shorts)
  • Fact-checking and bias detection
  • Mobile-first progressive web app

The codebase is a pnpm monorepo containing 95+ packages across:

  • Packages: 85+ individual TypeScript/Node.js modules in /packages/
  • Apps: 1 Next.js web application in /apps/web/
  • Services: Python microservices in /services/nlp/
  • Infrastructure: Docker, Terraform, database schemas

Technology Stack#

Primary Languages#

Language Usage
TypeScript All packages (strict mode)
Python NLP microservices (spaCy)

Runtime & Build#

Tool Version Purpose
Node.js 20+ JavaScript runtime
pnpm 10.0.0 Package manager
tsc Latest TypeScript compilation
esbuild Latest Fast bundling
swc Latest Fast compilation
Vitest Latest Testing framework

Frameworks#

Framework Version Usage
Next.js 16.1+ Web application (React 19)
Fastify 4.29+ Backend API
FastAPI Latest Python NLP services
BullMQ Latest Job queue orchestration

Databases & Storage#

Technology Purpose
PostgreSQL Primary database
pgvector Vector embeddings
Elasticsearch Full-text search
Redis Caching and job queues
AWS S3/MinIO Object storage

AI/ML#

Service Purpose
Anthropic Claude LLM orchestration
HeyGen AI video avatars
ElevenLabs Text-to-speech
Ghana NLP Local language processing
spaCy Named entity recognition

Package Directory#

1. AI Package#

Location: /packages/ai/ Sub-packages: 22 Purpose: AI/ML orchestration, content generation, bias detection, fact-checking

Dependencies#

json
{
  "@anthropic-ai/sdk": "LLM integration",
  "ioredis": "Redis caching",
  "@sentry/node": "Error tracking",
  "uuid": "Unique identifiers",
  "zod": "Schema validation"
}

Sub-packages#

Package Purpose Key Dependencies
ai-agents Autonomous AI agent framework (journalist, editor, fact-checker, strategist roles) @anthropic-ai/sdk, ioredis, zod
original-content AI-powered original journalism generation date-fns, @anthropic-ai/sdk
market-summary-generator Daily market summary synthesis NLP services
weather-report-generator Weather updates from sources NLP services
story-clustering-worker Groups semantically similar stories Embedding service, PostgreSQL
article-summarization-worker Abstractive summarization of articles Summarization service
headline-variant-generator Creates headline variations for A/B testing LLM-based
article-tagging-worker Auto-tags articles with topics/categories NLP services
trend-analysis-generator Identifies trending topics Embedding services
political-bias-trainer Trains bias detection models for Ghana context ML training pipeline
source-accuracy-worker Tracks source reliability metrics PostgreSQL
blindspot-detection Identifies coverage gaps (inspired by Ground News) Embedding analysis
fact-check-integrations Integrates fact-check data from external APIs API integrations
claim-linking-worker Links claims to fact-check results NLP + fact-check data
breaking-news-draft-generator Generates breaking news copy LLM-based
gpl-score-generator Ghana Political Lean scoring Ghana context bias analysis
ecg-load-shedding-tracker Tracks ECG load-shedding updates Web scraping
fuel-price-update-generator Tracks fuel price changes Data scraping
traffic-update-generator Generates traffic reports Mapping/GPS data
event-calendar-updater Maintains event calendar Calendar data sources
press-release-summarizer Processes press releases Summarization

AI Agents Internal Structure#

text
ai-agents/
├── agents/
│   ├── journalist/
│   ├── editor-in-chief/
│   ├── fact-checker/
│   ├── devops/
│   ├── content-strategist/
│   └── managing-editor/
├── core/              # Core orchestration logic
├── communication/     # Agent-to-agent messaging
├── orchestration/     # Workflow coordination
├── state/             # State management
├── deployment/        # Deployment utilities
├── monitoring/        # Agent health monitoring
├── logging/           # Structured logging
└── metrics/           # Performance metrics

2. Ingestion Package#

Location: /packages/ingestion/ Sub-packages: 12 Purpose: Content ingestion, deduplication, normalization, source discovery

Sub-packages#

Package Purpose Dependencies
rss-collector Collects articles from RSS feeds rss-parser, PostgreSQL
sitemap-collector Discovers content from sitemaps Web crawler
scraper General web scraping cheerio
social-x Monitors X/Twitter for news X API integration
processing-queue Main ingestion workflow orchestrator BullMQ, Redis
semantic-dedup Removes duplicate stories using embeddings PostgreSQL, pgvector
language-detection Auto-detects article language Language detection lib
content-normalization Standardizes article structure Content parsers
media-extraction Extracts images/videos from articles Image extraction libs
timestamp-normalization Standardizes publish/update times date-fns
source-attribution Tracks original sources Source database
source-discovery Finds new news sources Crawling + patterns
monitoring-dashboard Ingestion system monitoring UI Dashboard framework

Processing Pipeline Flow#

text
RSS/Scraper → Processing Queue → Language Detection →
Content Normalization → Media Extraction → Timestamp Normalization →
Semantic Dedup → Source Attribution → Database

3. NLP Package#

Location: /packages/nlp/ Sub-packages: 13 Purpose: Natural Language Processing, embeddings, sentiment analysis, entity recognition

Sub-packages#

Package Purpose Dependencies
service-kit Shared NLP utilities & base classes Foundation (no deps)
ghana-nlp Ghana NLP API client (Twi, Ewe, Ga, Hausa, Dagbani) service-kit
embedding-service Generates text embeddings service-kit
similarity-service Computes semantic similarity service-kit
sentiment-service Sentiment analysis service-kit
topic-classification-service Classifies articles by topic service-kit
keyword-extraction-service Extracts key phrases service-kit
evidence-retrieval-service Retrieves supporting evidence service-kit
extractive-summarization-service Extracts key sentences service-kit
abstractive-summarization-service Generates summaries service-kit
claim-extraction-service Extracts factual claims service-kit
language-service Language identification service-kit
khaya-client Integration with Ghana NLP Khaya API service-kit

Ghana Language Support#

Language Code Notes
English en Primary language
Twi tw Most widely spoken local language
Ewe ee Volta Region
Ga gaa Greater Accra
Hausa ha Northern Ghana
Dagbani dag Northern Ghana

4. Video Package#

Location: /packages/video/ Sub-packages: 21 Purpose: Video generation, processing, platform-specific optimization

Sub-packages#

Package Purpose Dependencies
article-to-video-service Converts articles to video AI agents, Ghana NLP
heygen-service HeyGen API integration (AI avatars) service-kit
avatar-profiles Manages AI avatar configurations Database
video-rendering-service Video composition & rendering service-kit
video-captioning-service Adds captions to videos Caption generation
video-vertical-converter Converts to vertical format Video processing
video-thumbnail-generator Creates thumbnails Image processing
video-generation-queue Orchestrates video pipeline BullMQ, Redis
video-hls-service HLS streaming conversion Streaming libs
video-bulletin-scheduler Schedules video publishing Database
tiktok-video-producer TikTok-optimized videos Captioning, vertical
youtube-shorts-upload-service Uploads to YouTube Shorts YouTube API
youtube-upload-service Uploads full videos to YouTube YouTube API
youtube-shorts-scheduler Schedules Shorts uploads Database
instagram-reels-producer Instagram Reels creation Reels specs
instagram-stories-producer Instagram Stories creation Story specs
instagram-content-templates Reusable content templates Design system
tiktok-sound-trends Tracks trending TikTok sounds TikTok API
tiktok-scheduler Schedules TikTok uploads TikTok API
tiktok-analytics TikTok performance metrics TikTok Analytics

Video Production Pipeline#

text
Article → Article-to-Video → Avatar Selection (HeyGen) →
Video Rendering → Captioning → Platform Optimization
(TikTok/Shorts/Reels/YouTube) → Scheduled Publishing

5. Audio Package#

Location: /packages/audio/ Sub-packages: 5 Purpose: Audio synthesis, podcast generation, local language TTS

Sub-packages#

Package Purpose Dependencies
article-to-audio-service Converts articles to audio service-kit, AWS S3
elevenlabs-service ElevenLabs TTS integration service-kit
ghana-nlp-tts Local language TTS service-kit, Ghana NLP
audio-generation-queue Orchestrates audio pipeline BullMQ, Redis
voice-profiles Voice configuration management Database

6. Social Package#

Location: /packages/social/ Sub-packages: 15 Purpose: Social media automation, platform-specific scheduling and analytics

Sub-packages#

Package Purpose Dependencies
unified-scheduler Central scheduling hub BullMQ
instagram-scheduler Instagram posting automation BullMQ
instagram-business-setup Instagram Business Account setup Instagram API
tiktok-business-setup TikTok Business Account setup TikTok API
youtube-channel-setup YouTube channel configuration YouTube API
x-account-setup X/Twitter account setup X API
linkedin-company-page-setup LinkedIn Company Page setup LinkedIn API
facebook-page-setup Facebook Page setup Facebook API
telegram-channel-setup Telegram channel creation Telegram Bot API
whatsapp-business-service WhatsApp Business API integration WhatsApp API
whatsapp-channel-poster WhatsApp channel posting WhatsApp API
content-calendar Editorial calendar management Database
posting-optimizer Best time to post analysis Analytics data
analytics-dashboard Social media metrics aggregation Platform APIs
buffer-integration Buffer API integration (fallback) Buffer API
youtube-branding-assets YouTube branding templates Design assets

7. Web Package#

Location: /packages/web/ Purpose: Frontend PWA for news consumption

Technology Stack#

json
{
  "next": "16.1.2",
  "react": "19.2.3",
  "react-dom": "19.2.3",
  "@auth/core": "Authentication",
  "firebase": "Backend services",
  "firebase-admin": "Server-side Firebase",
  "@reduxjs/toolkit": "State management",
  "tailwindcss": "Styling",
  "next-intl": "Internationalization",
  "next-themes": "Dark mode"
}

Features#

  • Multi-language support (Twi, Ewe, Ga, Hausa, Dagbani, English)
  • Progressive Web App (PWA) capabilities
  • Firebase authentication
  • Redux state management
  • Responsive design (mobile-first)
  • Dark mode support
  • Markdown content rendering

Directory Structure#

text
/src/
├── app/          # Next.js App Router pages
├── components/   # React components
├── styles/       # Global styles
└── utils/        # Utilities

8. API Package#

Location: /packages/api/ Purpose: Central REST API backend

Technology Stack#

json
{
  "fastify": "4.29.1",
  "@fastify/cors": "CORS support",
  "@fastify/swagger": "API documentation",
  "@elastic/elasticsearch": "Search integration",
  "pg": "PostgreSQL client",
  "ioredis": "Redis caching",
  "bcryptjs": "Password hashing",
  "zod": "Schema validation"
}

Features#

  • RESTful endpoints
  • OpenAPI/Swagger documentation
  • CORS support
  • Search indexing (Elasticsearch)
  • User authentication
  • Integration with AI agents and NLP services

Internal Dependencies#

json
{
  "@yaa/ai-agents": "AI orchestration",
  "@yaa/nlp-evidence-retrieval-service": "Evidence retrieval",
  "@yaa/nlp-topic-classification-service": "Topic classification"
}

Modes#

  • Server mode: HTTP endpoint (default port 3001)
  • Indexer mode: Batch search indexing

9. Models Package#

Location: /packages/models/ Purpose: Shared TypeScript types/interfaces for entire platform

Exports#

Export Description
common Timestamped, Uuid, UrlString types
language LanguageTag type (en, tw, ee, ha, etc.)
source News source metadata
author Article author information
taxonomy Categories, Topics, Tags
entity Named entities (PERSON, ORG, GPE)
media Image/video/audio assets
story-cluster Grouped related stories
fact-check Fact-checking status and claims
bias Bias scoring (political, gender, regional)
ghana-political-spectrum Ghana-specific political classification
article Complete article model with all metadata

Core Article Model#

typescript
interface Article {
  // Identity
  id: Uuid;
  slug: string;

  // Content
  title: string;
  content: string;
  summary: string;
  language: LanguageTag;

  // Status
  status: 'draft' | 'published' | 'scheduled' | 'archived';

  // Attribution
  source: Source;
  author?: Author;
  perspectives: Perspective[];

  // AI Generation
  aiGenerated: boolean;
  humanReviewed: boolean;

  // Fact-checking
  factCheckClaims: Claim[];
  factCheckStatus: FactCheckStatus;

  // Bias
  biasScores: BiasScores;
  ghPoliticalLean?: GhanaPoliticalSpectrum;

  // Media
  featuredImage?: Media;
  gallery?: Media[];

  // Relationships
  relatedArticles: ArticleReference[];
  storyCluster?: StoryClusterReference;

  // Translations
  translations: Record<LanguageTag, TranslatedContent>;

  // Metrics
  views: number;
  shares: number;

  // Timestamps
  publishedAt: Date;
  updatedAt: Date;
  createdAt: Date;
}

No External Dependencies (pure TypeScript types)


Applications#

Web App#

Location: /apps/web/ Purpose: Main editorial interface and content consumption platform

Aspect Details
Framework Next.js 14+ (App Router)
Language TypeScript
Styling Tailwind CSS
State Redux Toolkit
Auth Firebase
i18n next-intl

Services#

NLP NER Service#

Location: /services/nlp/ner-service/ Purpose: Named Entity Recognition microservice using Python/spaCy

Technology Stack#

Tool Version Purpose
Python 3.12+ Runtime
FastAPI Latest HTTP framework
uvicorn Latest ASGI server
spaCy Latest NLP library

API Endpoints#

Method Endpoint Description
GET /healthz Health check
GET /readyz Readiness with model availability
GET /version Runtime and model metadata
POST /v1/ner Extract entities from text
POST /v1/ner/batch Batch entity extraction

Ghana Entity Dictionary#

Custom entity patterns for:

  • Politicians (current and historical)
  • Traditional chiefs and queen mothers
  • Places (cities, towns, regions)
  • Organizations (government bodies, companies)
  • Cultural/religious entities

Environment Configuration#

Variable Default Description
SPACY_MODEL en_core_web_sm spaCy model name
YAA_ENABLE_GHANA_ENTITY_RULER true Enable Ghana patterns
YAA_GHANA_ENTITY_DICTIONARY_PATH ./app/ghana_entity_dictionary.json Custom entities

Cross-Package Dependencies#

Dependency Architecture#

text
┌─────────────────────────────────────────────────────┐
│               INGESTION LAYER                       │
│  rss-collector, scraper, social-x                   │
│  → processing-queue (orchestrates all)              │
└─────────────────────┬───────────────────────────────┘
                      ▼
┌─────────────────────────────────────────────────────┐
│            NORMALIZATION LAYER                      │
│  language-detection, content-normalization          │
│  timestamp-normalization, media-extraction          │
│  semantic-dedup, source-attribution                 │
└─────────────────────┬───────────────────────────────┘
                      ▼
┌─────────────────────────────────────────────────────┐
│            AI/NLP ANALYSIS LAYER                    │
│  NLP Services (embeddings, sentiment)               │
│  AI Agents (orchestration)                          │
│  story-clustering, summarization, tagging           │
│  bias detection, fact-check integrations            │
└─────────────────────┬───────────────────────────────┘
                      ▼
┌─────────────────────────────────────────────────────┐
│          CONTENT GENERATION LAYER                   │
│  original-content generator                         │
│  headline-variant-generator                         │
│  specialized generators (weather, market, etc)      │
└─────────────────────┬───────────────────────────────┘
                      ▼
┌─────────────────────────────────────────────────────┐
│        MULTIMEDIA PRODUCTION LAYER                  │
│  Video: article-to-video, HeyGen, rendering         │
│  Audio: article-to-audio, ElevenLabs, Ghana NLP     │
│  Social: platform-specific producers                │
└─────────────────────┬───────────────────────────────┘
                      ▼
┌─────────────────────────────────────────────────────┐
│         PUBLISHING & DISTRIBUTION                   │
│  social/unified-scheduler                           │
│  Platform schedulers (TikTok, Instagram, YouTube)   │
│  API (REST endpoints)                               │
│  Web (frontend PWA)                                 │
└─────────────────────────────────────────────────────┘

Key Integration Points#

Integration Description
service-kit Foundation for all NLP services
AI Agents Central orchestration of journalism workflow
Models Type contracts shared across all packages
Processing Queue Single ingestion hub for all sources
API Unified REST interface to all services
PostgreSQL Shared database across services
Redis + BullMQ Distributed task queues

External API Integrations#

AI/ML Services#

Service Package Purpose
Anthropic Claude ai-agents, original-content LLM orchestration
HeyGen video/heygen-service AI video avatars
ElevenLabs audio/elevenlabs-service TTS (English + accents)
Ghana NLP (Khaya) nlp/ghana-nlp, nlp/khaya-client Local language TTS, NER

Social Platforms#

Platform Packages Purpose
YouTube social/youtube-, video/youtube- Video hosting, Shorts
TikTok social/tiktok-, video/tiktok- Short-form video
Instagram social/instagram-, video/instagram- Reels, Stories, posts
X/Twitter ingestion/social-x, social/x-account-setup News monitoring, posting
Facebook social/facebook-page-setup Page management
WhatsApp social/whatsapp-* Messaging, channels
Telegram social/telegram-channel-setup Channel management
LinkedIn social/linkedin-company-page-setup Company updates

Infrastructure Services#

Service Package Purpose
Elasticsearch api/search-indexer Full-text search
Firebase web, api Authentication, real-time DB
Buffer social/buffer-integration Fallback scheduling
spaCy services/nlp/ner-service English NER

Database Schema Areas#

Schema Area Packages Using Purpose
Articles ingestion/, ai/, api Core article storage
Story Clusters ai/story-clustering Related article grouping
Sources ingestion/*, api News source registry
Authors ingestion/*, api Article author data
Entities nlp/*, api Extracted named entities
Embeddings nlp/embedding-service Vector storage (pgvector)
Bias Scores ai/political-bias-* Article bias metrics
Fact-Check Claims ai/fact-check-* Fact-check integration
Media Assets ingestion/media-extraction Image/video/audio URLs
Social Scheduling social/* Scheduled posts
Users api, web Editorial users, subscribers
Translations nlp/*, web Multi-language content

Infrastructure#

Docker Support#

Location: /infrastructure/docker/

  • docker-compose for local development
  • Observability stack (Grafana dashboards)
  • Service containers

Infrastructure as Code#

Location: /infrastructure/terraform/

  • AWS/GCP deployment configurations
  • Network and security settings
  • Scaling configurations

Key Files#

File Purpose
.env.example Environment configuration template
pnpm-workspace.yaml Monorepo workspace configuration
package.json Root package configuration
eslint.config.js Linting configuration

Summary Statistics#

Metric Count
Total Packages 95+
AI Sub-packages 22
Ingestion Sub-packages 12
NLP Sub-packages 13
Video Sub-packages 21
Audio Sub-packages 5
Social Sub-packages 15
Web Packages 1
API Packages 1
Model Exports 13 major types
Python Services 1 (NER)
Applications 1 (web)
External API Integrations 15+
Database Schema Areas 12+

Key Documentation#

Document Location Description
PRD /yaa/PRD.md 5,600+ line product requirements
TODOS /yaa/TODOS.md 1,047 implementation tasks
CLAUDE Guidelines /yaa/CLAUDE.md Development standards
AGENTS Guide /yaa/AGENTS.md Agent framework documentation
Architecture /yaa/packages/ai/agents/ARCHITECTURE.md Agent system architecture
Ghana Entities /yaa/services/nlp/ner-service/app/ghana_entity_dictionary.json Custom NER patterns
Integration Plan /yaa/PHASE_23_SESHAT_INTEGRATION.md Oshun integration plan

Quality Standards#

  • 100% TypeScript (strict mode) for all Node.js packages
  • Vitest for testing across all packages
  • ESLint + TypeScript for linting
  • Node 20+ requirement
  • pnpm workspaces for monorepo management
  • Conventional commits for version control

Last Updated: January 2026

This document is part of the Veritas domain integration into the Oshun monorepo.