This document provides a comprehensive inventory of all packages, services, and
applications in the Yaa codebase, which will be integrated into the Oshun
monorepo as the Veritas domain.
Domain Assignment: Yaa → Veritas (named after the Egyptian goddess of
writing, wisdom, knowledge, libraries, and record-keeping)
Library prefix: @veritas/* App prefix: veritas-*
Table of Contents#
Overview
Technology Stack
Package Directory
Applications
Services
Cross-Package Dependencies
External API Integrations
Database Schema Areas
Infrastructure
Summary Statistics
Overview#
Yaa is an autonomous AI-powered news agency for Ghana featuring:
Multi-source news aggregation and original journalism
AI-powered content generation (text, audio, video)
Multi-language support (English, Twi, Ewe, Ga, Hausa, Dagbani)
Multimedia production (AI video anchors, podcasts, TikTok/Shorts)
Fact-checking and bias detection
Mobile-first progressive web app
The codebase is a pnpm monorepo containing 95+ packages across:
Packages : 85+ individual TypeScript/Node.js modules in /packages/
Apps : 1 Next.js web application in /apps/web/
Services : Python microservices in /services/nlp/
Infrastructure : Docker, Terraform, database schemas
Technology Stack#
Primary Languages#
Language
Usage
TypeScript
All packages (strict mode)
Python
NLP microservices (spaCy)
Runtime & Build#
Tool
Version
Purpose
Node.js
20+
JavaScript runtime
pnpm
10.0.0
Package manager
tsc
Latest
TypeScript compilation
esbuild
Latest
Fast bundling
swc
Latest
Fast compilation
Vitest
Latest
Testing framework
Frameworks#
Framework
Version
Usage
Next.js
16.1+
Web application (React 19)
Fastify
4.29+
Backend API
FastAPI
Latest
Python NLP services
BullMQ
Latest
Job queue orchestration
Databases & Storage#
Technology
Purpose
PostgreSQL
Primary database
pgvector
Vector embeddings
Elasticsearch
Full-text search
Redis
Caching and job queues
AWS S3/MinIO
Object storage
AI/ML#
Service
Purpose
Anthropic Claude
LLM orchestration
HeyGen
AI video avatars
ElevenLabs
Text-to-speech
Ghana NLP
Local language processing
spaCy
Named entity recognition
Package Directory#
1. AI Package#
Location: /packages/ai/ Sub-packages: 22 Purpose: AI/ML
orchestration, content generation, bias detection, fact-checking
Dependencies#
json Copy
{
"@anthropic-ai/sdk": "LLM integration",
"ioredis": "Redis caching",
"@sentry/node": "Error tracking",
"uuid": "Unique identifiers",
"zod": "Schema validation"
}
Sub-packages#
Package
Purpose
Key Dependencies
ai-agents
Autonomous AI agent framework (journalist, editor, fact-checker, strategist roles)
@anthropic-ai/sdk, ioredis, zod
original-content
AI-powered original journalism generation
date-fns, @anthropic-ai/sdk
market-summary-generator
Daily market summary synthesis
NLP services
weather-report-generator
Weather updates from sources
NLP services
story-clustering-worker
Groups semantically similar stories
Embedding service, PostgreSQL
article-summarization-worker
Abstractive summarization of articles
Summarization service
headline-variant-generator
Creates headline variations for A/B testing
LLM-based
article-tagging-worker
Auto-tags articles with topics/categories
NLP services
trend-analysis-generator
Identifies trending topics
Embedding services
political-bias-trainer
Trains bias detection models for Ghana context
ML training pipeline
source-accuracy-worker
Tracks source reliability metrics
PostgreSQL
blindspot-detection
Identifies coverage gaps (inspired by Ground News)
Embedding analysis
fact-check-integrations
Integrates fact-check data from external APIs
API integrations
claim-linking-worker
Links claims to fact-check results
NLP + fact-check data
breaking-news-draft-generator
Generates breaking news copy
LLM-based
gpl-score-generator
Ghana Political Lean scoring
Ghana context bias analysis
ecg-load-shedding-tracker
Tracks ECG load-shedding updates
Web scraping
fuel-price-update-generator
Tracks fuel price changes
Data scraping
traffic-update-generator
Generates traffic reports
Mapping/GPS data
event-calendar-updater
Maintains event calendar
Calendar data sources
press-release-summarizer
Processes press releases
Summarization
AI Agents Internal Structure#
text Copy
ai-agents/
├── agents/
│ ├── journalist/
│ ├── editor-in-chief/
│ ├── fact-checker/
│ ├── devops/
│ ├── content-strategist/
│ └── managing-editor/
├── core/ # Core orchestration logic
├── communication/ # Agent-to-agent messaging
├── orchestration/ # Workflow coordination
├── state/ # State management
├── deployment/ # Deployment utilities
├── monitoring/ # Agent health monitoring
├── logging/ # Structured logging
└── metrics/ # Performance metrics
2. Ingestion Package#
Location: /packages/ingestion/ Sub-packages: 12 Purpose: Content
ingestion, deduplication, normalization, source discovery
Sub-packages#
Package
Purpose
Dependencies
rss-collector
Collects articles from RSS feeds
rss-parser, PostgreSQL
sitemap-collector
Discovers content from sitemaps
Web crawler
scraper
General web scraping
cheerio
social-x
Monitors X/Twitter for news
X API integration
processing-queue
Main ingestion workflow orchestrator
BullMQ, Redis
semantic-dedup
Removes duplicate stories using embeddings
PostgreSQL, pgvector
language-detection
Auto-detects article language
Language detection lib
content-normalization
Standardizes article structure
Content parsers
media-extraction
Extracts images/videos from articles
Image extraction libs
timestamp-normalization
Standardizes publish/update times
date-fns
source-attribution
Tracks original sources
Source database
source-discovery
Finds new news sources
Crawling + patterns
monitoring-dashboard
Ingestion system monitoring UI
Dashboard framework
Processing Pipeline Flow#
text Copy
RSS/Scraper → Processing Queue → Language Detection →
Content Normalization → Media Extraction → Timestamp Normalization →
Semantic Dedup → Source Attribution → Database
3. NLP Package#
Location: /packages/nlp/ Sub-packages: 13 Purpose: Natural
Language Processing, embeddings, sentiment analysis, entity recognition
Sub-packages#
Package
Purpose
Dependencies
service-kit
Shared NLP utilities & base classes
Foundation (no deps)
ghana-nlp
Ghana NLP API client (Twi, Ewe, Ga, Hausa, Dagbani)
service-kit
embedding-service
Generates text embeddings
service-kit
similarity-service
Computes semantic similarity
service-kit
sentiment-service
Sentiment analysis
service-kit
topic-classification-service
Classifies articles by topic
service-kit
keyword-extraction-service
Extracts key phrases
service-kit
evidence-retrieval-service
Retrieves supporting evidence
service-kit
extractive-summarization-service
Extracts key sentences
service-kit
abstractive-summarization-service
Generates summaries
service-kit
claim-extraction-service
Extracts factual claims
service-kit
language-service
Language identification
service-kit
khaya-client
Integration with Ghana NLP Khaya API
service-kit
Ghana Language Support#
Language
Code
Notes
English
en
Primary language
Twi
tw
Most widely spoken local language
Ewe
ee
Volta Region
Ga
gaa
Greater Accra
Hausa
ha
Northern Ghana
Dagbani
dag
Northern Ghana
4. Video Package#
Location: /packages/video/ Sub-packages: 21 Purpose: Video
generation, processing, platform-specific optimization
Sub-packages#
Package
Purpose
Dependencies
article-to-video-service
Converts articles to video
AI agents, Ghana NLP
heygen-service
HeyGen API integration (AI avatars)
service-kit
avatar-profiles
Manages AI avatar configurations
Database
video-rendering-service
Video composition & rendering
service-kit
video-captioning-service
Adds captions to videos
Caption generation
video-vertical-converter
Converts to vertical format
Video processing
video-thumbnail-generator
Creates thumbnails
Image processing
video-generation-queue
Orchestrates video pipeline
BullMQ, Redis
video-hls-service
HLS streaming conversion
Streaming libs
video-bulletin-scheduler
Schedules video publishing
Database
tiktok-video-producer
TikTok-optimized videos
Captioning, vertical
youtube-shorts-upload-service
Uploads to YouTube Shorts
YouTube API
youtube-upload-service
Uploads full videos to YouTube
YouTube API
youtube-shorts-scheduler
Schedules Shorts uploads
Database
instagram-reels-producer
Instagram Reels creation
Reels specs
instagram-stories-producer
Instagram Stories creation
Story specs
instagram-content-templates
Reusable content templates
Design system
tiktok-sound-trends
Tracks trending TikTok sounds
TikTok API
tiktok-scheduler
Schedules TikTok uploads
TikTok API
tiktok-analytics
TikTok performance metrics
TikTok Analytics
Video Production Pipeline#
text Copy
Article → Article-to-Video → Avatar Selection (HeyGen) →
Video Rendering → Captioning → Platform Optimization
(TikTok/Shorts/Reels/YouTube) → Scheduled Publishing
5. Audio Package#
Location: /packages/audio/ Sub-packages: 5 Purpose: Audio
synthesis, podcast generation, local language TTS
Sub-packages#
Package
Purpose
Dependencies
article-to-audio-service
Converts articles to audio
service-kit, AWS S3
elevenlabs-service
ElevenLabs TTS integration
service-kit
ghana-nlp-tts
Local language TTS
service-kit, Ghana NLP
audio-generation-queue
Orchestrates audio pipeline
BullMQ, Redis
voice-profiles
Voice configuration management
Database
6. Social Package#
Location: /packages/social/ Sub-packages: 15 Purpose: Social media
automation, platform-specific scheduling and analytics
Sub-packages#
Package
Purpose
Dependencies
unified-scheduler
Central scheduling hub
BullMQ
instagram-scheduler
Instagram posting automation
BullMQ
instagram-business-setup
Instagram Business Account setup
Instagram API
tiktok-business-setup
TikTok Business Account setup
TikTok API
youtube-channel-setup
YouTube channel configuration
YouTube API
x-account-setup
X/Twitter account setup
X API
linkedin-company-page-setup
LinkedIn Company Page setup
LinkedIn API
facebook-page-setup
Facebook Page setup
Facebook API
telegram-channel-setup
Telegram channel creation
Telegram Bot API
whatsapp-business-service
WhatsApp Business API integration
WhatsApp API
whatsapp-channel-poster
WhatsApp channel posting
WhatsApp API
content-calendar
Editorial calendar management
Database
posting-optimizer
Best time to post analysis
Analytics data
analytics-dashboard
Social media metrics aggregation
Platform APIs
buffer-integration
Buffer API integration (fallback)
Buffer API
youtube-branding-assets
YouTube branding templates
Design assets
7. Web Package#
Location: /packages/web/ Purpose: Frontend PWA for news consumption
Technology Stack#
json Copy
{
"next": "16.1.2",
"react": "19.2.3",
"react-dom": "19.2.3",
"@auth/core": "Authentication",
"firebase": "Backend services",
"firebase-admin": "Server-side Firebase",
"@reduxjs/toolkit": "State management",
"tailwindcss": "Styling",
"next-intl": "Internationalization",
"next-themes": "Dark mode"
}
Features#
Multi-language support (Twi, Ewe, Ga, Hausa, Dagbani, English)
Progressive Web App (PWA) capabilities
Firebase authentication
Redux state management
Responsive design (mobile-first)
Dark mode support
Markdown content rendering
Directory Structure#
text Copy
/src/
├── app/ # Next.js App Router pages
├── components/ # React components
├── styles/ # Global styles
└── utils/ # Utilities
8. API Package#
Location: /packages/api/ Purpose: Central REST API backend
Technology Stack#
json Copy
{
"fastify": "4.29.1",
"@fastify/cors": "CORS support",
"@fastify/swagger": "API documentation",
"@elastic/elasticsearch": "Search integration",
"pg": "PostgreSQL client",
"ioredis": "Redis caching",
"bcryptjs": "Password hashing",
"zod": "Schema validation"
}
Features#
RESTful endpoints
OpenAPI/Swagger documentation
CORS support
Search indexing (Elasticsearch)
User authentication
Integration with AI agents and NLP services
Internal Dependencies#
json Copy
{
"@yaa/ai-agents": "AI orchestration",
"@yaa/nlp-evidence-retrieval-service": "Evidence retrieval",
"@yaa/nlp-topic-classification-service": "Topic classification"
}
Modes#
Server mode : HTTP endpoint (default port 3001)
Indexer mode : Batch search indexing
9. Models Package#
Location: /packages/models/ Purpose: Shared TypeScript
types/interfaces for entire platform
Exports#
Export
Description
common
Timestamped, Uuid, UrlString types
language
LanguageTag type (en, tw, ee, ha, etc.)
source
News source metadata
author
Article author information
taxonomy
Categories, Topics, Tags
entity
Named entities (PERSON, ORG, GPE)
media
Image/video/audio assets
story-cluster
Grouped related stories
fact-check
Fact-checking status and claims
bias
Bias scoring (political, gender, regional)
ghana-political-spectrum
Ghana-specific political classification
article
Complete article model with all metadata
Core Article Model#
typescript Copy
interface Article {
// Identity
id: Uuid;
slug: string;
// Content
title: string;
content: string;
summary: string;
language: LanguageTag;
// Status
status: 'draft' | 'published' | 'scheduled' | 'archived';
// Attribution
source: Source;
author?: Author;
perspectives: Perspective[];
// AI Generation
aiGenerated: boolean;
humanReviewed: boolean;
// Fact-checking
factCheckClaims: Claim[];
factCheckStatus: FactCheckStatus;
// Bias
biasScores: BiasScores;
ghPoliticalLean?: GhanaPoliticalSpectrum;
// Media
featuredImage?: Media;
gallery?: Media[];
// Relationships
relatedArticles: ArticleReference[];
storyCluster?: StoryClusterReference;
// Translations
translations: Record<LanguageTag, TranslatedContent>;
// Metrics
views: number;
shares: number;
// Timestamps
publishedAt: Date;
updatedAt: Date;
createdAt: Date;
}
No External Dependencies (pure TypeScript types)
Applications#
Web App#
Location: /apps/web/ Purpose: Main editorial interface and content
consumption platform
Aspect
Details
Framework
Next.js 14+ (App Router)
Language
TypeScript
Styling
Tailwind CSS
State
Redux Toolkit
Auth
Firebase
i18n
next-intl
Services#
NLP NER Service#
Location: /services/nlp/ner-service/ Purpose: Named Entity Recognition
microservice using Python/spaCy
Technology Stack#
Tool
Version
Purpose
Python
3.12+
Runtime
FastAPI
Latest
HTTP framework
uvicorn
Latest
ASGI server
spaCy
Latest
NLP library
API Endpoints#
Method
Endpoint
Description
GET
/healthz
Health check
GET
/readyz
Readiness with model availability
GET
/version
Runtime and model metadata
POST
/v1/ner
Extract entities from text
POST
/v1/ner/batch
Batch entity extraction
Ghana Entity Dictionary#
Custom entity patterns for:
Politicians (current and historical)
Traditional chiefs and queen mothers
Places (cities, towns, regions)
Organizations (government bodies, companies)
Cultural/religious entities
Environment Configuration#
Variable
Default
Description
SPACY_MODEL
en_core_web_sm
spaCy model name
YAA_ENABLE_GHANA_ENTITY_RULER
true
Enable Ghana patterns
YAA_GHANA_ENTITY_DICTIONARY_PATH
./app/ghana_entity_dictionary.json
Custom entities
Cross-Package Dependencies#
Dependency Architecture#
text Copy
┌─────────────────────────────────────────────────────┐
│ INGESTION LAYER │
│ rss-collector, scraper, social-x │
│ → processing-queue (orchestrates all) │
└─────────────────────┬───────────────────────────────┘
▼
┌─────────────────────────────────────────────────────┐
│ NORMALIZATION LAYER │
│ language-detection, content-normalization │
│ timestamp-normalization, media-extraction │
│ semantic-dedup, source-attribution │
└─────────────────────┬───────────────────────────────┘
▼
┌─────────────────────────────────────────────────────┐
│ AI/NLP ANALYSIS LAYER │
│ NLP Services (embeddings, sentiment) │
│ AI Agents (orchestration) │
│ story-clustering, summarization, tagging │
│ bias detection, fact-check integrations │
└─────────────────────┬───────────────────────────────┘
▼
┌─────────────────────────────────────────────────────┐
│ CONTENT GENERATION LAYER │
│ original-content generator │
│ headline-variant-generator │
│ specialized generators (weather, market, etc) │
└─────────────────────┬───────────────────────────────┘
▼
┌─────────────────────────────────────────────────────┐
│ MULTIMEDIA PRODUCTION LAYER │
│ Video: article-to-video, HeyGen, rendering │
│ Audio: article-to-audio, ElevenLabs, Ghana NLP │
│ Social: platform-specific producers │
└─────────────────────┬───────────────────────────────┘
▼
┌─────────────────────────────────────────────────────┐
│ PUBLISHING & DISTRIBUTION │
│ social/unified-scheduler │
│ Platform schedulers (TikTok, Instagram, YouTube) │
│ API (REST endpoints) │
│ Web (frontend PWA) │
└─────────────────────────────────────────────────────┘
Key Integration Points#
Integration
Description
service-kit
Foundation for all NLP services
AI Agents
Central orchestration of journalism workflow
Models
Type contracts shared across all packages
Processing Queue
Single ingestion hub for all sources
API
Unified REST interface to all services
PostgreSQL
Shared database across services
Redis + BullMQ
Distributed task queues
External API Integrations#
AI/ML Services#
Service
Package
Purpose
Anthropic Claude
ai-agents, original-content
LLM orchestration
HeyGen
video/heygen-service
AI video avatars
ElevenLabs
audio/elevenlabs-service
TTS (English + accents)
Ghana NLP (Khaya)
nlp/ghana-nlp, nlp/khaya-client
Local language TTS, NER
Platform
Packages
Purpose
YouTube
social/youtube-, video/youtube-
Video hosting, Shorts
TikTok
social/tiktok-, video/tiktok-
Short-form video
Instagram
social/instagram-, video/instagram-
Reels, Stories, posts
X/Twitter
ingestion/social-x, social/x-account-setup
News monitoring, posting
Facebook
social/facebook-page-setup
Page management
WhatsApp
social/whatsapp-*
Messaging, channels
Telegram
social/telegram-channel-setup
Channel management
LinkedIn
social/linkedin-company-page-setup
Company updates
Infrastructure Services#
Service
Package
Purpose
Elasticsearch
api/search-indexer
Full-text search
Firebase
web, api
Authentication, real-time DB
Buffer
social/buffer-integration
Fallback scheduling
spaCy
services/nlp/ner-service
English NER
Database Schema Areas#
Schema Area
Packages Using
Purpose
Articles
ingestion/, ai/ , api
Core article storage
Story Clusters
ai/story-clustering
Related article grouping
Sources
ingestion/*, api
News source registry
Authors
ingestion/*, api
Article author data
Entities
nlp/*, api
Extracted named entities
Embeddings
nlp/embedding-service
Vector storage (pgvector)
Bias Scores
ai/political-bias-*
Article bias metrics
Fact-Check Claims
ai/fact-check-*
Fact-check integration
Media Assets
ingestion/media-extraction
Image/video/audio URLs
Social Scheduling
social/*
Scheduled posts
Users
api, web
Editorial users, subscribers
Translations
nlp/*, web
Multi-language content
Infrastructure#
Docker Support#
Location: /infrastructure/docker/
docker-compose for local development
Observability stack (Grafana dashboards)
Service containers
Infrastructure as Code#
Location: /infrastructure/terraform/
AWS/GCP deployment configurations
Network and security settings
Scaling configurations
Key Files#
File
Purpose
.env.example
Environment configuration template
pnpm-workspace.yaml
Monorepo workspace configuration
package.json
Root package configuration
eslint.config.js
Linting configuration
Summary Statistics#
Metric
Count
Total Packages
95+
AI Sub-packages
22
Ingestion Sub-packages
12
NLP Sub-packages
13
Video Sub-packages
21
Audio Sub-packages
5
Social Sub-packages
15
Web Packages
1
API Packages
1
Model Exports
13 major types
Python Services
1 (NER)
Applications
1 (web)
External API Integrations
15+
Database Schema Areas
12+
Key Documentation#
Document
Location
Description
PRD
/yaa/PRD.md
5,600+ line product requirements
TODOS
/yaa/TODOS.md
1,047 implementation tasks
CLAUDE Guidelines
/yaa/CLAUDE.md
Development standards
AGENTS Guide
/yaa/AGENTS.md
Agent framework documentation
Architecture
/yaa/packages/ai/agents/ARCHITECTURE.md
Agent system architecture
Ghana Entities
/yaa/services/nlp/ner-service/app/ghana_entity_dictionary.json
Custom NER patterns
Integration Plan
/yaa/PHASE_23_SESHAT_INTEGRATION.md
Oshun integration plan
Quality Standards#
100% TypeScript (strict mode) for all Node.js packages
Vitest for testing across all packages
ESLint + TypeScript for linting
Node 20+ requirement
pnpm workspaces for monorepo management
Conventional commits for version control
Last Updated: January 2026
This document is part of the Veritas domain integration into the Oshun
monorepo.