This document provides comprehensive documentation of the Veritas platform
system architecture --- an AI-powered news agency focused on Ghana and West
Africa.
Overview#
Veritas is a next-generation news platform that combines automated content
ingestion, AI-driven editorial workflows, fact-checking, multilingual support
for Ghanaian languages, and multimedia production. It operates as three primary
services within the Oshun monorepo, backed by a rich library ecosystem of 40+
domain modules.
Architecture Type : Modular monolith with service-oriented decomposition
Total Services : 3 applications + 40+ library modules Deployment Model :
Kubernetes with GitOps (ArgoCD) / ECS on AWS Primary Stack : Node.js/Hono,
TypeScript, PostgreSQL, Redis, Elasticsearch
Key Statistics#
Metric
Value
API endpoints
80+ REST endpoints across 29 route files
Database tables
50+ tables across 63 migrations
External integrations
30+ third-party APIs
Supported languages
6 Ghanaian languages (English, Twi, Ewe, Ga, Hausa, Dagbani)
AI agent types
7 specialized newsroom agents
NLP services
13 analysis modules
Content sources
300+ RSS feeds from Ghanaian news outlets
Payment providers
4 (Stripe, MTN MoMo, Vodafone Cash, AirtelTigo)
System Context (C4 Level 1)#
text Copy
┌─────────────────────────────────────────────────────────────────────────────┐
│ EXTERNAL SYSTEMS │
├───────────────┬──────────────┬──────────────┬──────────────┬───────────────┤
│ AI/LLM │ Media │ Search & │ Payment │ Social │
│ Providers │ Production │ Fact-Check │ Gateways │ Platforms │
│ │ │ │ │ │
│ Anthropic │ HeyGen │ Serper │ Stripe │ YouTube │
│ OpenAI │ ElevenLabs │ Brave Search │ MTN MoMo │ TikTok │
│ Cohere │ Ghana NLP │ ClaimBuster │ Vodafone │ Instagram │
│ HuggingFace │ Shotstack │ Google FCT │ AirtelTigo │ WhatsApp │
└───────┬───────┴──────┬───────┴──────┬───────┴──────┬───────┴───────┬───────┘
│ │ │ │ │
▼ ▼ ▼ ▼ ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ │
│ VERITAS PLATFORM │
│ │
│ ┌───────────────────────────────────────────────────────────────────────┐ │
│ │ VERITAS API (Hono REST) │ │
│ │ Port 3001 -- 80+ endpoints, v1 prefix │ │
│ └─────────────────────────────┬─────────────────────────────────────────┘ │
│ │ │
│ ┌──────────────┬──────────────┴──────────────┬──────────────────────────┐ │
│ │ Ingestion │ Domain Logic │ Library Modules │ │
│ │ Service │ │ │ │
│ │ (Scheduler, │ Articles, Claims, Sources, │ AI Agents (7 types) │ │
│ │ Producer, │ Subscriptions, Payments, │ NLP Services (13) │ │
│ │ Worker) │ Compliance, Ads, B2B │ Video/Audio (25+) │ │
│ └──────┬───────┴──────────────┬──────────────┴──────────────┬──────────┘ │
│ │ │ │ │
│ ┌──────┴──────────────────────┴──────────────────────────────┴──────────┐ │
│ │ DATA LAYER │ │
│ │ PostgreSQL │ Redis │ Elasticsearch │ MinIO (S3) │ │
│ └───────────────────────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
▲ ▲ ▲ ▲ ▲
│ │ │ │ │
┌───────┴───────┬──────┴──────┬───────┴──────┬───────┴──────┬────────┴───────┐
│ Web PWA │ Mobile App │ WhatsApp │ B2B API │ RSS Readers │
│ (Next.js 15) │ (React │ Channel │ Partners │ & Aggregators │
│ Port 3000 │ Native) │ │ │ │
└───────────────┴─────────────┴──────────────┴──────────────┴────────────────┘
CLIENT APPLICATIONS
Container Diagram (C4 Level 2)#
Application Services#
text Copy
┌──────────────────────────────────────────────────────────────────────────────┐
│ APPLICATION LAYER │
│ │
│ ┌────────────────────────────────────────────────────────────────────────┐ │
│ │ VERITAS API (Port 3001) │ │
│ │ │ │
│ │ Framework: Hono Transport: HTTP/REST │ │
│ │ Auth: JWT (HMAC-SHA256) Rate Limiting: Redis sliding window │ │
│ │ Validation: Zod Documentation: OpenAPI 3.0.3 │ │
│ │ │ │
│ │ Middleware Chain: │ │
│ │ Logger -> Timing -> Security Headers -> CORS -> Rate Limit -> Auth │ │
│ │ │ │
│ │ Routes (29 files): │ │
│ │ /api/v1/articles /api/v1/claims /api/v1/subscriptions │ │
│ │ /api/v1/auth /api/v1/sources /api/v1/payments │ │
│ │ /api/v1/stories /api/v1/taxonomy /api/v1/preferences │ │
│ │ /api/v1/bias /api/v1/compliance /api/v1/election-coverage │ │
│ │ /api/v1/tips /api/v1/paywall /api/v1/reading-mode │ │
│ │ /api/v1/ads /api/v1/ad-networks /api/v1/b2b-api │ │
│ │ /api/v1/api-keys /api/v1/avatars /api/v1/voices │ │
│ │ /api/v1/writing-assistant /api/v1/data-protection │ │
│ │ /api/v1/subscription-analytics /api/v1/stripe-payments │ │
│ │ /api/v1/vodafone-payments /api/v1/airteltigo-payments │ │
│ └────────────────────────────────────────────────────────────────────────┘ │
│ │
│ ┌─────────────────────────────────┐ ┌──────────────────────────────────┐ │
│ │ VERITAS WEB (Port 3000) │ │ VERITAS INGESTION │ │
│ │ │ │ │ │
│ │ Framework: Next.js 15 │ │ Framework: Node.js + BullMQ │ │
│ │ Renderer: App Router (SSR) │ │ Modes: scheduler, producer, │ │
│ │ Styling: Tailwind CSS │ │ worker, seed │ │
│ │ i18n: next-intl (6 locales) │ │ Collectors: RSS, Sitemap │ │
│ │ PWA: Standalone + Shortcuts │ │ Pipeline: Normalize, Dedupe, │ │
│ │ Auth: NextAuth │ │ Language, Timestamp, Media │ │
│ │ │ │ Queue: veritas:content- │ │
│ │ Locales: │ │ processing │ │
│ │ en, tw, ee, ga, ha, dag │ │ Concurrency: 10 RSS fetches, │ │
│ │ │ │ 4 processing workers │ │
│ └─────────────────────────────────┘ └──────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────────────────────┘
Library Module Ecosystem#
text Copy
┌──────────────────────────────────────────────────────────────────────────────┐
│ LIBRARY MODULES (libs/veritas/) │
│ │
│ ┌──────────────────────┐ ┌───────────────────────┐ ┌──────────────────┐ │
│ │ CORE LIBRARIES │ │ AI AGENT FRAMEWORK │ │ NLP SERVICES │ │
│ │ │ │ │ │ │ │
│ │ core/ │ │ agents-orchestrator/ │ │ nlp-core/ │ │
│ │ database/ │ │ agents-core/ │ │ ghana-nlp/ │ │
│ │ cache/ │ │ agents-journalism/ │ │ (13 analysis │ │
│ │ auth/ │ │ agents-fact-checking/│ │ modules) │ │
│ │ storage/ │ │ agents-editorial/ │ │ │ │
│ │ events/ │ │ agents-product/ │ │ Sentiment │ │
│ │ models/ │ │ agents-devops/ │ │ Topics │ │
│ │ llm/ │ │ agents-qa/ │ │ Keywords │ │
│ │ │ │ agents-social-media/ │ │ Summarization │ │
│ └──────────────────────┘ └───────────────────────┘ │ Claims │ │
│ │ Language │ │
│ ┌──────────────────────┐ ┌───────────────────────┐ │ Embeddings │ │
│ │ CONTENT & ANALYSIS │ │ MEDIA PRODUCTION │ │ Similarity │ │
│ │ │ │ │ │ Evidence │ │
│ │ fact-checking/ │ │ video-production/ │ └──────────────────┘ │
│ │ bias-detection/ │ │ audio-production/ │ │
│ │ claims/ │ │ (25+ sub-modules │ ┌──────────────────┐ │
│ │ content-classific./ │ │ for HeyGen, │ │ MONETIZATION │ │
│ │ article-generation/ │ │ ElevenLabs, │ │ │ │
│ │ headline-service/ │ │ YouTube, TikTok, │ │ payments/ │ │
│ │ story-clustering/ │ │ Instagram) │ │ analytics/ │ │
│ │ knowledge-graph/ │ │ │ │ notifications/ │ │
│ │ rag/ │ │ │ │ social-automat./│ │
│ │ recommendations/ │ │ │ │ │ │
│ │ search/ │ │ │ │ │ │
│ └──────────────────────┘ └───────────────────────┘ └──────────────────┘ │
│ │
│ ┌──────────────────────┐ │
│ │ INGESTION CORE │ │
│ │ │ │
│ │ ingestion-core/ │ │
│ │ (RSS, scraping, │ │
│ │ normalization, │ │
│ │ deduplication) │ │
│ └──────────────────────┘ │
└──────────────────────────────────────────────────────────────────────────────┘
Data Layer#
text Copy
┌──────────────────────────────────────────────────────────────────────────────┐
│ DATA LAYER │
│ │
│ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────────────┐ │
│ │ PostgreSQL │ │ Redis │ │ Elasticsearch │ │
│ │ (Port 5432) │ │ (Port 6379) │ │ (Port 9200) │ │
│ │ │ │ │ │ │ │
│ │ 50+ tables │ │ Rate limiting │ │ Full-text search │ │
│ │ 63 migrations │ │ Session cache │ │ Article indexing │ │
│ │ │ │ BullMQ queues │ │ Autocomplete │ │
│ │ Articles │ │ Pub/Sub events │ │ Search analytics │ │
│ │ Sources │ │ API caching │ │ │ │
│ │ Users │ │ │ │ Index: │ │
│ │ Claims │ │ Key Patterns: │ │ veritas-articles │ │
│ │ Subscriptions │ │ veritas:api:* │ │ │ │
│ │ Taxonomy │ │ veritas:queue:* │ │ │ │
│ │ Feed Items │ │ veritas:cache:* │ │ │ │
│ │ Media Assets │ │ │ │ │ │
│ └──────────────────┘ └──────────────────┘ └──────────────────────────┘ │
│ │
│ ┌──────────────────────────────────────────────────────────────────────┐ │
│ │ MinIO / S3 (Port 9000) │ │
│ │ │ │
│ │ S3-compatible object storage for media assets │ │
│ │ - Article images and thumbnails │ │
│ │ - Generated audio files (TTS) │ │
│ │ - Generated video files (HeyGen, Shotstack) │ │
│ │ - Static assets for CDN distribution │ │
│ └──────────────────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────────────────────┘
Service Catalog#
Tier 1: Core Applications#
Service
Port
Framework
Responsibility
Veritas API
3001
Hono
REST API: articles, auth, search, fact-checking, payments, compliance
Veritas Web
3000
Next.js 15
PWA frontend: SSR, i18n, offline support, responsive UI
Veritas Ingestion
--
Node.js
Content pipeline: RSS collection, processing, deduplication
Tier 2: AI Agent Services#
Agent
Library Path
Responsibility
Editor-in-Chief
libs/veritas/agents-orchestrator/
Coordinates editorial workflow, content prioritization
Journalism Agent
libs/veritas/agents-journalism/
Article drafting, source verification, story development
Fact-Checking Agent
libs/veritas/agents-fact-checking/
Claim verification, evidence gathering, verdict assignment
Editorial Agent
libs/veritas/agents-editorial/
Copy editing, style enforcement, headline optimization
Product Agent
libs/veritas/agents-product/
UX decisions, feature prioritization, A/B testing
DevOps Agent
libs/veritas/agents-devops/
Infrastructure monitoring, deployment automation
QA Agent
libs/veritas/agents-qa/
Test generation, regression detection, quality gates
Social Media Agent
libs/veritas/agents-social-media/
Cross-platform publishing, engagement optimization
Tier 3: NLP Analysis Modules#
Module
Library Path / Package
Responsibility
Sentiment Analysis
nlp/sentiment-service
HuggingFace sentiment + Ghana-specific calibration
Topic Classification
nlp/topic-classification
Zero-shot topic assignment via HuggingFace
Keyword Extraction
nlp/keyword-extraction
RAKE algorithm for keyword identification
Extractive Summarization
nlp/extractive-summarization
TextRank with TF-IDF for article summaries
Abstractive Summarization
nlp/abstractive-summarization
LLM-powered summaries (Claude, GPT-4)
Claim Extraction
nlp/claim-extraction
Heuristic + LLM extraction of factual claims
Language Detection
nlp/language-service
Ghanaian language detection, Twi code-switching
Embedding Generation
nlp/embedding-service
OpenAI/Cohere vector embeddings
Semantic Similarity
nlp/similarity-service
Cosine similarity for content matching
Evidence Retrieval
nlp/evidence-retrieval
Web search via Serper/Brave for fact-checking
Ghana NLP
nlp/ghana-nlp
Translation, TTS, STT, NER for Ghanaian languages
Khaya Client
nlp/khaya-client
Khaya translation API for local languages
Service Kit
nlp/service-kit
Base HTTP primitives shared across NLP modules
Tier 4: Content Production Modules#
Module Category
Count
Key Modules
Video Production
21
HeyGen avatars, Shotstack rendering, YouTube/TikTok upload, captioning, thumbnails
Audio Production
5
ElevenLabs TTS, Ghana NLP TTS, audio generation queue
Social Distribution
15
WhatsApp, Instagram, TikTok, Facebook, X, Telegram, LinkedIn, YouTube
Ingestion
14
RSS collector, scraper, dedup, language detection, media extraction
Data Flow Patterns#
Content Ingestion Pipeline#
text Copy
┌─────────────────────────────────────────────────────────────────────────────┐
│ CONTENT INGESTION FLOW │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ RSS Feeds │ │ Sitemaps │ │ Social APIs │ │
│ │ (300+ feeds) │ │ │ │ (X/Twitter) │ │
│ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ │
│ │ │ │ │
│ └───────────────────┼───────────────────┘ │
│ ▼ │
│ ┌──────────────────┐ │
│ │ SCHEDULER │ │
│ │ │ │
│ │ Adaptive poll │ │
│ │ ETag/304 │ │
│ │ Rate limiting │ │
│ │ Backoff │ │
│ └────────┬─────────┘ │
│ │ │
│ ▼ │
│ ┌──────────────────┐ │
│ │ feed_items │ │
│ │ (PostgreSQL) │ │
│ └────────┬─────────┘ │
│ │ │
│ ▼ │
│ ┌──────────────────┐ │
│ │ PRODUCER │ │
│ │ │ │
│ │ Polls DB for │ │
│ │ unprocessed │ │
│ │ items (100/tick)│ │
│ └────────┬─────────┘ │
│ │ │
│ ▼ │
│ ┌──────────────────┐ │
│ │ BullMQ Queue │ │
│ │ veritas:content- │ │
│ │ processing │ │
│ │ (Redis) │ │
│ └────────┬─────────┘ │
│ │ │
│ ▼ │
│ ┌──────────────────────────────────────────────────────────────────────┐ │
│ │ PROCESSING WORKER (4 concurrent) │ │
│ │ │ │
│ │ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ │ │
│ │ │Normalize│─►│ Dedupe │─►│Language │─►│Timestamp│─►│ Media │ │ │
│ │ │ Content │ │(Simhash │ │ Detect │ │Normalize│ │ Extract │ │ │
│ │ │ │ │ 64-bit)│ │ │ │ │ │ │ │ │
│ │ └─────────┘ └─────────┘ └─────────┘ └─────────┘ └─────────┘ │ │
│ │ │ │
│ └──────────────────────────────────────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌──────────────────┐ │
│ │normalized_feed_ │ │
│ │items (PostgreSQL)│ │
│ └──────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
Article Delivery Flow#
text Copy
┌─────────────────────────────────────────────────────────────────────────────┐
│ ARTICLE DELIVERY FLOW │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌────────┐ HTTPS ┌──────────────────────────────────────────┐ │
│ │ Client │ ──────────►│ VERITAS API │ │
│ │ (PWA) │ │ │ │
│ └────────┘ │ ┌────────────────────────────────────┐ │ │
│ ▲ │ │ Middleware Chain │ │ │
│ │ │ │ Logger -> Security -> CORS -> │ │ │
│ │ │ │ Rate Limit -> API Key Auth │ │ │
│ │ │ └──────────────┬─────────────────────┘ │ │
│ │ │ │ │ │
│ │ │ ▼ │ │
│ │ │ ┌────────────────────────────────────┐ │ │
│ │ │ │ Route Handler │ │ │
│ │ │ │ │ │ │
│ │ │ │ Zod validation -> Business logic │ │ │
│ │ │ │ -> Cursor pagination -> Response │ │ │
│ │ │ └──────┬─────────────────┬───────────┘ │ │
│ │ │ │ │ │ │
│ │ │ ▼ ▼ │ │
│ │ │ ┌─────────────┐ ┌─────────────────┐ │ │
│ │ │ │ PostgreSQL │ │ Elasticsearch │ │ │
│ │ │ │ (articles, │ │ (full-text │ │ │
│ │ │ │ sources, │ │ search, auto- │ │ │
│ │ │ │ claims) │ │ complete) │ │ │
│ │ │ └─────────────┘ └─────────────────┘ │ │
│ │ │ │ │
│ │ └──────────────────────────────────────────┘ │
│ │ │
│ └─────────────────────── JSON Response ──────────────────────────────┘
│ │
│ Response includes: │
│ - Article content with source attribution │
│ - Political bias scores and transparency data │
│ - Related articles and story clusters │
│ - Fact-check verdicts for embedded claims │
│ - Pagination cursors for infinite scroll │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
Fact-Checking Pipeline#
text Copy
┌─────────────────────────────────────────────────────────────────────────────┐
│ FACT-CHECKING PIPELINE │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌──────────────┐ ┌──────────────────┐ │
│ │ Ingested │─────────►│ Claim Extraction │ │
│ │ Article │ │ (NLP Module) │ │
│ └──────────────┘ └────────┬─────────┘ │
│ │ │
│ Extracted claims (factual statements) │
│ │ │
│ ▼ │
│ ┌──────────────────┐ │
│ │ Fact-Check │ │
│ │ Agent │ │
│ └────────┬─────────┘ │
│ │ │
│ ┌──────────────────────┼──────────────────────┐ │
│ │ │ │ │
│ ▼ ▼ ▼ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ Evidence │ │ Google │ │ ClaimBuster │ │
│ │ Retrieval │ │ Fact Check │ │ API │ │
│ │ (Serper/ │ │ Tools API │ │ │ │
│ │ Brave) │ │ │ │ │ │
│ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ │
│ │ │ │ │
│ └─────────────────────┼──────────────────────┘ │
│ │ │
│ Evidence aggregation │
│ │ │
│ ▼ │
│ ┌──────────────────┐ │
│ │ LLM Verdict │ │
│ │ (Claude/GPT-4) │ │
│ │ │ │
│ │ Verdicts: │ │
│ │ - True │ │
│ │ - Mostly True │ │
│ │ - Mixed │ │
│ │ - Mostly False │ │
│ │ - False │ │
│ │ - Unverifiable │ │
│ └────────┬─────────┘ │
│ │ │
│ ▼ │
│ ┌──────────────────┐ │
│ │ claims table │ │
│ │ (PostgreSQL) │ │
│ │ │ │
│ │ Verdict + score │ │
│ │ Evidence links │ │
│ │ Source refs │ │
│ └──────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
text Copy
┌─────────────────────────────────────────────────────────────────────────────┐
│ MEDIA GENERATION FLOW │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌──────────────────┐ │
│ │ articles │ │
│ │ (PostgreSQL) │ │
│ └────────┬─────────┘ │
│ │ │
│ ┌──────────────┴──────────────┐ │
│ │ │ │
│ ▼ ▼ │
│ ┌──────────────────┐ ┌──────────────────┐ │
│ │ Audio Queue │ │ Video Queue │ │
│ │ Producer │ │ Producer │ │
│ └────────┬─────────┘ └────────┬─────────┘ │
│ │ │ │
│ ▼ ▼ │
│ ┌──────────────────┐ ┌──────────────────┐ │
│ │ yaa:audio- │ │ yaa:video- │ │
│ │ generation │ │ generation │ │
│ │ (BullMQ/Redis) │ │ (BullMQ/Redis) │ │
│ └────────┬─────────┘ └────────┬─────────┘ │
│ │ │ │
│ ▼ ▼ │
│ ┌──────────────────┐ ┌──────────────────┐ │
│ │ Audio Worker │ │ Video Worker │ │
│ │ │ │ │ │
│ │ ┌────────────┐ │ │ ┌────────────┐ │ │
│ │ │ ElevenLabs │ │ │ │ HeyGen │ │ │
│ │ │ TTS │ │ │ │ Avatar │ │ │
│ │ └────────────┘ │ │ └────────────┘ │ │
│ │ ┌────────────┐ │ │ ┌────────────┐ │ │
│ │ │ Ghana NLP │ │ │ │ Shotstack │ │ │
│ │ │ TTS │ │ │ │ Renderer │ │ │
│ │ └────────────┘ │ │ └────────────┘ │ │
│ │ │ │ ┌────────────┐ │ │
│ │ │ │ │ Caption │ │ │
│ │ │ │ │ Service │ │ │
│ │ │ │ └────────────┘ │ │
│ └────────┬─────────┘ └────────┬─────────┘ │
│ │ │ │
│ └──────────────┬──────────────┘ │
│ ▼ │
│ ┌──────────────────┐ │
│ │ MinIO / S3 │ │
│ │ (Media Storage) │ │
│ └──────────────────┘ │
│ │ │
│ ┌──────────────┼──────────────┐ │
│ │ │ │ │
│ ▼ ▼ ▼ │
│ ┌──────────────┐ ┌──────────┐ ┌──────────────┐ │
│ │ YouTube │ │ TikTok │ │ Instagram │ │
│ │ Upload │ │ Upload │ │ Upload │ │
│ └──────────────┘ └──────────┘ └──────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
AI Agent Communication Flow#
text Copy
┌─────────────────────────────────────────────────────────────────────────────┐
│ AGENT COMMUNICATION FLOW │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────────┐ │
│ │ Editor-in- │ │
│ │ Chief Agent │ │
│ │ (Orchestrator) │ │
│ └────────┬────────┘ │
│ │ │
│ │ broadcast() │
│ ▼ │
│ ┌─────────────────┐ │
│ │ MessageBus │ │
│ │ (Redis Pub/Sub) │ │
│ └────────┬────────┘ │
│ │ │
│ ┌──────────────────────┼──────────────────────┐ │
│ │ │ │ │
│ ▼ ▼ ▼ │
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │
│ │ Journalism │ │ Fact-Check │ │ Editorial │ │
│ │ Agent │ │ Agent │ │ Agent │ │
│ │ │ │ │ │ │ │
│ │ - Draft │ │ - Verify │ │ - Copy edit │ │
│ │ - Source check │ │ - Evidence │ │ - Style │ │
│ │ - Story dev │ │ - Verdict │ │ - Headlines │ │
│ └────────┬────────┘ └────────┬────────┘ └────────┬────────┘ │
│ │ │ │ │
│ │ send() │ send() │ send() │
│ │ │ │ │
│ └──────────────────────┼──────────────────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────┐ │
│ │ Editor-in- │ │
│ │ Chief Agent │ │
│ │ (aggregates │ │
│ │ results) │ │
│ └────────┬────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────┐ │
│ │ Social Media │ │
│ │ Agent │ │
│ │ │ │
│ │ Distributes │ │
│ │ across: │ │
│ │ - WhatsApp │ │
│ │ - YouTube │ │
│ │ - TikTok │ │
│ │ - Instagram │ │
│ │ - X/Twitter │ │
│ └─────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
Communication Patterns#
Synchronous Communication#
Pattern
Use Case
Example
HTTP/REST
Client-to-API requests
Web PWA -> Veritas API
HTTP/REST
Service-to-external-API
API -> HeyGen, ElevenLabs
SQL
Direct database queries
API -> PostgreSQL (raw SQL, pg)
HTTP
Search queries
API -> Elasticsearch
Asynchronous Communication#
Pattern
Use Case
Example
BullMQ (Redis)
Content processing pipeline
Scheduler -> Producer -> Worker
BullMQ (Redis)
Audio generation jobs
API -> Audio Queue -> ElevenLabs Worker
BullMQ (Redis)
Video generation jobs
API -> Video Queue -> HeyGen Worker
Redis Pub/Sub
Agent-to-agent messaging
Orchestrator <-> Journalism Agent
Redis Pub/Sub
Event broadcasting
Article published -> Social distribution
Webhooks
External callback handling
MTN MoMo payment status -> API callback
Cross-Cutting Communication Matrix#
From / To
Database
BullMQ
Redis PubSub
REST API
External API
Ingestion
Write (feed_items)
Enqueue (content-processing)
--
--
RSS/Sitemap
Processing Worker
Read/Write
Consume
--
--
--
Audio Worker
Read/Write
Consume
--
--
ElevenLabs, Ghana NLP
Video Worker
Read/Write
Consume
--
--
HeyGen, Shotstack
AI Agents
Read
--
Pub/Sub
--
Anthropic, OpenAI
API Server
Read/Write
--
Cache
Serve
Various
Social Scheduler
Read
Enqueue/Consume
--
--
YouTube, TikTok, etc.
API Architecture#
Layered Structure#
The Veritas API follows a four-layer architecture:
text Copy
┌──────────────────────────────────────────────────────────────────────┐
│ INTERFACES LAYER (HTTP) │
│ - Route handlers (29 files) │
│ - Middleware (logging, security, rate limiting, auth) │
│ - Zod request/response validation │
│ - OpenAPI specification generation (1,079 lines) │
│ - Error response formatting │
├──────────────────────────────────────────────────────────────────────┤
│ APPLICATION LAYER │
│ - Cursor-based pagination logic │
│ - Use case orchestration │
├──────────────────────────────────────────────────────────────────────┤
│ DOMAIN LAYER (11,669 lines) │
│ - Subscriptions (2,422 lines: tiers, pricing, family, student) │
│ - B2B API (1,681 lines: white-label, partner access) │
│ - Ad Networks (2,299 lines: Google AdSense, Facebook AN) │
│ - Advertisements (1,293 lines: scheduling, targeting) │
│ - API Keys (985 lines: generation, rotation, usage) │
│ - Compliance (300+ lines: election, NMC, DPA) │
│ - Payments (Stripe, MTN MoMo, Vodafone Cash, AirtelTigo) │
│ - Sources (bias scoring, ownership, regional emphasis) │
│ - Analytics (1,504 lines: conversion, churn, cohorts) │
│ - Reading Mode (689 lines), Paywall (796 lines) │
├──────────────────────────────────────────────────────────────────────┤
│ INFRASTRUCTURE LAYER (416 lines) │
│ - PostgreSQL connection pool (configurable max, idle, timeout) │
│ - Redis client with retry strategy and lazy connect │
│ - Elasticsearch client with auth and TLS │
│ - JWT token management (custom HMAC-SHA256, no external lib) │
│ - AES-256-GCM encryption for sensitive data (anonymous tips) │
└──────────────────────────────────────────────────────────────────────┘
Middleware Chain (Execution Order)#
text Copy
Request
│
├──► 1. Logger (hono/logger) ............... Request method, path, timing
├──► 2. Timing (hono/timing) ............... Server-Timing header
├──► 3. Security Headers (hono/secure-headers) X-Frame-Options, CSP, etc.
├──► 4. CORS (hono/cors) ................... Configurable origins
├──► 5. Rate Limiting (custom) ............. Redis sliding window, 120/60s
├──► 6. API Key Auth (custom) .............. Timing-safe comparison
└──► 7. Route Handler
│
├──► Zod validation (body, query, params)
├──► Business logic
└──► JSON response
│
└──► Error Handler (HTTPException, ZodError, generic)
Rate Limiting#
Algorithm : Sliding window using Redis sorted sets
Default : 120 requests per 60-second window
Fallback : In-memory store if Redis unavailable
Client ID : X-Forwarded-For -> X-Real-IP -> "unknown"
Response Headers : X-RateLimit-Limit, X-RateLimit-Remaining,
X-RateLimit-Reset
429 Response : { "error": "Too Many Requests", "retryAfterSeconds": <n> }
Database Architecture#
Schema Overview#
The database uses 50+ tables across 63 migrations, organized into the following
domains:
Core Content Tables#
Table
Purpose
sources
News source metadata (300+ Ghanaian outlets)
articles
Published article content
authors
Article author profiles
feed_items
Raw ingested RSS/Atom content
normalized_feed_items
Processed, deduplicated content
content_fingerprints
Simhash64 fingerprints for dedup
content_dedupe_mappings
Duplicate item -> canonical mappings
Taxonomy & Classification#
Table
Purpose
categories
Content categories (politics, business)
topics
Fine-grained topics
tags
Keyword tags from extraction
article_categories
Article-to-category junction
article_topics
Article-to-topic junction
article_tags
Article-to-tag junction
Fact-Checking & Trust#
Table
Purpose
claims
Extracted factual claims
article_political_bias_scores
Political bias analysis per article
source_reliability_scores
Source credibility tracking
misinformation_flags
Content flagging
Users & Engagement#
Table
Purpose
users
User accounts
subscriptions
Subscription tiers and status
user_preferences
Reading preferences, feed customization
article_views
View tracking for analytics
api_keys
B2B API key management
Table
Purpose
voice_profiles
ElevenLabs voice configurations
avatar_profiles
HeyGen avatar configurations
audio_renders
Generated audio file references
video_renders
Generated video file references
Custom ENUM Types#
sql Copy
source_kind: 'newspaper' | 'radio' | 'tv' | 'online' | 'wire' | 'government' | 'blog'
article_status: 'draft' | 'review' | 'published' | 'archived' | 'retracted'
media_type: 'image' | 'video' | 'audio' | 'document'
fact_check_verdict: 'true' | 'mostly_true' | 'mixed' | 'mostly_false' | 'false' | 'unverifiable'
subscription_tier: 'free' | 'premium' | 'student' | 'family'
Database Access Pattern#
Veritas uses direct SQL queries with the pg library (no ORM):
typescript Copy
// Example: Article listing with source join
const { rows } = await pool.query<ArticleRow>(
`SELECT a.id, a.title, a.slug, a.content, a.published_at,
s.name as source_name, s.canonical_domain
FROM articles a
LEFT JOIN sources s ON a.source_id = s.id
WHERE a.status = 'published'
ORDER BY a.published_at DESC
LIMIT $1 OFFSET $2`,
[limit, offset]
);
Redis Key Patterns#
Pattern
Purpose
veritas:api:rate-limit:*
Sliding window rate limit state
veritas:content-processing
BullMQ content processing queue
veritas:audio-generation
BullMQ audio generation queue
veritas:video-generation
BullMQ video generation queue
veritas:cache:*
API response caching
Deployment Architecture#
Kubernetes / ECS Topology#
text Copy
┌──────────────────────────────────────────────────────────────────────────────┐
│ AWS INFRASTRUCTURE │
│ │
│ ┌────────────────────────────────────────────────────────────────────────┐ │
│ │ INGRESS LAYER │ │
│ │ ┌──────────────┐ ┌──────────────┐ ┌────────────────────────────┐ │ │
│ │ │ ALB / nginx │ │ WAF / Shield │ │ SSL/TLS (ACM) │ │ │
│ │ │ Ingress │ │ │ │ CloudFront CDN │ │ │
│ │ └──────┬───────┘ └──────────────┘ └────────────────────────────┘ │ │
│ └─────────┼─────────────────────────────────────────────────────────────┘ │
│ │ │
│ ┌─────────┼─────────────────────────────────────────────────────────────┐ │
│ │ ▼ APPLICATION LAYER │ │
│ │ │ │
│ │ ┌──────────┐ ┌──────────────┐ ┌──────────────────────────────┐ │ │
│ │ │ Veritas │ │ Veritas │ │ Veritas Ingestion │ │ │
│ │ │ Web │ │ API │ │ │ │ │
│ │ │ (Next.js)│ │ (Hono) │ │ ┌───────────┐ ┌─────────┐ │ │ │
│ │ │ │ │ │ │ │ Scheduler │ │ Worker │ │ │ │
│ │ │ 2-10 │ │ 2-10 │ │ │ (1 pod) │ │(2-8 pods│ │ │ │
│ │ │ replicas │ │ replicas │ │ └───────────┘ └─────────┘ │ │ │
│ │ └──────────┘ └──────────────┘ └──────────────────────────────┘ │ │
│ │ │ │
│ └───────────────────────────────────────────────────────────────────────┘ │
│ │ │ │ │
│ ┌─────────┼──────────────┼─────────────────────┼────────────────────────┐ │
│ │ ▼ ▼ ▼ │ │
│ │ DATA LAYER │ │
│ │ │ │
│ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ │
│ │ │ PostgreSQL │ │ Redis │ │Elasticsearch │ │ │
│ │ │ (RDS or │ │ (ElastiCache │ │ (Managed or │ │ │
│ │ │ StatefulSet)│ │ or pod) │ │ OpenSearch) │ │ │
│ │ └──────────────┘ └──────────────┘ └──────────────┘ │ │
│ │ │ │
│ │ ┌──────────────────────────────────────────────────────────────┐ │ │
│ │ │ S3 / MinIO │ │ │
│ │ │ Media asset storage │ │ │
│ │ └──────────────────────────────────────────────────────────────┘ │ │
│ └───────────────────────────────────────────────────────────────────────┘ │
│ │
│ ┌───────────────────────────────────────────────────────────────────────┐ │
│ │ OBSERVABILITY LAYER │ │
│ │ ┌────────────┐ ┌────────────┐ ┌────────────┐ ┌────────────┐ │ │
│ │ │ Prometheus │ │ Grafana │ │ Jaeger │ │ CloudWatch │ │ │
│ │ └────────────┘ └────────────┘ └────────────┘ └────────────┘ │ │
│ └───────────────────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────────────────────┘
GitOps Flow (ArgoCD)#
text Copy
┌──────────────┐ push ┌──────────────┐ sync ┌──────────────┐
│ Developer │ ──────────►│ GitHub │ ◄────────►│ ArgoCD │
│ │ │ │ │ │
└──────────────┘ └──────────────┘ └──────┬───────┘
│
┌─────────────────────────────────────────────────────────────┼────────┐
│ ▼ │
│ KUBERNETES CLUSTER │
│ │
│ deploy/argocd/ │
│ │ │
│ ├──► apps/dev/veritas-api.yaml │
│ ├──► apps/dev/veritas-web.yaml │
│ ├──► apps/dev/veritas-ingestion.yaml │
│ │ │
│ ├──► apps/staging/veritas-api.yaml │
│ ├──► apps/staging/veritas-web.yaml │
│ │ │
│ └──► apps/prod/veritas-api.yaml │
│ apps/prod/veritas-web.yaml │
│ apps/prod/veritas-ingestion.yaml │
│ │
└──────────────────────────────────────────────────────────────────────┘
CI/CD Pipeline (GitHub Actions)#
text Copy
┌──────────────────────────────────────────────────────────────────────┐
│ CI PIPELINE │
│ │
│ Push / PR │
│ │ │
│ ├──► Quality Checks │
│ │ ├── Lint (ESLint) │
│ │ ├── Type Check (TypeScript) │
│ │ └── Domain Boundary Validation │
│ │ │
│ ├──► Build │
│ │ ├── nx build veritas-api │
│ │ ├── nx build veritas-web │
│ │ └── nx build veritas-ingestion │
│ │ │
│ ├──► Tests │
│ │ ├── Unit Tests (Vitest) │
│ │ ├── Integration Tests │
│ │ └── Security Audit (npm audit) │
│ │ │
│ ├──► Container Build │
│ │ ├── Multi-arch Docker (amd64, arm64) │
│ │ └── Trivy security scanning │
│ │ │
│ └──► Deploy (main branch) │
│ ├── Staging (auto-deploy) │
│ ├── Canary (10% traffic) │
│ └── Production (manual approval) │
└──────────────────────────────────────────────────────────────────────┘
Infrastructure is managed via Terraform in infra/terraform/:
Component
Purpose
VPC
Network isolation, public/private subnets
ECS
Container orchestration (Fargate)
ALB
Load balancing with health checks
ECR
Docker image registry
S3
Media storage, static assets
SQS/SNS
Message queuing and notifications
CloudWatch
Logging, metrics, alarms
Secrets Manager
API keys, database credentials
SSM Parameter
Configuration management
Security Architecture#
Authentication Flow#
text Copy
┌────────┐ 1. Login ┌──────────────┐ 2. Validate ┌──────────┐
│ Client │ ───────────────►│ Veritas API │ ────────────────►│PostgreSQL│
│ (PWA) │ │ /auth/login │ │ (users) │
└────────┘ └──────┬───────┘ └──────────┘
▲ │
│ │ 3. Generate JWT
│ │ (HMAC-SHA256, custom impl)
│ │ TTL: 7 days
│ 4. Bearer Token │
└────────────────────────────┘
Authorization Model#
Role
Permissions
Anonymous
Read published articles, search, view sources, browse taxonomy
Reader
+ Personalized feed, preferences, bookmarks
Premium
+ Full article access, ad-free, audio/video, advanced search
Student
+ Discounted premium access with .edu verification
Family
+ Up to 5 family members, shared subscription
Journalist
+ Writing assistant, source management, draft articles
B2B Partner
+ API key access, white-label endpoints, bulk data
Admin
+ User management, compliance, system configuration
Data Protection#
Layer
Protection
In Transit
TLS 1.2+, HTTPS only
At Rest
AES-256-GCM (tips encryption), RDS encryption
Secrets
AWS Secrets Manager, SSM Parameter Store
PII
Anonymized analytics, encrypted tips
API Keys
Timing-safe comparison, rotation support
Rate Limits
Per-IP sliding window, Redis-backed
Compliance Framework#
Regulation
Implementation
Ghana Data Protection Act 2012
Data protection endpoints, consent management
GDPR (for EU users)
Right to erasure, data export
Ghana NMC Guidelines
Political coverage balance, fair reporting
Ghana Electoral Commission
24h campaign silence, equal coverage enforcement
Ghana Criminal Offences Act
False news detection, election offense monitoring
Observability Stack#
Metrics (Prometheus/CloudWatch)#
yaml Copy
Service Metrics:
- http_request_duration_seconds (p50, p95, p99)
- http_requests_total (by status, method, route)
- rate_limit_hits_total
- api_key_auth_failures_total
Application Metrics:
- articles_ingested_total
- articles_published_total
- fact_checks_completed_total
- search_queries_total
- subscriptions_active_gauge
- payments_processed_total
Queue Metrics:
- bullmq_jobs_active
- bullmq_jobs_completed
- bullmq_jobs_failed
- bullmq_jobs_delayed
- bullmq_queue_depth
Infrastructure Metrics:
- container_cpu_usage
- container_memory_usage
- database_connections_active
- redis_connected_clients
- elasticsearch_query_latency
Logging (Structured JSON)#
json Copy
{
"level": "info",
"service": "veritas-api",
"timestamp": "2026-01-26T12:00:00.000Z",
"trace_id": "abc123",
"method": "GET",
"path": "/api/v1/articles",
"status_code": 200,
"duration_ms": 45,
"user_id": "usr_xxx",
"rate_limit_remaining": 118
}
Health Checks#
Endpoint
Purpose
GET /health
Basic liveness (always returns 200)
GET /healthz
Kubernetes liveness probe
GET /readyz
Readiness (checks DB, Redis, Elasticsearch)
GET /health/detailed
Full status with component health breakdown
Component
Metric
Target
API - Article list
p50 latency
< 100ms
API - Article list
p95 latency
< 300ms
API - Search
p50 latency
< 200ms
API - Search
p95 latency
< 500ms
API - Auth
p95 latency
< 150ms
API - Fact check
p95 latency
< 2s
Web PWA
Lighthouse score
> 90
Web PWA
LCP
< 2.5s (4G)
Web PWA
FID
< 100ms
Web PWA
CLS
< 0.1
Ingestion
RSS collection
300 feeds / tick
Ingestion
Processing latency
< 30s per item
CDN
Static asset TTFB
< 50ms (cached)
Rate Limit
Max requests
120 / 60s window
Scaling Configuration#
Horizontal Pod Autoscaler#
yaml Copy
# Veritas API HPA
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: veritas-api
minReplicas: 2
maxReplicas: 10
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 70
- type: Resource
resource:
name: memory
target:
type: Utilization
averageUtilization: 80
Resource Defaults#
yaml Copy
# API Service
resources:
requests:
memory: "256Mi"
cpu: "200m"
limits:
memory: "1Gi"
cpu: "1000m"
# Ingestion Worker
resources:
requests:
memory: "512Mi"
cpu: "200m"
limits:
memory: "2Gi"
cpu: "1000m"
# Web (Next.js)
resources:
requests:
memory: "256Mi"
cpu: "100m"
limits:
memory: "512Mi"
cpu: "500m"
Database Connection Pools#
Service
Max Connections
Idle Timeout
Connect Timeout
API
20
30s
5s
Ingestion
10
30s
10s
Technology Stack#
Layer
Technology
Runtime
Node.js 22+ (LTS)
Language
TypeScript 6.0.3 (ESM)
API Framework
Hono 4.x (edge-compatible, lightweight)
Web Framework
Next.js 15 (App Router, RSC, SSR)
Styling
Tailwind CSS 3.x
i18n
next-intl (6 Ghanaian locales)
Validation
Zod 3.x
Database
PostgreSQL 16 (with pgvector)
Cache/Queue
Redis 7.x (via ioredis)
Job Queue
BullMQ 5.x
Search
Elasticsearch 8.x
Object Storage
MinIO / AWS S3
Authentication
Custom JWT (HMAC-SHA256) + NextAuth
AI/LLM
Anthropic Claude, OpenAI GPT-4, Cohere
TTS
ElevenLabs, Ghana NLP
Video
HeyGen, Shotstack
NLP
HuggingFace Inference, spaCy (Python NER)
Testing
Vitest, Playwright
Build
Nx 20.x with esbuild
CI/CD
GitHub Actions (18 workflows)
IaC
Terraform (80+ .tf files)
GitOps
ArgoCD 2.x (100+ manifests)
Container
Docker (multi-arch: amd64, arm64)
Orchestration
Kubernetes / AWS ECS Fargate
CDN
CloudFront / Cloudflare
Monitoring
Prometheus, Grafana, Jaeger, CloudWatch
External API Integrations#
AI & Machine Learning#
Provider
Purpose
Integration Point
Anthropic
Claude for abstractive summarization, fact-check
NLP, AI Agents
OpenAI
GPT-4 for content generation, embeddings
NLP, AI Agents
Cohere
Embeddings, reranking
NLP Embedding Service
HuggingFace
Sentiment analysis, topic classification
NLP Services
Provider
Purpose
Integration Point
HeyGen
AI avatar video generation
Video Production
Shotstack
Video rendering and composition
Video Production
ElevenLabs
High-quality TTS voice synthesis
Audio Production
Ghana NLP
Ghanaian language TTS, translation, NER
NLP, Audio
Search & Fact-Checking#
Provider
Purpose
Integration Point
Serper
Web search for evidence retrieval
Evidence Retrieval
Brave Search
Alternative web search
Evidence Retrieval
Google Fact Check
Existing fact-check database
Fact-Check Agent
ClaimBuster
Claim worthiness scoring
Claim Extraction
Payments (Ghana-Focused)#
Provider
Purpose
Integration Point
Stripe
International card payments
Payments Domain
MTN MoMo
Mobile money (MTN Ghana)
Payments Domain
Vodafone Cash
Mobile money (Vodafone Ghana)
Payments Domain
AirtelTigo Money
Mobile money (AirtelTigo Ghana)
Payments Domain
Social Distribution#
Platform
Purpose
Integration Point
YouTube
Video and Shorts publishing
Social Automation
TikTok
Short-form video publishing
Social Automation
Instagram
Stories and Reels publishing
Social Automation
WhatsApp
Channel and business messaging
Social Automation
X (Twitter)
News distribution
Social Automation
Telegram
Channel publishing
Social Automation
LinkedIn
Professional content distribution
Social Automation
Facebook
Page publishing
Social Automation
Last Updated: January 2026
Part of the Veritas domain architecture documentation