Eve library

Veritas System Architecture

Architecture

Revision 1

On this page

This document provides comprehensive documentation of the Veritas platform system architecture --- an AI-powered news agency focused on Ghana and West Africa.

Overview#

Veritas is a next-generation news platform that combines automated content ingestion, AI-driven editorial workflows, fact-checking, multilingual support for Ghanaian languages, and multimedia production. It operates as three primary services within the Oshun monorepo, backed by a rich library ecosystem of 40+ domain modules.

Architecture Type: Modular monolith with service-oriented decomposition Total Services: 3 applications + 40+ library modules Deployment Model: Kubernetes with GitOps (ArgoCD) / ECS on AWS Primary Stack: Node.js/Hono, TypeScript, PostgreSQL, Redis, Elasticsearch

Key Statistics#

Metric Value
API endpoints 80+ REST endpoints across 29 route files
Database tables 50+ tables across 63 migrations
External integrations 30+ third-party APIs
Supported languages 6 Ghanaian languages (English, Twi, Ewe, Ga, Hausa, Dagbani)
AI agent types 7 specialized newsroom agents
NLP services 13 analysis modules
Content sources 300+ RSS feeds from Ghanaian news outlets
Payment providers 4 (Stripe, MTN MoMo, Vodafone Cash, AirtelTigo)

System Context (C4 Level 1)#

text
┌─────────────────────────────────────────────────────────────────────────────┐
│                            EXTERNAL SYSTEMS                                │
├───────────────┬──────────────┬──────────────┬──────────────┬───────────────┤
│ AI/LLM        │ Media        │ Search &     │ Payment      │ Social        │
│ Providers     │ Production   │ Fact-Check   │ Gateways     │ Platforms     │
│               │              │              │              │               │
│ Anthropic     │ HeyGen       │ Serper       │ Stripe       │ YouTube       │
│ OpenAI        │ ElevenLabs   │ Brave Search │ MTN MoMo     │ TikTok        │
│ Cohere        │ Ghana NLP    │ ClaimBuster  │ Vodafone     │ Instagram     │
│ HuggingFace   │ Shotstack    │ Google FCT   │ AirtelTigo   │ WhatsApp      │
└───────┬───────┴──────┬───────┴──────┬───────┴──────┬───────┴───────┬───────┘
        │              │              │              │               │
        ▼              ▼              ▼              ▼               ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│                                                                             │
│                          VERITAS PLATFORM                                   │
│                                                                             │
│  ┌───────────────────────────────────────────────────────────────────────┐  │
│  │                    VERITAS API (Hono REST)                             │  │
│  │              Port 3001 -- 80+ endpoints, v1 prefix                    │  │
│  └─────────────────────────────┬─────────────────────────────────────────┘  │
│                                │                                            │
│  ┌──────────────┬──────────────┴──────────────┬──────────────────────────┐  │
│  │  Ingestion   │         Domain Logic        │     Library Modules     │  │
│  │  Service     │                             │                         │  │
│  │  (Scheduler, │  Articles, Claims, Sources, │  AI Agents (7 types)    │  │
│  │  Producer,   │  Subscriptions, Payments,   │  NLP Services (13)      │  │
│  │  Worker)     │  Compliance, Ads, B2B       │  Video/Audio (25+)      │  │
│  └──────┬───────┴──────────────┬──────────────┴──────────────┬──────────┘  │
│         │                      │                              │             │
│  ┌──────┴──────────────────────┴──────────────────────────────┴──────────┐  │
│  │                         DATA LAYER                                    │  │
│  │  PostgreSQL │ Redis │ Elasticsearch │ MinIO (S3)                     │  │
│  └───────────────────────────────────────────────────────────────────────┘  │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘
        ▲              ▲              ▲              ▲               ▲
        │              │              │              │               │
┌───────┴───────┬──────┴──────┬───────┴──────┬───────┴──────┬────────┴───────┐
│ Web PWA       │ Mobile App  │ WhatsApp     │ B2B API      │ RSS Readers    │
│ (Next.js 15)  │ (React      │ Channel      │ Partners     │ & Aggregators  │
│ Port 3000     │ Native)     │              │              │                │
└───────────────┴─────────────┴──────────────┴──────────────┴────────────────┘
                          CLIENT APPLICATIONS

Container Diagram (C4 Level 2)#

Application Services#

text
┌──────────────────────────────────────────────────────────────────────────────┐
│                            APPLICATION LAYER                                 │
│                                                                              │
│  ┌────────────────────────────────────────────────────────────────────────┐  │
│  │                      VERITAS API (Port 3001)                           │  │
│  │                                                                        │  │
│  │  Framework: Hono          Transport: HTTP/REST                         │  │
│  │  Auth: JWT (HMAC-SHA256)  Rate Limiting: Redis sliding window          │  │
│  │  Validation: Zod          Documentation: OpenAPI 3.0.3                 │  │
│  │                                                                        │  │
│  │  Middleware Chain:                                                      │  │
│  │  Logger -> Timing -> Security Headers -> CORS -> Rate Limit -> Auth    │  │
│  │                                                                        │  │
│  │  Routes (29 files):                                                    │  │
│  │  /api/v1/articles     /api/v1/claims       /api/v1/subscriptions       │  │
│  │  /api/v1/auth         /api/v1/sources      /api/v1/payments            │  │
│  │  /api/v1/stories      /api/v1/taxonomy     /api/v1/preferences         │  │
│  │  /api/v1/bias         /api/v1/compliance   /api/v1/election-coverage   │  │
│  │  /api/v1/tips         /api/v1/paywall      /api/v1/reading-mode        │  │
│  │  /api/v1/ads          /api/v1/ad-networks  /api/v1/b2b-api             │  │
│  │  /api/v1/api-keys     /api/v1/avatars      /api/v1/voices              │  │
│  │  /api/v1/writing-assistant  /api/v1/data-protection                    │  │
│  │  /api/v1/subscription-analytics  /api/v1/stripe-payments               │  │
│  │  /api/v1/vodafone-payments  /api/v1/airteltigo-payments                │  │
│  └────────────────────────────────────────────────────────────────────────┘  │
│                                                                              │
│  ┌─────────────────────────────────┐  ┌──────────────────────────────────┐  │
│  │   VERITAS WEB (Port 3000)       │  │   VERITAS INGESTION              │  │
│  │                                 │  │                                  │  │
│  │  Framework: Next.js 15         │  │  Framework: Node.js + BullMQ     │  │
│  │  Renderer: App Router (SSR)    │  │  Modes: scheduler, producer,     │  │
│  │  Styling: Tailwind CSS         │  │         worker, seed              │  │
│  │  i18n: next-intl (6 locales)   │  │  Collectors: RSS, Sitemap        │  │
│  │  PWA: Standalone + Shortcuts   │  │  Pipeline: Normalize, Dedupe,    │  │
│  │  Auth: NextAuth                │  │    Language, Timestamp, Media     │  │
│  │                                 │  │  Queue: veritas:content-         │  │
│  │  Locales:                      │  │         processing               │  │
│  │  en, tw, ee, ga, ha, dag       │  │  Concurrency: 10 RSS fetches,    │  │
│  │                                 │  │    4 processing workers          │  │
│  └─────────────────────────────────┘  └──────────────────────────────────┘  │
└──────────────────────────────────────────────────────────────────────────────┘

Library Module Ecosystem#

text
┌──────────────────────────────────────────────────────────────────────────────┐
│                          LIBRARY MODULES (libs/veritas/)                      │
│                                                                              │
│  ┌──────────────────────┐  ┌───────────────────────┐  ┌──────────────────┐  │
│  │  CORE LIBRARIES      │  │  AI AGENT FRAMEWORK   │  │  NLP SERVICES    │  │
│  │                      │  │                       │  │                  │  │
│  │  core/               │  │  agents-orchestrator/ │  │  nlp-core/       │  │
│  │  database/           │  │  agents-core/         │  │  ghana-nlp/      │  │
│  │  cache/              │  │  agents-journalism/   │  │  (13 analysis    │  │
│  │  auth/               │  │  agents-fact-checking/│  │   modules)       │  │
│  │  storage/            │  │  agents-editorial/    │  │                  │  │
│  │  events/             │  │  agents-product/      │  │  Sentiment       │  │
│  │  models/             │  │  agents-devops/       │  │  Topics          │  │
│  │  llm/                │  │  agents-qa/           │  │  Keywords        │  │
│  │                      │  │  agents-social-media/ │  │  Summarization   │  │
│  └──────────────────────┘  └───────────────────────┘  │  Claims          │  │
│                                                        │  Language        │  │
│  ┌──────────────────────┐  ┌───────────────────────┐  │  Embeddings      │  │
│  │  CONTENT & ANALYSIS  │  │  MEDIA PRODUCTION     │  │  Similarity      │  │
│  │                      │  │                       │  │  Evidence         │  │
│  │  fact-checking/      │  │  video-production/    │  └──────────────────┘  │
│  │  bias-detection/     │  │  audio-production/    │                        │
│  │  claims/             │  │  (25+ sub-modules     │  ┌──────────────────┐  │
│  │  content-classific./ │  │   for HeyGen,         │  │  MONETIZATION    │  │
│  │  article-generation/ │  │   ElevenLabs,         │  │                  │  │
│  │  headline-service/   │  │   YouTube, TikTok,    │  │  payments/       │  │
│  │  story-clustering/   │  │   Instagram)          │  │  analytics/      │  │
│  │  knowledge-graph/    │  │                       │  │  notifications/  │  │
│  │  rag/                │  │                       │  │  social-automat./│  │
│  │  recommendations/    │  │                       │  │                  │  │
│  │  search/             │  │                       │  │                  │  │
│  └──────────────────────┘  └───────────────────────┘  └──────────────────┘  │
│                                                                              │
│  ┌──────────────────────┐                                                    │
│  │  INGESTION CORE      │                                                    │
│  │                      │                                                    │
│  │  ingestion-core/     │                                                    │
│  │  (RSS, scraping,     │                                                    │
│  │  normalization,      │                                                    │
│  │  deduplication)      │                                                    │
│  └──────────────────────┘                                                    │
└──────────────────────────────────────────────────────────────────────────────┘

Data Layer#

text
┌──────────────────────────────────────────────────────────────────────────────┐
│                              DATA LAYER                                      │
│                                                                              │
│  ┌──────────────────┐  ┌──────────────────┐  ┌──────────────────────────┐   │
│  │   PostgreSQL     │  │     Redis        │  │    Elasticsearch         │   │
│  │   (Port 5432)    │  │   (Port 6379)    │  │    (Port 9200)           │   │
│  │                  │  │                  │  │                          │   │
│  │  50+ tables      │  │  Rate limiting   │  │  Full-text search        │   │
│  │  63 migrations   │  │  Session cache   │  │  Article indexing        │   │
│  │                  │  │  BullMQ queues   │  │  Autocomplete            │   │
│  │  Articles        │  │  Pub/Sub events  │  │  Search analytics        │   │
│  │  Sources         │  │  API caching     │  │                          │   │
│  │  Users           │  │                  │  │  Index:                  │   │
│  │  Claims          │  │  Key Patterns:   │  │  veritas-articles        │   │
│  │  Subscriptions   │  │  veritas:api:*   │  │                          │   │
│  │  Taxonomy        │  │  veritas:queue:* │  │                          │   │
│  │  Feed Items      │  │  veritas:cache:* │  │                          │   │
│  │  Media Assets    │  │                  │  │                          │   │
│  └──────────────────┘  └──────────────────┘  └──────────────────────────┘   │
│                                                                              │
│  ┌──────────────────────────────────────────────────────────────────────┐    │
│  │                       MinIO / S3 (Port 9000)                         │    │
│  │                                                                      │    │
│  │  S3-compatible object storage for media assets                       │    │
│  │  - Article images and thumbnails                                     │    │
│  │  - Generated audio files (TTS)                                       │    │
│  │  - Generated video files (HeyGen, Shotstack)                         │    │
│  │  - Static assets for CDN distribution                                │    │
│  └──────────────────────────────────────────────────────────────────────┘    │
└──────────────────────────────────────────────────────────────────────────────┘

Service Catalog#

Tier 1: Core Applications#

Service Port Framework Responsibility
Veritas API 3001 Hono REST API: articles, auth, search, fact-checking, payments, compliance
Veritas Web 3000 Next.js 15 PWA frontend: SSR, i18n, offline support, responsive UI
Veritas Ingestion -- Node.js Content pipeline: RSS collection, processing, deduplication

Tier 2: AI Agent Services#

Agent Library Path Responsibility
Editor-in-Chief libs/veritas/agents-orchestrator/ Coordinates editorial workflow, content prioritization
Journalism Agent libs/veritas/agents-journalism/ Article drafting, source verification, story development
Fact-Checking Agent libs/veritas/agents-fact-checking/ Claim verification, evidence gathering, verdict assignment
Editorial Agent libs/veritas/agents-editorial/ Copy editing, style enforcement, headline optimization
Product Agent libs/veritas/agents-product/ UX decisions, feature prioritization, A/B testing
DevOps Agent libs/veritas/agents-devops/ Infrastructure monitoring, deployment automation
QA Agent libs/veritas/agents-qa/ Test generation, regression detection, quality gates
Social Media Agent libs/veritas/agents-social-media/ Cross-platform publishing, engagement optimization

Tier 3: NLP Analysis Modules#

Module Library Path / Package Responsibility
Sentiment Analysis nlp/sentiment-service HuggingFace sentiment + Ghana-specific calibration
Topic Classification nlp/topic-classification Zero-shot topic assignment via HuggingFace
Keyword Extraction nlp/keyword-extraction RAKE algorithm for keyword identification
Extractive Summarization nlp/extractive-summarization TextRank with TF-IDF for article summaries
Abstractive Summarization nlp/abstractive-summarization LLM-powered summaries (Claude, GPT-4)
Claim Extraction nlp/claim-extraction Heuristic + LLM extraction of factual claims
Language Detection nlp/language-service Ghanaian language detection, Twi code-switching
Embedding Generation nlp/embedding-service OpenAI/Cohere vector embeddings
Semantic Similarity nlp/similarity-service Cosine similarity for content matching
Evidence Retrieval nlp/evidence-retrieval Web search via Serper/Brave for fact-checking
Ghana NLP nlp/ghana-nlp Translation, TTS, STT, NER for Ghanaian languages
Khaya Client nlp/khaya-client Khaya translation API for local languages
Service Kit nlp/service-kit Base HTTP primitives shared across NLP modules

Tier 4: Content Production Modules#

Module Category Count Key Modules
Video Production 21 HeyGen avatars, Shotstack rendering, YouTube/TikTok upload, captioning, thumbnails
Audio Production 5 ElevenLabs TTS, Ghana NLP TTS, audio generation queue
Social Distribution 15 WhatsApp, Instagram, TikTok, Facebook, X, Telegram, LinkedIn, YouTube
Ingestion 14 RSS collector, scraper, dedup, language detection, media extraction

Data Flow Patterns#

Content Ingestion Pipeline#

text
┌─────────────────────────────────────────────────────────────────────────────┐
│                       CONTENT INGESTION FLOW                                │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│  ┌──────────────┐    ┌──────────────┐    ┌──────────────┐                  │
│  │ RSS Feeds    │    │ Sitemaps     │    │ Social APIs  │                  │
│  │ (300+ feeds) │    │              │    │ (X/Twitter)  │                  │
│  └──────┬───────┘    └──────┬───────┘    └──────┬───────┘                  │
│         │                   │                   │                           │
│         └───────────────────┼───────────────────┘                           │
│                             ▼                                               │
│                   ┌──────────────────┐                                      │
│                   │   SCHEDULER      │                                      │
│                   │                  │                                      │
│                   │  Adaptive poll   │                                      │
│                   │  ETag/304        │                                      │
│                   │  Rate limiting   │                                      │
│                   │  Backoff         │                                      │
│                   └────────┬─────────┘                                      │
│                            │                                                │
│                            ▼                                                │
│                   ┌──────────────────┐                                      │
│                   │  feed_items      │                                      │
│                   │  (PostgreSQL)    │                                      │
│                   └────────┬─────────┘                                      │
│                            │                                                │
│                            ▼                                                │
│                   ┌──────────────────┐                                      │
│                   │   PRODUCER       │                                      │
│                   │                  │                                      │
│                   │  Polls DB for    │                                      │
│                   │  unprocessed     │                                      │
│                   │  items (100/tick)│                                      │
│                   └────────┬─────────┘                                      │
│                            │                                                │
│                            ▼                                                │
│                   ┌──────────────────┐                                      │
│                   │ BullMQ Queue     │                                      │
│                   │ veritas:content- │                                      │
│                   │ processing       │                                      │
│                   │ (Redis)          │                                      │
│                   └────────┬─────────┘                                      │
│                            │                                                │
│                            ▼                                                │
│  ┌──────────────────────────────────────────────────────────────────────┐   │
│  │                      PROCESSING WORKER (4 concurrent)                │   │
│  │                                                                      │   │
│  │  ┌─────────┐  ┌─────────┐  ┌─────────┐  ┌─────────┐  ┌─────────┐  │   │
│  │  │Normalize│─►│ Dedupe  │─►│Language │─►│Timestamp│─►│ Media   │  │   │
│  │  │ Content │  │(Simhash │  │ Detect  │  │Normalize│  │ Extract │  │   │
│  │  │         │  │  64-bit)│  │         │  │         │  │         │  │   │
│  │  └─────────┘  └─────────┘  └─────────┘  └─────────┘  └─────────┘  │   │
│  │                                                                      │   │
│  └──────────────────────────────────────────────────────────────────────┘   │
│                            │                                                │
│                            ▼                                                │
│                   ┌──────────────────┐                                      │
│                   │normalized_feed_  │                                      │
│                   │items (PostgreSQL)│                                      │
│                   └──────────────────┘                                      │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘

Article Delivery Flow#

text
┌─────────────────────────────────────────────────────────────────────────────┐
│                       ARTICLE DELIVERY FLOW                                 │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│  ┌────────┐   HTTPS    ┌──────────────────────────────────────────┐         │
│  │ Client │ ──────────►│            VERITAS API                   │         │
│  │ (PWA)  │            │                                          │         │
│  └────────┘            │  ┌────────────────────────────────────┐  │         │
│       ▲                │  │         Middleware Chain            │  │         │
│       │                │  │  Logger -> Security -> CORS ->      │  │         │
│       │                │  │  Rate Limit -> API Key Auth          │  │         │
│       │                │  └──────────────┬─────────────────────┘  │         │
│       │                │                 │                        │         │
│       │                │                 ▼                        │         │
│       │                │  ┌────────────────────────────────────┐  │         │
│       │                │  │         Route Handler              │  │         │
│       │                │  │                                    │  │         │
│       │                │  │  Zod validation -> Business logic  │  │         │
│       │                │  │  -> Cursor pagination -> Response  │  │         │
│       │                │  └──────┬─────────────────┬───────────┘  │         │
│       │                │         │                 │              │         │
│       │                │         ▼                 ▼              │         │
│       │                │  ┌─────────────┐  ┌─────────────────┐   │         │
│       │                │  │ PostgreSQL  │  │ Elasticsearch   │   │         │
│       │                │  │ (articles,  │  │ (full-text      │   │         │
│       │                │  │  sources,   │  │  search, auto-  │   │         │
│       │                │  │  claims)    │  │  complete)      │   │         │
│       │                │  └─────────────┘  └─────────────────┘   │         │
│       │                │                                          │         │
│       │                └──────────────────────────────────────────┘         │
│       │                                                                     │
│       └─────────────────────── JSON Response ──────────────────────────────┘
│                                                                             │
│  Response includes:                                                         │
│  - Article content with source attribution                                  │
│  - Political bias scores and transparency data                              │
│  - Related articles and story clusters                                      │
│  - Fact-check verdicts for embedded claims                                  │
│  - Pagination cursors for infinite scroll                                   │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘

Fact-Checking Pipeline#

text
┌─────────────────────────────────────────────────────────────────────────────┐
│                       FACT-CHECKING PIPELINE                                │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│  ┌──────────────┐          ┌──────────────────┐                            │
│  │  Ingested    │─────────►│  Claim Extraction │                            │
│  │  Article     │          │  (NLP Module)     │                            │
│  └──────────────┘          └────────┬─────────┘                            │
│                                     │                                       │
│                    Extracted claims (factual statements)                     │
│                                     │                                       │
│                                     ▼                                       │
│                            ┌──────────────────┐                            │
│                            │  Fact-Check      │                            │
│                            │  Agent           │                            │
│                            └────────┬─────────┘                            │
│                                     │                                       │
│              ┌──────────────────────┼──────────────────────┐               │
│              │                      │                      │               │
│              ▼                      ▼                      ▼               │
│     ┌──────────────┐      ┌──────────────┐      ┌──────────────┐          │
│     │  Evidence    │      │  Google      │      │  ClaimBuster │          │
│     │  Retrieval   │      │  Fact Check  │      │  API         │          │
│     │  (Serper/    │      │  Tools API   │      │              │          │
│     │  Brave)      │      │              │      │              │          │
│     └──────┬───────┘      └──────┬───────┘      └──────┬───────┘          │
│            │                     │                      │                  │
│            └─────────────────────┼──────────────────────┘                  │
│                                  │                                         │
│                      Evidence aggregation                                  │
│                                  │                                         │
│                                  ▼                                         │
│                         ┌──────────────────┐                               │
│                         │  LLM Verdict     │                               │
│                         │  (Claude/GPT-4)  │                               │
│                         │                  │                               │
│                         │  Verdicts:       │                               │
│                         │  - True          │                               │
│                         │  - Mostly True   │                               │
│                         │  - Mixed         │                               │
│                         │  - Mostly False  │                               │
│                         │  - False         │                               │
│                         │  - Unverifiable  │                               │
│                         └────────┬─────────┘                               │
│                                  │                                         │
│                                  ▼                                         │
│                         ┌──────────────────┐                               │
│                         │  claims table    │                               │
│                         │  (PostgreSQL)    │                               │
│                         │                  │                               │
│                         │  Verdict + score │                               │
│                         │  Evidence links  │                               │
│                         │  Source refs     │                               │
│                         └──────────────────┘                               │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘

Media Generation Pipeline#

text
┌─────────────────────────────────────────────────────────────────────────────┐
│                       MEDIA GENERATION FLOW                                 │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│                   ┌──────────────────┐                                      │
│                   │    articles      │                                      │
│                   │   (PostgreSQL)   │                                      │
│                   └────────┬─────────┘                                      │
│                            │                                                │
│             ┌──────────────┴──────────────┐                                │
│             │                             │                                │
│             ▼                             ▼                                │
│   ┌──────────────────┐          ┌──────────────────┐                      │
│   │  Audio Queue     │          │  Video Queue     │                      │
│   │  Producer        │          │  Producer        │                      │
│   └────────┬─────────┘          └────────┬─────────┘                      │
│            │                             │                                │
│            ▼                             ▼                                │
│   ┌──────────────────┐          ┌──────────────────┐                      │
│   │ yaa:audio-       │          │ yaa:video-       │                      │
│   │ generation       │          │ generation       │                      │
│   │ (BullMQ/Redis)   │          │ (BullMQ/Redis)   │                      │
│   └────────┬─────────┘          └────────┬─────────┘                      │
│            │                             │                                │
│            ▼                             ▼                                │
│   ┌──────────────────┐          ┌──────────────────┐                      │
│   │  Audio Worker    │          │  Video Worker    │                      │
│   │                  │          │                  │                      │
│   │  ┌────────────┐  │          │  ┌────────────┐  │                      │
│   │  │ ElevenLabs │  │          │  │   HeyGen   │  │                      │
│   │  │    TTS     │  │          │  │  Avatar    │  │                      │
│   │  └────────────┘  │          │  └────────────┘  │                      │
│   │  ┌────────────┐  │          │  ┌────────────┐  │                      │
│   │  │ Ghana NLP  │  │          │  │  Shotstack │  │                      │
│   │  │    TTS     │  │          │  │  Renderer  │  │                      │
│   │  └────────────┘  │          │  └────────────┘  │                      │
│   │                  │          │  ┌────────────┐  │                      │
│   │                  │          │  │  Caption   │  │                      │
│   │                  │          │  │  Service   │  │                      │
│   │                  │          │  └────────────┘  │                      │
│   └────────┬─────────┘          └────────┬─────────┘                      │
│            │                             │                                │
│            └──────────────┬──────────────┘                                │
│                           ▼                                               │
│                  ┌──────────────────┐                                     │
│                  │  MinIO / S3      │                                     │
│                  │  (Media Storage) │                                     │
│                  └──────────────────┘                                     │
│                           │                                               │
│            ┌──────────────┼──────────────┐                                │
│            │              │              │                                │
│            ▼              ▼              ▼                                │
│   ┌──────────────┐ ┌──────────┐ ┌──────────────┐                        │
│   │  YouTube     │ │  TikTok  │ │  Instagram   │                        │
│   │  Upload      │ │  Upload  │ │  Upload      │                        │
│   └──────────────┘ └──────────┘ └──────────────┘                        │
│                                                                           │
└─────────────────────────────────────────────────────────────────────────────┘

AI Agent Communication Flow#

text
┌─────────────────────────────────────────────────────────────────────────────┐
│                       AGENT COMMUNICATION FLOW                              │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│                         ┌─────────────────┐                                │
│                         │   Editor-in-    │                                │
│                         │   Chief Agent   │                                │
│                         │  (Orchestrator) │                                │
│                         └────────┬────────┘                                │
│                                  │                                         │
│                                  │ broadcast()                             │
│                                  ▼                                         │
│                         ┌─────────────────┐                                │
│                         │   MessageBus    │                                │
│                         │ (Redis Pub/Sub) │                                │
│                         └────────┬────────┘                                │
│                                  │                                         │
│           ┌──────────────────────┼──────────────────────┐                  │
│           │                      │                      │                  │
│           ▼                      ▼                      ▼                  │
│  ┌─────────────────┐   ┌─────────────────┐   ┌─────────────────┐          │
│  │  Journalism     │   │   Fact-Check    │   │   Editorial     │          │
│  │  Agent          │   │   Agent         │   │   Agent         │          │
│  │                 │   │                 │   │                 │          │
│  │  - Draft        │   │  - Verify       │   │  - Copy edit    │          │
│  │  - Source check │   │  - Evidence     │   │  - Style        │          │
│  │  - Story dev    │   │  - Verdict      │   │  - Headlines    │          │
│  └────────┬────────┘   └────────┬────────┘   └────────┬────────┘          │
│           │                      │                      │                  │
│           │ send()               │ send()               │ send()           │
│           │                      │                      │                  │
│           └──────────────────────┼──────────────────────┘                  │
│                                  │                                         │
│                                  ▼                                         │
│                         ┌─────────────────┐                                │
│                         │   Editor-in-    │                                │
│                         │   Chief Agent   │                                │
│                         │  (aggregates    │                                │
│                         │   results)      │                                │
│                         └────────┬────────┘                                │
│                                  │                                         │
│                                  ▼                                         │
│                         ┌─────────────────┐                                │
│                         │  Social Media   │                                │
│                         │  Agent          │                                │
│                         │                 │                                │
│                         │  Distributes    │                                │
│                         │  across:        │                                │
│                         │  - WhatsApp     │                                │
│                         │  - YouTube      │                                │
│                         │  - TikTok       │                                │
│                         │  - Instagram    │                                │
│                         │  - X/Twitter    │                                │
│                         └─────────────────┘                                │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘

Communication Patterns#

Synchronous Communication#

Pattern Use Case Example
HTTP/REST Client-to-API requests Web PWA -> Veritas API
HTTP/REST Service-to-external-API API -> HeyGen, ElevenLabs
SQL Direct database queries API -> PostgreSQL (raw SQL, pg)
HTTP Search queries API -> Elasticsearch

Asynchronous Communication#

Pattern Use Case Example
BullMQ (Redis) Content processing pipeline Scheduler -> Producer -> Worker
BullMQ (Redis) Audio generation jobs API -> Audio Queue -> ElevenLabs Worker
BullMQ (Redis) Video generation jobs API -> Video Queue -> HeyGen Worker
Redis Pub/Sub Agent-to-agent messaging Orchestrator <-> Journalism Agent
Redis Pub/Sub Event broadcasting Article published -> Social distribution
Webhooks External callback handling MTN MoMo payment status -> API callback

Cross-Cutting Communication Matrix#

From / To Database BullMQ Redis PubSub REST API External API
Ingestion Write (feed_items) Enqueue (content-processing) -- -- RSS/Sitemap
Processing Worker Read/Write Consume -- -- --
Audio Worker Read/Write Consume -- -- ElevenLabs, Ghana NLP
Video Worker Read/Write Consume -- -- HeyGen, Shotstack
AI Agents Read -- Pub/Sub -- Anthropic, OpenAI
API Server Read/Write -- Cache Serve Various
Social Scheduler Read Enqueue/Consume -- -- YouTube, TikTok, etc.

API Architecture#

Layered Structure#

The Veritas API follows a four-layer architecture:

text
┌──────────────────────────────────────────────────────────────────────┐
│  INTERFACES LAYER (HTTP)                                             │
│  - Route handlers (29 files)                                         │
│  - Middleware (logging, security, rate limiting, auth)                │
│  - Zod request/response validation                                   │
│  - OpenAPI specification generation (1,079 lines)                    │
│  - Error response formatting                                         │
├──────────────────────────────────────────────────────────────────────┤
│  APPLICATION LAYER                                                   │
│  - Cursor-based pagination logic                                     │
│  - Use case orchestration                                            │
├──────────────────────────────────────────────────────────────────────┤
│  DOMAIN LAYER (11,669 lines)                                         │
│  - Subscriptions (2,422 lines: tiers, pricing, family, student)      │
│  - B2B API (1,681 lines: white-label, partner access)                │
│  - Ad Networks (2,299 lines: Google AdSense, Facebook AN)            │
│  - Advertisements (1,293 lines: scheduling, targeting)               │
│  - API Keys (985 lines: generation, rotation, usage)                 │
│  - Compliance (300+ lines: election, NMC, DPA)                       │
│  - Payments (Stripe, MTN MoMo, Vodafone Cash, AirtelTigo)            │
│  - Sources (bias scoring, ownership, regional emphasis)              │
│  - Analytics (1,504 lines: conversion, churn, cohorts)               │
│  - Reading Mode (689 lines), Paywall (796 lines)                     │
├──────────────────────────────────────────────────────────────────────┤
│  INFRASTRUCTURE LAYER (416 lines)                                    │
│  - PostgreSQL connection pool (configurable max, idle, timeout)      │
│  - Redis client with retry strategy and lazy connect                 │
│  - Elasticsearch client with auth and TLS                            │
│  - JWT token management (custom HMAC-SHA256, no external lib)        │
│  - AES-256-GCM encryption for sensitive data (anonymous tips)        │
└──────────────────────────────────────────────────────────────────────┘

Middleware Chain (Execution Order)#

text
Request
  │
  ├──► 1. Logger (hono/logger) ............... Request method, path, timing
  ├──► 2. Timing (hono/timing) ............... Server-Timing header
  ├──► 3. Security Headers (hono/secure-headers) X-Frame-Options, CSP, etc.
  ├──► 4. CORS (hono/cors) ................... Configurable origins
  ├──► 5. Rate Limiting (custom) ............. Redis sliding window, 120/60s
  ├──► 6. API Key Auth (custom) .............. Timing-safe comparison
  └──► 7. Route Handler
          │
          ├──► Zod validation (body, query, params)
          ├──► Business logic
          └──► JSON response
              │
              └──► Error Handler (HTTPException, ZodError, generic)

Rate Limiting#

  • Algorithm: Sliding window using Redis sorted sets
  • Default: 120 requests per 60-second window
  • Fallback: In-memory store if Redis unavailable
  • Client ID: X-Forwarded-For -> X-Real-IP -> "unknown"
  • Response Headers: X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset
  • 429 Response: { "error": "Too Many Requests", "retryAfterSeconds": <n> }

Database Architecture#

Schema Overview#

The database uses 50+ tables across 63 migrations, organized into the following domains:

Core Content Tables#

Table Purpose
sources News source metadata (300+ Ghanaian outlets)
articles Published article content
authors Article author profiles
feed_items Raw ingested RSS/Atom content
normalized_feed_items Processed, deduplicated content
content_fingerprints Simhash64 fingerprints for dedup
content_dedupe_mappings Duplicate item -> canonical mappings

Taxonomy & Classification#

Table Purpose
categories Content categories (politics, business)
topics Fine-grained topics
tags Keyword tags from extraction
article_categories Article-to-category junction
article_topics Article-to-topic junction
article_tags Article-to-tag junction

Fact-Checking & Trust#

Table Purpose
claims Extracted factual claims
article_political_bias_scores Political bias analysis per article
source_reliability_scores Source credibility tracking
misinformation_flags Content flagging

Users & Engagement#

Table Purpose
users User accounts
subscriptions Subscription tiers and status
user_preferences Reading preferences, feed customization
article_views View tracking for analytics
api_keys B2B API key management

Multimedia Production#

Table Purpose
voice_profiles ElevenLabs voice configurations
avatar_profiles HeyGen avatar configurations
audio_renders Generated audio file references
video_renders Generated video file references

Custom ENUM Types#

sql
source_kind:     'newspaper' | 'radio' | 'tv' | 'online' | 'wire' | 'government' | 'blog'
article_status:  'draft' | 'review' | 'published' | 'archived' | 'retracted'
media_type:      'image' | 'video' | 'audio' | 'document'
fact_check_verdict: 'true' | 'mostly_true' | 'mixed' | 'mostly_false' | 'false' | 'unverifiable'
subscription_tier:  'free' | 'premium' | 'student' | 'family'

Database Access Pattern#

Veritas uses direct SQL queries with the pg library (no ORM):

typescript
// Example: Article listing with source join
const { rows } = await pool.query<ArticleRow>(
  `SELECT a.id, a.title, a.slug, a.content, a.published_at,
          s.name as source_name, s.canonical_domain
   FROM articles a
   LEFT JOIN sources s ON a.source_id = s.id
   WHERE a.status = 'published'
   ORDER BY a.published_at DESC
   LIMIT $1 OFFSET $2`,
  [limit, offset]
);

Redis Key Patterns#

Pattern Purpose
veritas:api:rate-limit:* Sliding window rate limit state
veritas:content-processing BullMQ content processing queue
veritas:audio-generation BullMQ audio generation queue
veritas:video-generation BullMQ video generation queue
veritas:cache:* API response caching

Deployment Architecture#

Kubernetes / ECS Topology#

text
┌──────────────────────────────────────────────────────────────────────────────┐
│                        AWS INFRASTRUCTURE                                    │
│                                                                              │
│  ┌────────────────────────────────────────────────────────────────────────┐  │
│  │                         INGRESS LAYER                                  │  │
│  │  ┌──────────────┐  ┌──────────────┐  ┌────────────────────────────┐   │  │
│  │  │ ALB / nginx  │  │ WAF / Shield │  │ SSL/TLS (ACM)              │   │  │
│  │  │ Ingress      │  │              │  │ CloudFront CDN             │   │  │
│  │  └──────┬───────┘  └──────────────┘  └────────────────────────────┘   │  │
│  └─────────┼─────────────────────────────────────────────────────────────┘  │
│            │                                                                 │
│  ┌─────────┼─────────────────────────────────────────────────────────────┐  │
│  │         ▼                    APPLICATION LAYER                         │  │
│  │                                                                       │  │
│  │  ┌──────────┐  ┌──────────────┐  ┌──────────────────────────────┐    │  │
│  │  │ Veritas  │  │  Veritas     │  │  Veritas Ingestion           │    │  │
│  │  │  Web     │  │   API        │  │                              │    │  │
│  │  │ (Next.js)│  │  (Hono)      │  │  ┌───────────┐ ┌─────────┐  │    │  │
│  │  │          │  │              │  │  │ Scheduler │ │ Worker  │  │    │  │
│  │  │ 2-10     │  │  2-10        │  │  │ (1 pod)   │ │(2-8 pods│  │    │  │
│  │  │ replicas │  │  replicas    │  │  └───────────┘ └─────────┘  │    │  │
│  │  └──────────┘  └──────────────┘  └──────────────────────────────┘    │  │
│  │                                                                       │  │
│  └───────────────────────────────────────────────────────────────────────┘  │
│            │              │                     │                            │
│  ┌─────────┼──────────────┼─────────────────────┼────────────────────────┐  │
│  │         ▼              ▼                     ▼                        │  │
│  │                       DATA LAYER                                      │  │
│  │                                                                       │  │
│  │  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐                │  │
│  │  │ PostgreSQL   │  │    Redis     │  │Elasticsearch │                │  │
│  │  │ (RDS or      │  │ (ElastiCache │  │ (Managed or  │                │  │
│  │  │  StatefulSet)│  │  or pod)     │  │  OpenSearch) │                │  │
│  │  └──────────────┘  └──────────────┘  └──────────────┘                │  │
│  │                                                                       │  │
│  │  ┌──────────────────────────────────────────────────────────────┐     │  │
│  │  │                    S3 / MinIO                                 │     │  │
│  │  │              Media asset storage                             │     │  │
│  │  └──────────────────────────────────────────────────────────────┘     │  │
│  └───────────────────────────────────────────────────────────────────────┘  │
│                                                                              │
│  ┌───────────────────────────────────────────────────────────────────────┐  │
│  │                    OBSERVABILITY LAYER                                 │  │
│  │  ┌────────────┐  ┌────────────┐  ┌────────────┐  ┌────────────┐      │  │
│  │  │ Prometheus │  │  Grafana   │  │   Jaeger   │  │ CloudWatch │      │  │
│  │  └────────────┘  └────────────┘  └────────────┘  └────────────┘      │  │
│  └───────────────────────────────────────────────────────────────────────┘  │
└──────────────────────────────────────────────────────────────────────────────┘

GitOps Flow (ArgoCD)#

text
┌──────────────┐    push    ┌──────────────┐    sync    ┌──────────────┐
│  Developer   │ ──────────►│    GitHub     │ ◄────────►│   ArgoCD     │
│              │            │              │           │              │
└──────────────┘            └──────────────┘           └──────┬───────┘
                                                              │
┌─────────────────────────────────────────────────────────────┼────────┐
│                                                              ▼        │
│                        KUBERNETES CLUSTER                             │
│                                                                      │
│  deploy/argocd/                                                      │
│       │                                                              │
│       ├──► apps/dev/veritas-api.yaml                                 │
│       ├──► apps/dev/veritas-web.yaml                                 │
│       ├──► apps/dev/veritas-ingestion.yaml                           │
│       │                                                              │
│       ├──► apps/staging/veritas-api.yaml                             │
│       ├──► apps/staging/veritas-web.yaml                             │
│       │                                                              │
│       └──► apps/prod/veritas-api.yaml                                │
│            apps/prod/veritas-web.yaml                                 │
│            apps/prod/veritas-ingestion.yaml                           │
│                                                                      │
└──────────────────────────────────────────────────────────────────────┘

CI/CD Pipeline (GitHub Actions)#

text
┌──────────────────────────────────────────────────────────────────────┐
│                          CI PIPELINE                                  │
│                                                                      │
│  Push / PR                                                           │
│  │                                                                   │
│  ├──► Quality Checks                                                 │
│  │    ├── Lint (ESLint)                                              │
│  │    ├── Type Check (TypeScript)                                    │
│  │    └── Domain Boundary Validation                                 │
│  │                                                                   │
│  ├──► Build                                                          │
│  │    ├── nx build veritas-api                                       │
│  │    ├── nx build veritas-web                                       │
│  │    └── nx build veritas-ingestion                                 │
│  │                                                                   │
│  ├──► Tests                                                          │
│  │    ├── Unit Tests (Vitest)                                        │
│  │    ├── Integration Tests                                          │
│  │    └── Security Audit (npm audit)                                 │
│  │                                                                   │
│  ├──► Container Build                                                │
│  │    ├── Multi-arch Docker (amd64, arm64)                           │
│  │    └── Trivy security scanning                                    │
│  │                                                                   │
│  └──► Deploy (main branch)                                           │
│       ├── Staging (auto-deploy)                                      │
│       ├── Canary (10% traffic)                                       │
│       └── Production (manual approval)                               │
└──────────────────────────────────────────────────────────────────────┘

Terraform Infrastructure#

Infrastructure is managed via Terraform in infra/terraform/:

Component Purpose
VPC Network isolation, public/private subnets
ECS Container orchestration (Fargate)
ALB Load balancing with health checks
ECR Docker image registry
S3 Media storage, static assets
SQS/SNS Message queuing and notifications
CloudWatch Logging, metrics, alarms
Secrets Manager API keys, database credentials
SSM Parameter Configuration management

Security Architecture#

Authentication Flow#

text
┌────────┐   1. Login      ┌──────────────┐   2. Validate    ┌──────────┐
│ Client │ ───────────────►│  Veritas API │ ────────────────►│PostgreSQL│
│ (PWA)  │                 │  /auth/login │                  │ (users)  │
└────────┘                 └──────┬───────┘                  └──────────┘
     ▲                            │
     │                            │ 3. Generate JWT
     │                            │    (HMAC-SHA256, custom impl)
     │                            │    TTL: 7 days
     │   4. Bearer Token          │
     └────────────────────────────┘

Authorization Model#

Role Permissions
Anonymous Read published articles, search, view sources, browse taxonomy
Reader + Personalized feed, preferences, bookmarks
Premium + Full article access, ad-free, audio/video, advanced search
Student + Discounted premium access with .edu verification
Family + Up to 5 family members, shared subscription
Journalist + Writing assistant, source management, draft articles
B2B Partner + API key access, white-label endpoints, bulk data
Admin + User management, compliance, system configuration

Data Protection#

Layer Protection
In Transit TLS 1.2+, HTTPS only
At Rest AES-256-GCM (tips encryption), RDS encryption
Secrets AWS Secrets Manager, SSM Parameter Store
PII Anonymized analytics, encrypted tips
API Keys Timing-safe comparison, rotation support
Rate Limits Per-IP sliding window, Redis-backed

Compliance Framework#

Regulation Implementation
Ghana Data Protection Act 2012 Data protection endpoints, consent management
GDPR (for EU users) Right to erasure, data export
Ghana NMC Guidelines Political coverage balance, fair reporting
Ghana Electoral Commission 24h campaign silence, equal coverage enforcement
Ghana Criminal Offences Act False news detection, election offense monitoring

Observability Stack#

Metrics (Prometheus/CloudWatch)#

yaml
Service Metrics:
  - http_request_duration_seconds (p50, p95, p99)
  - http_requests_total (by status, method, route)
  - rate_limit_hits_total
  - api_key_auth_failures_total

Application Metrics:
  - articles_ingested_total
  - articles_published_total
  - fact_checks_completed_total
  - search_queries_total
  - subscriptions_active_gauge
  - payments_processed_total

Queue Metrics:
  - bullmq_jobs_active
  - bullmq_jobs_completed
  - bullmq_jobs_failed
  - bullmq_jobs_delayed
  - bullmq_queue_depth

Infrastructure Metrics:
  - container_cpu_usage
  - container_memory_usage
  - database_connections_active
  - redis_connected_clients
  - elasticsearch_query_latency

Logging (Structured JSON)#

json
{
  "level": "info",
  "service": "veritas-api",
  "timestamp": "2026-01-26T12:00:00.000Z",
  "trace_id": "abc123",
  "method": "GET",
  "path": "/api/v1/articles",
  "status_code": 200,
  "duration_ms": 45,
  "user_id": "usr_xxx",
  "rate_limit_remaining": 118
}

Health Checks#

Endpoint Purpose
GET /health Basic liveness (always returns 200)
GET /healthz Kubernetes liveness probe
GET /readyz Readiness (checks DB, Redis, Elasticsearch)
GET /health/detailed Full status with component health breakdown

Performance Targets#

Component Metric Target
API - Article list p50 latency < 100ms
API - Article list p95 latency < 300ms
API - Search p50 latency < 200ms
API - Search p95 latency < 500ms
API - Auth p95 latency < 150ms
API - Fact check p95 latency < 2s
Web PWA Lighthouse score > 90
Web PWA LCP < 2.5s (4G)
Web PWA FID < 100ms
Web PWA CLS < 0.1
Ingestion RSS collection 300 feeds / tick
Ingestion Processing latency < 30s per item
CDN Static asset TTFB < 50ms (cached)
Rate Limit Max requests 120 / 60s window

Scaling Configuration#

Horizontal Pod Autoscaler#

yaml
# Veritas API HPA
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: veritas-api
  minReplicas: 2
  maxReplicas: 10
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70
    - type: Resource
      resource:
        name: memory
        target:
          type: Utilization
          averageUtilization: 80

Resource Defaults#

yaml
# API Service
resources:
  requests:
    memory: "256Mi"
    cpu: "200m"
  limits:
    memory: "1Gi"
    cpu: "1000m"

# Ingestion Worker
resources:
  requests:
    memory: "512Mi"
    cpu: "200m"
  limits:
    memory: "2Gi"
    cpu: "1000m"

# Web (Next.js)
resources:
  requests:
    memory: "256Mi"
    cpu: "100m"
  limits:
    memory: "512Mi"
    cpu: "500m"

Database Connection Pools#

Service Max Connections Idle Timeout Connect Timeout
API 20 30s 5s
Ingestion 10 30s 10s

Technology Stack#

Layer Technology
Runtime Node.js 22+ (LTS)
Language TypeScript 6.0.3 (ESM)
API Framework Hono 4.x (edge-compatible, lightweight)
Web Framework Next.js 15 (App Router, RSC, SSR)
Styling Tailwind CSS 3.x
i18n next-intl (6 Ghanaian locales)
Validation Zod 3.x
Database PostgreSQL 16 (with pgvector)
Cache/Queue Redis 7.x (via ioredis)
Job Queue BullMQ 5.x
Search Elasticsearch 8.x
Object Storage MinIO / AWS S3
Authentication Custom JWT (HMAC-SHA256) + NextAuth
AI/LLM Anthropic Claude, OpenAI GPT-4, Cohere
TTS ElevenLabs, Ghana NLP
Video HeyGen, Shotstack
NLP HuggingFace Inference, spaCy (Python NER)
Testing Vitest, Playwright
Build Nx 20.x with esbuild
CI/CD GitHub Actions (18 workflows)
IaC Terraform (80+ .tf files)
GitOps ArgoCD 2.x (100+ manifests)
Container Docker (multi-arch: amd64, arm64)
Orchestration Kubernetes / AWS ECS Fargate
CDN CloudFront / Cloudflare
Monitoring Prometheus, Grafana, Jaeger, CloudWatch

External API Integrations#

AI & Machine Learning#

Provider Purpose Integration Point
Anthropic Claude for abstractive summarization, fact-check NLP, AI Agents
OpenAI GPT-4 for content generation, embeddings NLP, AI Agents
Cohere Embeddings, reranking NLP Embedding Service
HuggingFace Sentiment analysis, topic classification NLP Services

Media Production#

Provider Purpose Integration Point
HeyGen AI avatar video generation Video Production
Shotstack Video rendering and composition Video Production
ElevenLabs High-quality TTS voice synthesis Audio Production
Ghana NLP Ghanaian language TTS, translation, NER NLP, Audio

Search & Fact-Checking#

Provider Purpose Integration Point
Serper Web search for evidence retrieval Evidence Retrieval
Brave Search Alternative web search Evidence Retrieval
Google Fact Check Existing fact-check database Fact-Check Agent
ClaimBuster Claim worthiness scoring Claim Extraction

Payments (Ghana-Focused)#

Provider Purpose Integration Point
Stripe International card payments Payments Domain
MTN MoMo Mobile money (MTN Ghana) Payments Domain
Vodafone Cash Mobile money (Vodafone Ghana) Payments Domain
AirtelTigo Money Mobile money (AirtelTigo Ghana) Payments Domain

Social Distribution#

Platform Purpose Integration Point
YouTube Video and Shorts publishing Social Automation
TikTok Short-form video publishing Social Automation
Instagram Stories and Reels publishing Social Automation
WhatsApp Channel and business messaging Social Automation
X (Twitter) News distribution Social Automation
Telegram Channel publishing Social Automation
LinkedIn Professional content distribution Social Automation
Facebook Page publishing Social Automation


Last Updated: January 2026

Part of the Veritas domain architecture documentation