# blue-jays/news_bias_check_companion architecture

> Architecture diagram of the GitHub repository blue-jays/news_bias_check_companion, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-11-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/blue-jays/news_bias_check_companion
- Repository: https://github.com/blue-jays/news_bias_check_companion

## Overview

##### Project Type and Purpose
- This is a data-driven full-stack system for ingesting, analyzing, and bias-scoring news articles, with an accompanying browser extension UI, a REST API backend, ML feature pipelines, orchestrated data workflows, data validation, and monitoring/observability.

##### Main Architectural Components
a. Browser Extension
- Files under `extension/` (manifest.json, content_script.js, popup & options UI)
- Provides user interface for fetching or displaying bias scores in the browser.
b. API Service (Backend)
- FastAPI application in `src/app/` (routers, service modules, DB layer, telemetry)
- Exposes endpoints for health checks, article analysis, source management.
c. Database
- SQL database accessed via `src/app/db/*.py` and configured in `configs/app.yaml`
- Stores articles, sources, analysis results.
d. CLI Tool
- `src/cli/main.py` for ad-hoc commands or dev workflows.
e. ML Pipelines
- Extraction, feature generation, bias-ranking models under `src/ml/`
- Topic extraction, sentiment features, vectorization, model inference.
f. Orchestrators & Pipelines
- `src/orchestrators/actions/nightly_fetch.py` triggers scheduled tasks.
- `src/pipelines/{ingest,preprocess,register,evaluate}.py` define ETL/data-science steps.
g. Data Validation
- Great Expectations under `validation/great_expectations/` for schema and data-quality checks.
h. Infrastructure & Monitoring
- `infra/prometheus.yml`, `infra/grafana/` dashboards collect metrics from API and pipelines.
i. Configuration & Deployment
- Dockerfile, `compose.yaml` for local or containerized deployment.
- GitHub Actions CI and nightly‐fetch jobs in `.github/workflows/`.

##### Component Relationships and Data Flows
- Browser Extension → (HTTPS) → API Service → (SQL queries) → Database
- Orchestrator Scheduler → Python scripts (pipelines) →
- Fetch articles from external news sources (`configs/sources.yaml`)
- Preprocess & load into Database → Validate with Great Expectations →
- Feature extraction & model inference → Write results back to Database
- API Service invokes ML service modules for on-demand analysis.
- Monitoring agents scrape metrics from API and pipeline processes → Prometheus → Grafana dashboards.
- CLI Tool may directly invoke pipelines or API for testing/dev.

##### Architectural Patterns & Principles
- Separation of concerns: UI, API, data, ML, orchestration, monitoring are discrete.
- Modular ML: feature extraction, drift detection, bias scoring are pluggable under `src/ml/`.
- Configuration-driven deployment: YAML files in `configs/`, `compose.yaml`.
- Observability by default: logging, metrics, dashboards.

## Diagram (Mermaid)

```mermaid
flowchart TD
  %% Frontend Layer
  subgraph "Frontend Layer"
    BE["Browser Extension"]:::frontend
  end

  %% Backend/API Layer
  subgraph "Backend/API Layer"
    API["FastAPI Service"]:::backend
    CLI["CLI Tool"]:::backend
  end

  %% Data Layer
  subgraph "Data Layer"
    DB["PostgreSQL DB"]:::data
    VAL["Great Expectations"]:::data
  end

  %% ML Services
  subgraph "ML Services"
    EXTRACT["Data Extraction & Topic Modeling"]:::compute
    DRIFT["Drift Detection"]:::compute
    FEATURES["Feature Generation"]:::compute
    INFER["Model Inference"]:::compute
  end

  %% Orchestration & Pipelines
  subgraph "Orchestration & Pipelines"
    ORCH["Orchestrator (nightly_fetch)"]:::compute
    PIPE_ING["Ingest Pipeline"]:::compute
    PIPE_PRE["Preprocess Pipeline"]:::compute
    PIPE_REG["Register Pipeline"]:::compute
    PIPE_EVAL["Evaluate Pipeline"]:::compute
  end

  %% Monitoring & Infra
  subgraph "Monitoring & Infra"
    PROM["Prometheus"]:::infra
    GRAF["Grafana"]:::infra
  end

  %% Connections
  BE -->|HTTP/JSON| API
  CLI -->|invoke| API
  API -->|SQL Queries| DB
  VAL -->|validate data| DB
  ORCH -->|schedule| PIPE_ING
  PIPE_ING -->|writes| DB
  PIPE_ING -->|loads| VAL
  PIPE_ING -->|feeds| EXTRACT
  EXTRACT -->|features| DB
  DRIFT -->|metrics| DB
  FEATURES -->|features| DB
  INFER -->|results| DB
  API -->|calls ML modules| INFER

  %% Monitoring metrics flow
  API -.-> PROM
  ORCH -.-> PROM
  PIPE_ING -.-> PROM
  PROM -->|dashboard data| GRAF

  %% Click Events
  click BE "https://github.com/blue-jays/news_bias_check_companion/tree/main/extension/"
  click API "https://github.com/blue-jays/news_bias_check_companion/blob/main/src/app/api/main.py"
  click CLI "https://github.com/blue-jays/news_bias_check_companion/blob/main/src/cli/main.py"
  click ORCH "https://github.com/blue-jays/news_bias_check_companion/blob/main/src/orchestrators/actions/nightly_fetch.py"
  click PIPE_ING "https://github.com/blue-jays/news_bias_check_companion/blob/main/src/pipelines/ingest.py"
  click PIPE_PRE "https://github.com/blue-jays/news_bias_check_companion/blob/main/src/pipelines/preprocess.py"
  click PIPE_REG "https://github.com/blue-jays/news_bias_check_companion/blob/main/src/pipelines/register.py"
  click PIPE_EVAL "https://github.com/blue-jays/news_bias_check_companion/blob/main/src/pipelines/evaluate.py"
  click VAL "https://github.com/blue-jays/news_bias_check_companion/tree/main/validation/great_expectations/"
  click PROM "https://github.com/blue-jays/news_bias_check_companion/blob/main/infra/monitoring/prometheus.yml"
  click GRAF "https://github.com/blue-jays/news_bias_check_companion/blob/main/infra/grafana/dashboards/bias_dashboard.json"

  %% Styles
  classDef frontend fill:#D0E6FF,stroke:#0366D6,color:#0366D6
  classDef backend fill:#FFE0B2,stroke:#E65100,color:#E65100
  classDef data fill:#C8E6C9,stroke:#2E7D32,color:#2E7D32
  classDef compute fill:#FFE0B2,stroke:#F57C00,color:#F57C00
  classDef infra fill:#E1BEE7,stroke:#6A1B9A,color:#6A1B9A
```
