# kanaries/rath architecture

> Architecture diagram of the GitHub repository kanaries/rath, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kanaries/rath
- Repository: https://github.com/kanaries/rath

## Overview

##### Project type and purpose
- This is a monorepo for **RATH**, an open-source, full-stack, automated EDA & BI platform. It includes:
- A **web client** (React/TypeScript, MobX, Web Workers, Vega)
- A **desktop client** (Electron/Node.js)
- Multiple **backend microservices** (Python Flask, AWS Lambda functions)
- **Data connectors** to SQL/NoSQL sources and various analytics engines

##### File structure
- **Root/**
– Docker orchestration: `docker-compose.yml`, `client.dockerfile`, Nginx config
– CI/CD: `.github/workflows/*.yml`, `buildspec.yml`
- **apps/**
– `test-datasets/`: sample SQL files
- **packages/** (Lerna/workspaces)
– `rath-client/`: full React SPA with extensive modules, pages, stores, services, and Web Workers
– `olap-connector/`: backend–for–frontend proxy layer to various OLAP engines
- **services/**
– `connector/`: Python Flask + AWS Lambda handlers for database connectivity
– `causal-service/`: Python microservice for causal discovery (multiple algorithms)
– `narrative-service/`: Jupyter/Flask service for narrative insight generation
– `prediction/`: Python classification & regression service
– `text-pattern-service/`: Python service for pattern extraction

##### README for insights
- Purpose: automate EDA workflow (AutoPilot, Copilot, Natural-Language Interface)
- Features: data profiling, data wrangling, one-click data insight, data painting, causal analysis, dashboard generation
- Supported Sources: Athena, Redshift, Spark, Doris, ClickHouse, MySQL, PostgreSQL, etc.

##### Main components & interactions
a. Components
- **Desktop Client** (Electron)
- **Web Client** (React SPA + Web Workers)
- **API Gateway / Reverse Proxy** (Nginx)
- **OLAP Connector Service** (proxies queries to external DBs)
- **Microservices**:
– Causal-Service
– Narrative-Service
– Prediction-Service
– Text-Pattern-Service
- **External Data Sources** (various SQL/NoSQL DBs)

b. Relationships & data flow
1. **Client** (desktop or browser) issues HTTP/WS requests → **Nginx**
2. Nginx routes to:
– **OLAP Connector** for CRUD and query operations on external DBs
– **Causal/Narrative/Prediction/Text services** for analysis tasks
3. **OLAP Connector** translates requests → executes on target DB → returns results
4. **Analysis services** compute models/insights and return structured JSON
5. **Client** receives data → employs Web Workers & Vega libs to render visualizations

c. Architectural patterns & design principles
- **Monorepo & workspaces** for shared code and consistent versioning
- **Microservices**: separation of analytical engines by concern (causal, narrative, prediction…)
- **Backend-for-Frontend** (OLAP connector) abstracts DB connectivity
- **Containerization** (Docker Compose) for local integration testing
- **Web Workers** to offload heavy data processing in the browser
- **MVVM/MVC-like** separation in the React client (pages, components, stores, services)

d. Key technologies/frameworks
- Frontend: React, TypeScript, MobX, Webpack, Babel, CSS modules, Web Workers
- Visualization: Vega-Lite scenegraph, WebGL renderer
- Desktop: Electron, NSIS installer
- Backend: Python (Flask), AWS Lambda handlers, Jupyter notebooks (narrative service)
- Containerization: Docker, Nginx, Docker-Compose
- CI/CD: GitHub Actions, AWS CodeBuild (`buildspec.yml`)

##### Detailed capture of essential elements
- **monorepo** structure at a high level (packages vs apps vs services)
- **Web Worker** threads offloading data parsing/analysis
- **Docker Compose** orchestration layering (Nginx + services)
- **AWS Lambda** functions under Connector as a sub-component

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Clients Layer
    subgraph "Clients" 
        direction TB
        DesktopClient["Electron Desktop Client"]:::client
        WebClient["Web Client (React SPA)"]:::client
        WebWorker["Web Workers"]:::shared
    end

    %% API Gateway
    Nginx["Nginx API Gateway"]:::gateway

    %% Backend Layer
    subgraph "Backend Services"
        direction TB
        OLAPConnector["OLAP Connector Service"]:::service
        subgraph "Microservices"
            direction TB
            Causal["Causal Service"]:::service
            Narrative["Narrative Service"]:::service
            Prediction["Prediction Service"]:::service
            TextPattern["Text-Pattern Service"]:::service
        end
        ConnectorLambdas["Database Connector Lambdas"]:::service
    end

    %% External Systems
    ExternalDBs["External Databases"]:::database
    NLPService["GPT/OpenAI"]:::external

    %% Shared Libraries
    subgraph "Shared Libraries"
        direction TB
        Icons["Icons Library"]:::shared
        Utils["Utilities Library"]:::shared
        VegaRenderer["Vega Renderer"]:::shared
        VegaScenegraph["Vega Scenegraph"]:::shared
    end

    %% Flows
    DesktopClient -->|HTTP/WS requests| Nginx
    WebClient -->|HTTP/WS requests| Nginx
    WebClient -->|spawns| WebWorker
    Nginx -->|routes to| OLAPConnector
    Nginx -->|routes to| Causal
    Nginx -->|routes to| Narrative
    Nginx -->|routes to| Prediction
    Nginx -->|routes to| TextPattern
    OLAPConnector -->|invokes| ConnectorLambdas
    ConnectorLambdas -->|queries| ExternalDBs
    Causal -->|analysis results| Nginx
    Narrative -->|insights| Nginx
    Prediction -->|predictions| Nginx
    TextPattern -->|patterns| Nginx
    WebWorker -->|uses| VegaRenderer
    WebWorker -->|uses| VegaScenegraph
    WebClient -->|imports| Icons
    WebClient -->|imports| Utils
    OLAPConnector -->|imports| Utils
    OLAPConnector -->|imports| Icons
    Causal -->|may call| NLPService
    Narrative -->|may call| NLPService

    %% Click events
    click DesktopClient "https://github.com/kanaries/rath/blob/master/apps/desktop/main.js"
    click WebClient "https://github.com/kanaries/rath/tree/master/packages/rath-client/"
    click Nginx "https://github.com/kanaries/rath/blob/master/docker/nginx.conf"
    click OLAPConnector "https://github.com/kanaries/rath/tree/master/packages/olap-connector/app/"
    click Causal "https://github.com/kanaries/rath/tree/master/services/causal-service/"
    click Narrative "https://github.com/kanaries/rath/tree/master/services/narrative-service/"
    click Prediction "https://github.com/kanaries/rath/tree/master/services/prediction/"
    click TextPattern "https://github.com/kanaries/rath/tree/master/services/text-pattern-service/"
    click ConnectorLambdas "https://github.com/kanaries/rath/tree/master/services/connector/lambda/"
    click Icons "https://github.com/kanaries/rath/tree/master/packages/icons/"
    click Utils "https://github.com/kanaries/rath/tree/master/packages/utils/"
    click VegaRenderer "https://github.com/kanaries/rath/tree/master/packages/vega-renderer/"
    click VegaScenegraph "https://github.com/kanaries/rath/tree/master/packages/vega-scenegraph/"

    %% Styles
    classDef client fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef gateway fill:#D3D3D3,stroke:#333,stroke-width:1px
    classDef service fill:#90EE90,stroke:#333,stroke-width:1px
    classDef database fill:#FFA500,stroke:#333,stroke-width:1px
    classDef shared fill:#DDA0DD,stroke:#333,stroke-width:1px
    classDef external fill:#FFB6C1,stroke:#333,stroke-width:1px
```
