# orionshinesbright/global-gdp-analysis architecture

> Architecture diagram of the GitHub repository orionshinesbright/global-gdp-analysis, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/orionshinesbright/global-gdp-analysis
- Repository: https://github.com/orionshinesbright/global-gdp-analysis

## Overview

This is a configurable, concurrent dataset analysis and visualization pipeline. The main workflow streams CSV/JSON rows through field mapping and integrity checks, computes a sliding average, then emits console output and live charts in a browser dashboard. Separate branches monitor queue backpressure and expose pipeline status and latest-packet data. The sampled entry point shows process wiring and queue flow; unsampled output modules are included because the source index and README identify them, but uncertain internal links are omitted.

Main entry points: [`main.py`](https://github.com/orionshinesbright/global-gdp-analysis/blob/main/main.py) (Bootstrap), [`plugins/output/pipeline_telemetry.py`](https://github.com/orionshinesbright/global-gdp-analysis/blob/main/plugins/output/pipeline_telemetry.py) (Queue telemetry), [`plugins/output/protocols.py`](https://github.com/orionshinesbright/global-gdp-analysis/blob/main/plugins/output/protocols.py) (Output manager), [`stream/Stream.py`](https://github.com/orionshinesbright/global-gdp-analysis/blob/main/stream/Stream.py) (Raw queue)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_orchestration["Pipeline orchestration"]
  node_main["Bootstrap<br/>[main.py]"]
  node_config["Pipeline config<br/>[config.json]"]
  node_rawq["Raw queue<br/>[Stream.py]"]
  node_interq["Intermediate queue<br/>[Stream.py]"]
  node_processedq["Processed queue<br/>[Stream.py]"]
end

subgraph group_processing["Data processing"]
  node_input["Dataset reader<br/>[data_loader.py]"]
  node_processor["Transformer<br/>[data_processor.py]"]
  node_protocols["Stream contracts<br/>[protocols.py]"]
  node_aggregator["Sliding aggregator<br/>[aggregator.py]"]
end

subgraph group_outputs["Output and visualization"]
  node_output["Output manager<br/>[protocols.py]"]
  node_console["Console writer<br/>[console_writer.py]"]
  node_chartwriter["Chart writer<br/>[chart_writer.py]"]
  node_chartimpl["Chart renderer"]
  node_server["Web dashboard server<br/>[server.py]"]
  node_dashboard["Dashboard template<br/>[index.html]"]
end

subgraph group_observability["Pipeline monitoring"]
  node_monitor["Queue monitor"]
  node_telemetry["Queue telemetry"]
end

node_user(("Operator"))
node_browser(("Browser user"))

node_user -->|"starts"| node_main
node_main -->|"loads"| node_config
node_main -->|"starts reader"| node_input
node_main -->|"starts workers"| node_processor
node_main -->|"starts"| node_aggregator
node_main -->|"starts"| node_output
node_main -->|"configures"| node_monitor
node_input -->|"writes rows"| node_rawq
node_processor -->|"reads rows"| node_rawq
node_processor -->|"writes transformed"| node_interq
node_aggregator -->|"reads packets"| node_interq
node_aggregator -->|"writes averages"| node_processedq
node_output -->|"reads packets"| node_processedq
node_output -->|"writes output"| node_console
node_output -->|"renders charts"| node_chartwriter
node_chartwriter -->|"updates plots"| node_chartimpl
node_chartimpl -->|"starts server"| node_server
node_server -->|"renders page"| node_dashboard
node_browser -.->|"views dashboard"| node_server
node_main -->|"creates queue"| node_rawq
node_main -->|"creates queue"| node_interq
node_main -->|"creates queue"| node_processedq
node_monitor -->|"starts polling"| node_telemetry
node_telemetry -->|"notifies snapshots"| node_monitor
node_telemetry -->|"polls size"| node_rawq
node_telemetry -->|"polls size"| node_interq
node_telemetry -->|"polls size"| node_processedq

click node_main "https://github.com/orionshinesbright/global-gdp-analysis/blob/main/main.py"
click node_config "https://github.com/orionshinesbright/global-gdp-analysis/blob/main/config.json"
click node_input "https://github.com/orionshinesbright/global-gdp-analysis/blob/main/plugins/input/data_loader.py"
click node_rawq "https://github.com/orionshinesbright/global-gdp-analysis/blob/main/stream/Stream.py"
click node_processor "https://github.com/orionshinesbright/global-gdp-analysis/blob/main/core/data_processor.py"
click node_protocols "https://github.com/orionshinesbright/global-gdp-analysis/blob/main/core/protocols.py"
click node_interq "https://github.com/orionshinesbright/global-gdp-analysis/blob/main/stream/Stream.py"
click node_aggregator "https://github.com/orionshinesbright/global-gdp-analysis/blob/main/core/aggregator.py"
click node_processedq "https://github.com/orionshinesbright/global-gdp-analysis/blob/main/stream/Stream.py"
click node_output "https://github.com/orionshinesbright/global-gdp-analysis/blob/main/plugins/output/protocols.py"
click node_console "https://github.com/orionshinesbright/global-gdp-analysis/blob/main/plugins/output/console_writer.py"
click node_chartwriter "https://github.com/orionshinesbright/global-gdp-analysis/blob/main/plugins/output/chart_writer.py"
click node_chartimpl "https://github.com/orionshinesbright/global-gdp-analysis/blob/main/plugins/output/chart_implementations.py"
click node_server "https://github.com/orionshinesbright/global-gdp-analysis/blob/main/plugins/output/web/server.py"
click node_dashboard "https://github.com/orionshinesbright/global-gdp-analysis/blob/main/plugins/output/web/templates/index.html"
click node_monitor "https://github.com/orionshinesbright/global-gdp-analysis/blob/main/plugins/output/pipeline_monitor.py"
click node_telemetry "https://github.com/orionshinesbright/global-gdp-analysis/blob/main/plugins/output/pipeline_telemetry.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_main,node_config,node_rawq,node_interq,node_processedq,node_browser toneBlue
class node_input,node_processor,node_protocols,node_aggregator toneAmber
class node_output,node_console,node_chartwriter,node_chartimpl,node_server,node_dashboard toneMint
class node_monitor,node_telemetry toneRose
class node_user toneIndigo
```

## Components

### Pipeline orchestration

- [Bootstrap](https://github.com/orionshinesbright/global-gdp-analysis/blob/main/main.py): `main.py`
- [Pipeline config](https://github.com/orionshinesbright/global-gdp-analysis/blob/main/config.json): `config.json`
- [Raw queue](https://github.com/orionshinesbright/global-gdp-analysis/blob/main/stream/Stream.py): `stream/Stream.py`
- [Intermediate queue](https://github.com/orionshinesbright/global-gdp-analysis/blob/main/stream/Stream.py): `stream/Stream.py`
- [Processed queue](https://github.com/orionshinesbright/global-gdp-analysis/blob/main/stream/Stream.py): `stream/Stream.py`

### Data processing

- [Dataset reader](https://github.com/orionshinesbright/global-gdp-analysis/blob/main/plugins/input/data_loader.py): `plugins/input/data_loader.py`
- [Transformer](https://github.com/orionshinesbright/global-gdp-analysis/blob/main/core/data_processor.py): `core/data_processor.py`
- [Stream contracts](https://github.com/orionshinesbright/global-gdp-analysis/blob/main/core/protocols.py): `core/protocols.py`
- [Sliding aggregator](https://github.com/orionshinesbright/global-gdp-analysis/blob/main/core/aggregator.py): `core/aggregator.py`

### Output and visualization

- [Output manager](https://github.com/orionshinesbright/global-gdp-analysis/blob/main/plugins/output/protocols.py): `plugins/output/protocols.py`
- [Console writer](https://github.com/orionshinesbright/global-gdp-analysis/blob/main/plugins/output/console_writer.py): `plugins/output/console_writer.py`
- [Chart writer](https://github.com/orionshinesbright/global-gdp-analysis/blob/main/plugins/output/chart_writer.py): `plugins/output/chart_writer.py`
- [Chart renderer](https://github.com/orionshinesbright/global-gdp-analysis/blob/main/plugins/output/chart_implementations.py): `plugins/output/chart_implementations.py`
- [Web dashboard server](https://github.com/orionshinesbright/global-gdp-analysis/blob/main/plugins/output/web/server.py): `plugins/output/web/server.py`
- [Dashboard template](https://github.com/orionshinesbright/global-gdp-analysis/blob/main/plugins/output/web/templates/index.html): `plugins/output/web/templates/index.html`

### Pipeline monitoring

- [Queue monitor](https://github.com/orionshinesbright/global-gdp-analysis/blob/main/plugins/output/pipeline_monitor.py): `plugins/output/pipeline_monitor.py`
- [Queue telemetry](https://github.com/orionshinesbright/global-gdp-analysis/blob/main/plugins/output/pipeline_telemetry.py): `plugins/output/pipeline_telemetry.py`

### Other components

- **Operator**
- **Browser user**

## Connections

- Operator → Bootstrap: starts (evidence: `README.md`)
- Bootstrap → Pipeline config: loads (evidence: `main.py`)
- Bootstrap → Dataset reader: starts reader (evidence: `main.py`)
- Bootstrap → Transformer: starts workers (evidence: `main.py`)
- Bootstrap → Sliding aggregator: starts (evidence: `main.py`)
- Bootstrap → Output manager: starts (evidence: `main.py`)
- Bootstrap → Queue monitor: configures (evidence: `main.py`)
- Dataset reader → Raw queue: writes rows (evidence: `plugins/input/data_loader.py`)
- Transformer → Raw queue: reads rows (evidence: `core/data_processor.py`)
- Transformer → Intermediate queue: writes transformed (evidence: `core/data_processor.py`)
- Sliding aggregator → Intermediate queue: reads packets (evidence: `core/aggregator.py`)
- Sliding aggregator → Processed queue: writes averages (evidence: `core/aggregator.py`)
- Output manager → Processed queue: reads packets (evidence: `plugins/output/protocols.py`)
- Output manager → Console writer: writes output (evidence: `plugins/output/protocols.py`)
- Output manager → Chart writer: renders charts (evidence: `plugins/output/protocols.py`)
- Chart writer → Chart renderer: updates plots (evidence: `plugins/output/chart_writer.py`)
- Chart renderer → Web dashboard server: starts server (evidence: `plugins/output/chart_implementations.py`)
- Web dashboard server → Dashboard template: renders page (evidence: `plugins/output/web/server.py`)
- Browser user → Web dashboard server: views dashboard
- Bootstrap → Raw queue: creates queue (evidence: `main.py`)
- Bootstrap → Intermediate queue: creates queue (evidence: `main.py`)
- Bootstrap → Processed queue: creates queue (evidence: `main.py`)
- Queue monitor → Queue telemetry: starts polling (evidence: `plugins/output/pipeline_monitor.py`)
- Queue telemetry → Queue monitor: notifies snapshots (evidence: `plugins/output/pipeline_telemetry.py`)
- Queue telemetry → Raw queue: polls size (evidence: `plugins/output/pipeline_telemetry.py`)
- Queue telemetry → Intermediate queue: polls size (evidence: `plugins/output/pipeline_telemetry.py`)
- Queue telemetry → Processed queue: polls size (evidence: `plugins/output/pipeline_telemetry.py`)