# shaurya212121/streamquant architecture

> Architecture diagram of the GitHub repository shaurya212121/streamquant, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/shaurya212121/streamquant
- Repository: https://github.com/shaurya212121/streamquant

## Overview

StreamQuant is a stock-market data and signal application: market history is fetched, cleaned and feature-engineered, then stored; a scheduled worker refreshes it, while the API serves history and model predictions to a Streamlit dashboard. The API can also enqueue a background task. The README describes a broader C++/Redis Streams/FinBERT architecture, but those components are not present in the supplied tree; the excerpts show Celery with a Redis broker and XGBoost inference. Some documented capabilities, including explainability, are present as standalone modules with no shown runtime caller.

Main entry points: [`backend/data_engine.py`](https://github.com/shaurya212121/streamquant/blob/main/backend/data_engine.py) (Data Engine), [`backend/main.py`](https://github.com/shaurya212121/streamquant/blob/main/backend/main.py) (FastAPI), [`db/database.py`](https://github.com/shaurya212121/streamquant/blob/main/db/database.py) (DB Connection), [`db/models.py`](https://github.com/shaurya212121/streamquant/blob/main/db/models.py) (Stock Price Schema)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ingest["Market Data"]
  node_market(("Market Data Provider"))
  node_engine["Data Engine<br/>[data_engine.py]"]
  node_cleaner["Data Cleaner<br/>[cleaner.py]"]
  node_validator["Record Validator<br/>[stock.py]"]
end

subgraph group_runtime["Async Runtime"]
  node_worker["Celery Worker<br/>[worker.py]"]
  node_redis["Redis Broker"]
end

subgraph group_signals["ML Signals"]
  node_prediction["XGBoost Prediction<br/>[main.py]"]
  node_models["Trained XGBoost Models"]
  node_training["Model Training<br/>[train_xgb.py]"]
  node_anomaly["Volume Anomaly Detection<br/>[anomaly.py]"]
  node_explain["Trade Explanation<br/>[explainability.py]"]
end

subgraph group_persistence["Persistence"]
  node_dbconfig["DB Connection<br/>[database.py]"]
  node_schema["Stock Price Schema<br/>[models.py]"]
  node_store[("Stock Price Store")]
end

subgraph group_clients["API and UI"]
  node_api["FastAPI<br/>[main.py]"]
  node_dashboard["Streamlit Dashboard<br/>[app.py]"]
end

node_engine -->|"fetches prices"| node_market
node_engine -->|"cleans data"| node_cleaner
node_engine -->|"validates rows"| node_validator
node_engine -->|"writes through"| node_dbconfig
node_engine -->|"inserts records"| node_schema
node_dbconfig -->|"connects to"| node_store
node_schema -->|"defines table"| node_store
node_worker -->|"runs daily job"| node_engine
node_worker -->|"uses broker"| node_redis
node_api -->|"enqueues task"| node_worker
node_api -.->|"dispatches via"| node_redis
node_api -->|"opens session"| node_dbconfig
node_api -->|"queries records"| node_schema
node_api -->|"serves forecast"| node_prediction
node_prediction -->|"loads model"| node_models
node_dashboard -->|"requests data"| node_api
node_training -->|"reads history"| node_dbconfig
node_training -->|"selects prices"| node_schema
node_training -->|"saves models"| node_models
node_anomaly -->|"reads and updates"| node_dbconfig
node_anomaly -->|"queries and updates"| node_schema

click node_engine "https://github.com/shaurya212121/streamquant/blob/main/backend/data_engine.py"
click node_cleaner "https://github.com/shaurya212121/streamquant/blob/main/backend/cleaner.py"
click node_validator "https://github.com/shaurya212121/streamquant/blob/main/backend/stock.py"
click node_api "https://github.com/shaurya212121/streamquant/blob/main/backend/main.py"
click node_dashboard "https://github.com/shaurya212121/streamquant/blob/main/frontend/app.py"
click node_worker "https://github.com/shaurya212121/streamquant/blob/main/backend/worker.py"
click node_dbconfig "https://github.com/shaurya212121/streamquant/blob/main/db/database.py"
click node_schema "https://github.com/shaurya212121/streamquant/blob/main/db/models.py"
click node_prediction "https://github.com/shaurya212121/streamquant/blob/main/backend/main.py"
click node_models "https://github.com/shaurya212121/streamquant/tree/main/ml_models"
click node_training "https://github.com/shaurya212121/streamquant/blob/main/ml_models/train_xgb.py"
click node_anomaly "https://github.com/shaurya212121/streamquant/blob/main/ml_models/anomaly.py"
click node_explain "https://github.com/shaurya212121/streamquant/blob/main/ml_models/explainability.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_market,node_engine,node_cleaner,node_validator toneBlue
class node_worker,node_redis toneAmber
class node_prediction,node_models,node_training,node_anomaly,node_explain toneMint
class node_dbconfig,node_schema,node_store toneRose
class node_api,node_dashboard toneIndigo
```

## Components

### Market Data

- **Market Data Provider**
- [Data Engine](https://github.com/shaurya212121/streamquant/blob/main/backend/data_engine.py): `backend/data_engine.py`
- [Data Cleaner](https://github.com/shaurya212121/streamquant/blob/main/backend/cleaner.py): `backend/cleaner.py`
- [Record Validator](https://github.com/shaurya212121/streamquant/blob/main/backend/stock.py): `backend/stock.py`

### Async Runtime

- [Celery Worker](https://github.com/shaurya212121/streamquant/blob/main/backend/worker.py): `backend/worker.py`
- **Redis Broker**

### ML Signals

- [XGBoost Prediction](https://github.com/shaurya212121/streamquant/blob/main/backend/main.py): `backend/main.py`
- [Trained XGBoost Models](https://github.com/shaurya212121/streamquant/tree/main/ml_models): `ml_models`
- [Model Training](https://github.com/shaurya212121/streamquant/blob/main/ml_models/train_xgb.py): `ml_models/train_xgb.py`
- [Volume Anomaly Detection](https://github.com/shaurya212121/streamquant/blob/main/ml_models/anomaly.py): `ml_models/anomaly.py`
- [Trade Explanation](https://github.com/shaurya212121/streamquant/blob/main/ml_models/explainability.py): `ml_models/explainability.py`

### Persistence

- [DB Connection](https://github.com/shaurya212121/streamquant/blob/main/db/database.py): `db/database.py`
- [Stock Price Schema](https://github.com/shaurya212121/streamquant/blob/main/db/models.py): `db/models.py`
- **Stock Price Store**

### API and UI

- [FastAPI](https://github.com/shaurya212121/streamquant/blob/main/backend/main.py): `backend/main.py`
- [Streamlit Dashboard](https://github.com/shaurya212121/streamquant/blob/main/frontend/app.py): `frontend/app.py`

## Connections

- Data Engine → Market Data Provider: fetches prices (evidence: `backend/data_engine.py`)
- Data Engine → Data Cleaner: cleans data (evidence: `backend/data_engine.py`)
- Data Engine → Record Validator: validates rows (evidence: `backend/data_engine.py`)
- Data Engine → DB Connection: writes through (evidence: `backend/data_engine.py`)
- Data Engine → Stock Price Schema: inserts records (evidence: `backend/data_engine.py`)
- DB Connection → Stock Price Store: connects to (evidence: `db/database.py`)
- Stock Price Schema → Stock Price Store: defines table (evidence: `db/models.py`)
- Celery Worker → Data Engine: runs daily job (evidence: `backend/worker.py`)
- Celery Worker → Redis Broker: uses broker (evidence: `backend/worker.py`)
- FastAPI → Celery Worker: enqueues task (evidence: `backend/main.py`)
- FastAPI → Redis Broker: dispatches via
- FastAPI → DB Connection: opens session (evidence: `backend/main.py`)
- FastAPI → Stock Price Schema: queries records (evidence: `backend/main.py`)
- FastAPI → XGBoost Prediction: serves forecast (evidence: `backend/main.py`)
- XGBoost Prediction → Trained XGBoost Models: loads model (evidence: `backend/main.py`)
- Streamlit Dashboard → FastAPI: requests data (evidence: `frontend/app.py`)
- Model Training → DB Connection: reads history (evidence: `ml_models/train_xgb.py`)
- Model Training → Stock Price Schema: selects prices (evidence: `ml_models/train_xgb.py`)
- Model Training → Trained XGBoost Models: saves models (evidence: `ml_models/train_xgb.py`)
- Volume Anomaly Detection → DB Connection: reads and updates (evidence: `ml_models/anomaly.py`)
- Volume Anomaly Detection → Stock Price Schema: queries and updates (evidence: `ml_models/anomaly.py`)
