# kedarg63/hedgefund architecture

> Architecture diagram of the GitHub repository kedarg63/hedgefund, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kedarg63/hedgefund
- Repository: https://github.com/kedarg63/hedgefund

## Overview

quantdata is a primary-source research-data pipeline and point-in-time research terminal for Indian and US equities, commodities, and macro data. Its central workflow collects exchange, regulator, ministry, and broker data; archives original bytes; parses them into a knowledge-date-stamped parquet warehouse; and exposes research views through a Streamlit dashboard and authenticated API/React terminal. Analytics and quality monitoring support derived signals and trust checks. The README documents additional connector and analytics capabilities not represented by sampled implementation excerpts; those are shown at subsystem level, with uncertain internal wiring omitted.

Main entry points: [`core/storage.py`](https://github.com/kedarg63/hedgefund/blob/terminal-foundations/core/storage.py) (Parquet warehouse), [`run_daily.py`](https://github.com/kedarg63/hedgefund/blob/terminal-foundations/run_daily.py) (Daily jobs), [`connectors`](https://github.com/kedarg63/hedgefund/tree/terminal-foundations/connectors) (India connectors), [`connectors/stream.py`](https://github.com/kedarg63/hedgefund/blob/terminal-foundations/connectors/stream.py) (Live stream)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_collection["Data collection"]
  node_india_sources(("India sources"))
  node_us_sources(("US sources"))
  node_global_sources(("Global and macro sources"))
  node_orchestrator["Daily jobs<br/>[run_daily.py]"]
  node_india_connectors["India connectors"]
  node_us_connectors["US filing connectors"]
  node_macro_connectors["Macro and commodity connectors"]
end

subgraph group_warehouse["Archive and warehouse"]
  node_raw_archive[("Immutable raw archive<br/>[storage.py]")]
  node_parquet[("Parquet warehouse<br/>[storage.py]")]
  node_duckdb[("DuckDB views<br/>[storage.py]")]
end

subgraph group_research["Research and monitoring"]
  node_analytics["Signal analytics"]
  node_quality["Data quality<br/>[quality.py]"]
  node_alerts["Failure alerts<br/>[alerting.py]"]
end

subgraph group_interfaces["Research interfaces"]
  node_dashboard["Streamlit dashboard<br/>[app.py]"]
  node_api["Terminal API<br/>[main.py]"]
  node_api_routers["Data routes"]
  node_web["React terminal<br/>[App.tsx]"]
  node_panels["Research panels"]
end

subgraph group_realtime["Live market data"]
  node_stream["Live stream<br/>[stream.py]"]
  node_redis[("Tick cache<br/>[redis_client.py]")]
end

node_researcher(("Researcher"))
node_broker(("Broker feed"))

node_orchestrator -->|"runs jobs"| node_india_connectors
node_orchestrator -->|"runs jobs"| node_us_connectors
node_orchestrator -->|"runs jobs"| node_macro_connectors
node_india_sources -->|"supplies data"| node_india_connectors
node_us_sources -->|"supplies filings"| node_us_connectors
node_global_sources -->|"supplies data"| node_macro_connectors
node_india_connectors -->|"archives bytes"| node_raw_archive
node_us_connectors -->|"archives bytes"| node_raw_archive
node_macro_connectors -->|"archives bytes"| node_raw_archive
node_india_connectors -->|"writes parsed data"| node_parquet
node_us_connectors -->|"writes parsed data"| node_parquet
node_macro_connectors -->|"writes parsed data"| node_parquet
node_parquet -->|"backs views"| node_duckdb
node_analytics -->|"writes derived data"| node_parquet
node_dashboard -->|"queries views"| node_duckdb
node_dashboard -->|"checks freshness"| node_quality
node_orchestrator -->|"reports failures"| node_alerts
node_api -->|"registers routes"| node_api_routers
node_api_routers -->|"queries as-of data"| node_duckdb
node_web -->|"requests data"| node_api
node_web -->|"hosts panels"| node_panels
node_researcher -->|"explores research"| node_dashboard
node_researcher -->|"uses terminal"| node_web
node_broker -->|"streams ticks"| node_stream
node_stream -->|"publishes ticks"| node_redis
node_stream -->|"archives stream"| node_raw_archive
node_redis -.->|"serves live data"| node_api

click node_orchestrator "https://github.com/kedarg63/hedgefund/blob/terminal-foundations/run_daily.py"
click node_india_connectors "https://github.com/kedarg63/hedgefund/tree/terminal-foundations/connectors"
click node_us_connectors "https://github.com/kedarg63/hedgefund/tree/terminal-foundations/connectors"
click node_macro_connectors "https://github.com/kedarg63/hedgefund/tree/terminal-foundations/connectors"
click node_stream "https://github.com/kedarg63/hedgefund/blob/terminal-foundations/connectors/stream.py"
click node_raw_archive "https://github.com/kedarg63/hedgefund/blob/terminal-foundations/core/storage.py"
click node_parquet "https://github.com/kedarg63/hedgefund/blob/terminal-foundations/core/storage.py"
click node_duckdb "https://github.com/kedarg63/hedgefund/blob/terminal-foundations/core/storage.py"
click node_redis "https://github.com/kedarg63/hedgefund/blob/terminal-foundations/core/redis_client.py"
click node_analytics "https://github.com/kedarg63/hedgefund/tree/terminal-foundations/analytics"
click node_quality "https://github.com/kedarg63/hedgefund/blob/terminal-foundations/core/quality.py"
click node_alerts "https://github.com/kedarg63/hedgefund/blob/terminal-foundations/core/alerting.py"
click node_dashboard "https://github.com/kedarg63/hedgefund/blob/terminal-foundations/dashboard/app.py"
click node_api "https://github.com/kedarg63/hedgefund/blob/terminal-foundations/api/main.py"
click node_api_routers "https://github.com/kedarg63/hedgefund/tree/terminal-foundations/api/routers"
click node_web "https://github.com/kedarg63/hedgefund/blob/terminal-foundations/web/src/App.tsx"
click node_panels "https://github.com/kedarg63/hedgefund/tree/terminal-foundations/web/src/panels"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_india_sources,node_us_sources,node_global_sources,node_orchestrator,node_india_connectors,node_us_connectors,node_macro_connectors toneBlue
class node_raw_archive,node_parquet,node_duckdb toneAmber
class node_analytics,node_quality,node_alerts toneMint
class node_dashboard,node_api,node_api_routers,node_web,node_panels toneRose
class node_stream,node_redis,node_researcher,node_broker toneIndigo
```

## Components

### Data collection

- **India sources**
- **US sources**
- **Global and macro sources**
- [Daily jobs](https://github.com/kedarg63/hedgefund/blob/terminal-foundations/run_daily.py): `run_daily.py`
- [India connectors](https://github.com/kedarg63/hedgefund/tree/terminal-foundations/connectors): `connectors`
- [US filing connectors](https://github.com/kedarg63/hedgefund/tree/terminal-foundations/connectors): `connectors`
- [Macro and commodity connectors](https://github.com/kedarg63/hedgefund/tree/terminal-foundations/connectors): `connectors`

### Archive and warehouse

- [Immutable raw archive](https://github.com/kedarg63/hedgefund/blob/terminal-foundations/core/storage.py): `core/storage.py`
- [Parquet warehouse](https://github.com/kedarg63/hedgefund/blob/terminal-foundations/core/storage.py): `core/storage.py`
- [DuckDB views](https://github.com/kedarg63/hedgefund/blob/terminal-foundations/core/storage.py): `core/storage.py`

### Research and monitoring

- [Signal analytics](https://github.com/kedarg63/hedgefund/tree/terminal-foundations/analytics): `analytics`
- [Data quality](https://github.com/kedarg63/hedgefund/blob/terminal-foundations/core/quality.py): `core/quality.py`
- [Failure alerts](https://github.com/kedarg63/hedgefund/blob/terminal-foundations/core/alerting.py): `core/alerting.py`

### Research interfaces

- [Streamlit dashboard](https://github.com/kedarg63/hedgefund/blob/terminal-foundations/dashboard/app.py): `dashboard/app.py`
- [Terminal API](https://github.com/kedarg63/hedgefund/blob/terminal-foundations/api/main.py): `api/main.py`
- [Data routes](https://github.com/kedarg63/hedgefund/tree/terminal-foundations/api/routers): `api/routers`
- [React terminal](https://github.com/kedarg63/hedgefund/blob/terminal-foundations/web/src/App.tsx): `web/src/App.tsx`
- [Research panels](https://github.com/kedarg63/hedgefund/tree/terminal-foundations/web/src/panels): `web/src/panels`

### Live market data

- [Live stream](https://github.com/kedarg63/hedgefund/blob/terminal-foundations/connectors/stream.py): `connectors/stream.py`
- [Tick cache](https://github.com/kedarg63/hedgefund/blob/terminal-foundations/core/redis_client.py): `core/redis_client.py`

### Other components

- **Researcher**
- **Broker feed**

## Connections

- Daily jobs → India connectors: runs jobs
- Daily jobs → US filing connectors: runs jobs
- Daily jobs → Macro and commodity connectors: runs jobs
- India sources → India connectors: supplies data
- US sources → US filing connectors: supplies filings
- Global and macro sources → Macro and commodity connectors: supplies data
- India connectors → Immutable raw archive: archives bytes
- US filing connectors → Immutable raw archive: archives bytes
- Macro and commodity connectors → Immutable raw archive: archives bytes
- India connectors → Parquet warehouse: writes parsed data
- US filing connectors → Parquet warehouse: writes parsed data
- Macro and commodity connectors → Parquet warehouse: writes parsed data
- Parquet warehouse → DuckDB views: backs views
- Signal analytics → Parquet warehouse: writes derived data
- Streamlit dashboard → DuckDB views: queries views
- Streamlit dashboard → Data quality: checks freshness
- Daily jobs → Failure alerts: reports failures
- Terminal API → Data routes: registers routes
- Data routes → DuckDB views: queries as-of data
- React terminal → Terminal API: requests data
- React terminal → Research panels: hosts panels
- Researcher → Streamlit dashboard: explores research
- Researcher → React terminal: uses terminal
- Broker feed → Live stream: streams ticks
- Live stream → Tick cache: publishes ticks
- Live stream → Immutable raw archive: archives stream
- Tick cache → Terminal API: serves live data