# mikekenneth/duckdb_streamlit_arrow_flight architecture

> Architecture diagram of the GitHub repository mikekenneth/duckdb_streamlit_arrow_flight, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mikekenneth/duckdb_streamlit_arrow_flight
- Repository: https://github.com/mikekenneth/duckdb_streamlit_arrow_flight

## Overview

##### Main Components
a. Presentation Layer (Client/Web UI)
- Streamlit app (apps/client/web.py) – renders the web interface.
b. API Layer (Flight RPC Server)
- Flight server (apps/server/server.py) – listens on gRPC port, handles incoming Flight RPC calls.
- Query Executor – inside the server module, translates Flight calls into DuckDB SQL execution.
c. Data Layer
- DuckDB file (data/duck.db) – primary relational store.
- (Future) S3/minio endpoint abstraction.
d. Configuration & Orchestration
- `.env` / example.env – runtime settings (host, port, storage type, file paths).
- docker-compose.yaml – spins up server, client (future), and optional storage services.

##### Relationships & Data Flow
1. User opens Streamlit UI in browser.
2. UI calls Flight Client library over gRPC to the Flight Server.
3. Flight Server receives RPC, forwards SQL to DuckDB engine.
4. DuckDB executes query against `data/duck.db`.
6. Server streams flight-ready result back over gRPC.

##### Architectural Patterns & Technologies
- Data Persistence: Embedded DuckDB file + pluggable object storage.
- UI Framework: Streamlit for rapid prototyping.
- Python modules organized under `apps/client` and `apps/server`.

- Presentation (Streamlit UI)
- API/Logic (Flight RPC Client ↔ Flight RPC Server + Query Executor)
- Storage (DuckDB file + Datasets directory / S3)
- “Streamlit Web UI” (port 8501)
- “Flight RPC Client”
- “gRPC Channel”
- “Flight RPC Server”
- “DuckDB Engine”
- “Result Storage (local/S3)”
- UI → Flight Client → gRPC → Flight Server
- Flight Server → DuckDB Engine
- Flight Server ↔ Storage
- Flight Server → gRPC → Flight Client → UI
- Blue for UI components
- Green for API/server components
- Orange for storage components

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph "Docker Network"
        direction TB
        A["Streamlit Web UI\n(port 8501)"]:::ui
        B["Flight RPC Client"]:::ui
        C["gRPC Channel"]:::api
        D["Flight RPC Server"]:::api
        E["Query Executor"]:::api
        F[(DuckDB File)]:::storage
        G["Result Storage\n(local/S3)"]:::storage
    end

    H[".env / example.env"]:::config
    I["docker-compose.yaml"]:::config

    A -->|"user interaction"| B
    B -->|"Arrow Flight RPC"| C
    C -->|"RPC Call"| D
    D -->|"exec SQL query"| E
    E -->|"reads/writes"| F
    E -->|"reads/writes"| G
    D -->|"streams results (Arrow RecordBatch)"| C
    C -->|"receives data"| B
    B -->|"render/download"| A

    click A "https://github.com/mikekenneth/duckdb_streamlit_arrow_flight/blob/main/apps/client/web.py"
    click B "https://github.com/mikekenneth/duckdb_streamlit_arrow_flight/blob/main/apps/client/xlib/flight.py"
    click D "https://github.com/mikekenneth/duckdb_streamlit_arrow_flight/blob/main/apps/server/server.py"
    click E "https://github.com/mikekenneth/duckdb_streamlit_arrow_flight/blob/main/apps/server/server.py"
    click F "https://github.com/mikekenneth/duckdb_streamlit_arrow_flight/blob/main/data/duck.db"
    click H "https://github.com/mikekenneth/duckdb_streamlit_arrow_flight/blob/main/example.env"
    click I "https://github.com/mikekenneth/duckdb_streamlit_arrow_flight/blob/main/docker-compose.yaml"

    classDef ui fill:#add8e6,stroke:#000,stroke-width:1px
    classDef api fill:#90ee90,stroke:#000,stroke-width:1px
    classDef storage fill:#ffcc99,stroke:#000,stroke-width:1px
    classDef config fill:#d3d3d3,stroke:#000,stroke-width:1px
```
