# zaibys/us-census-chatbot architecture

> Architecture diagram of the GitHub repository zaibys/us-census-chatbot, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/zaibys/us-census-chatbot
- Repository: https://github.com/zaibys/us-census-chatbot

## Overview

##### Project Type & Purpose
- This is a full-stack, AI-powered web application built in Python with a Streamlit front end.

##### File Structure
Top-level folders/files:
- `streamlit_app.py` – Streamlit UI entry point
- `src/` – core application logic
– `main.py` – orchestration of the request pipeline
– `system_prompt.py` & `embeddings.py` – LLM prompt and embedding logic
– `big_query_manager.py` – BigQuery client wrapper
– `data_handler.py` & `response_handler.py` – result formatting & fallback handling
– `logger.py` – centralized logging
– `components.py` – UI components for Streamlit
- `chroma_langchain_db/chroma.sqlite3` – local vector store for schema embeddings
- `.env-template` – environment variables (GCP project ID, BigQuery credentials)

##### README Insights
- Confirms tech stack: Streamlit, LangChain, Gemini AI, Google BigQuery, Docker/GCP infra.
- High-level flow described (user → NL to SQL → BigQuery → visualization).

##### Main Components & Their Relationships
a. Frontend (Streamlit)
- `streamlit_app.py` + `components.py`
- Collects natural-language input, renders charts/tables.
b. Orchestration Layer
- `main.py` – central controller that sequences sub-components.
c. NLP & SQL Generation
- `system_prompt.py` – builds prompts for Gemini AI
- `embeddings.py` – creates/queries embeddings in local Chroma DB (`chroma.sqlite3`) to fetch relevant schema context
- LangChain + Gemini API (external service)
d. Query Execution
- `big_query_manager.py` – sends SQL to Google BigQuery, retrieves raw results
e. Data Processing & Response Handling
- `data_handler.py` – transforms BigQuery rows into pandas/dataframe
- `response_handler.py` – formats fallback or success responses
f. Persistence & Logging
- Local Chroma vector store (`chroma.sqlite3`) for cached schema embeddings
- `logger.py` – logs each step (prompt, SQL, errors)
g. External Services
- Google BigQuery (data warehouse)
- LangChain + Google’s Gemini (LLM + embeddings)
h. Infrastructure
- Docker containerization (Dockerfile not shown but implied by README)
- GCP environment variables (`.env-template`)

##### Architectural Patterns & Design Principles
- Pipeline / Chain-of-Responsibility: each step handles a specific concern (prompt → embed → LLM → SQL → execute → display).
- Separation of concerns: UI vs orchestration vs data access vs processing.
- Caching via vector store to reduce repeated schema lookups.
- Fallback mechanism if LLM fails to generate SQL (handled in `response_handler.py`).

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Client"
        User("User"):::user
    end

    subgraph "Frontend Layer"
        StreamlitApp["Streamlit UI<br>streamlit_app.py"]:::frontend
        Components["UI Components<br>components.py"]:::frontend
    end

    subgraph "Business Logic"
        Main["Orchestrator<br>main.py"]:::logic
        SystemPrompt["Prompt Builder<br>system_prompt.py"]:::logic
        Embeddings["Embeddings Module<br>embeddings.py"]:::logic
        BigQueryManager["BigQuery Manager<br>big_query_manager.py"]:::logic
        DataHandler["Data Handler<br>data_handler.py"]:::logic
        ResponseHandler["Response Handler<br>response_handler.py"]:::logic
        Logger["Logger<br>logger.py"]:::logic
    end

    subgraph "Data Stores"
        ChromaDB[(Chroma DB<br>chroma.sqlite3)]:::datastore
    end

    subgraph "Static Assets"
        SchemaDef["Schema Definitions<br>demographics_Schema.txt"]:::datastore
        FlowDiagram["Flow Diagram<br>flow_diagram.png"]:::datastore
    end

    subgraph "Infrastructure"
        Env["Environment Config<br>.env-template"]:::infra
    end

    subgraph "External Services"
        LLMService["Gemini AI (LLM)"]:::external
        BigQuery["Google BigQuery"]:::external
    end

    User --> StreamlitApp
    StreamlitApp --> Components
    StreamlitApp -->|"user query"| Main
    Main -->|"build prompt"| SystemPrompt
    Main -->|"fetch embeddings"| Embeddings
    SystemPrompt -->|"prompt"| LLMService
    Embeddings -->|"query schema"| ChromaDB
    LLMService -->|"SQL"| Main
    Main -->|"execute SQL"| BigQueryManager
    BigQueryManager -->|"run query"| BigQuery
    BigQuery -->|"raw data"| DataHandler
    DataHandler -->|"transform data"| ResponseHandler
    ResponseHandler -->|"render results"| StreamlitApp

    Main -.-> Logger
    SystemPrompt -.-> Logger
    Embeddings -.-> Logger
    BigQueryManager -.-> Logger
    DataHandler -.-> Logger
    ResponseHandler -.-> Logger

    click StreamlitApp "https://github.com/zaibys/us-census-chatbot/blob/main/streamlit_app.py"
    click Components "https://github.com/zaibys/us-census-chatbot/blob/main/src/components.py"
    click Main "https://github.com/zaibys/us-census-chatbot/blob/main/src/main.py"
    click SystemPrompt "https://github.com/zaibys/us-census-chatbot/blob/main/src/system_prompt.py"
    click Embeddings "https://github.com/zaibys/us-census-chatbot/blob/main/src/embeddings.py"
    click ChromaDB "https://github.com/zaibys/us-census-chatbot/blob/main/chroma_langchain_db/chroma.sqlite3"
    click BigQueryManager "https://github.com/zaibys/us-census-chatbot/blob/main/src/big_query_manager.py"
    click DataHandler "https://github.com/zaibys/us-census-chatbot/blob/main/src/data_handler.py"
    click ResponseHandler "https://github.com/zaibys/us-census-chatbot/blob/main/src/response_handler.py"
    click Logger "https://github.com/zaibys/us-census-chatbot/blob/main/src/logger.py"
    click SchemaDef "https://github.com/zaibys/us-census-chatbot/blob/main/data/demographics_Schema.txt"
    click FlowDiagram "https://github.com/zaibys/us-census-chatbot/blob/main/data/flow_diagram.png"
    click Env "https://github.com/zaibys/us-census-chatbot/blob/main/.env-template"

    classDef frontend fill:#cce5ff,stroke:#333,stroke-width:1px
    classDef logic fill:#d4edda,stroke:#333,stroke-width:1px
    classDef datastore fill:#fff3cd,stroke:#333,stroke-width:1px
    classDef external fill:#e2e3e5,stroke:#333,stroke-width:1px
    classDef infra fill:#f8d7da,stroke:#333,stroke-width:1px
    classDef user fill:#f5c6cb,stroke:#333,stroke-width:1px
```
