# yogeshvarm/data_analysis_assistant architecture

> Architecture diagram of the GitHub repository yogeshvarm/data_analysis_assistant, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/yogeshvarm/data_analysis_assistant
- Repository: https://github.com/yogeshvarm/data_analysis_assistant

## Overview

##### Project Overview
- This project is a data analysis tool that uses a Streamlit dashboard to allow users to interact with their datasets naturally. It combines data visualization with automatic SQL query generation and leverages a Groq-backed LLM for natural language processing.
- The system follows a layered architecture that divides the project into three main layers:
a. Configuration Layer – For initial setup, file uploads, API key validation, and creating the SQLite database.
b. Streamlit Interface – The interactive dashboard that shows data metrics, charts, and hosts the chat interface.
c. Core Processing – The SQL Agent Workflow that processes user queries through a series of steps (parse_question, get_unique_nouns, generate_sql, validate_and_fix_sql, execute_sql, and response generation).

##### Key Components (Based on the File Tree and README)
- Main Application (app.py): Acts as the entry point that initializes the Streamlit interface.
- langgraph Directory: Contains core modules:
- WorkflowManager: Orchestrates the end-to-end workflow.
- SQLAgent: Generates and executes SQL queries from natural language inputs.
- DataFormatter: Formats data for varied types of visualizations.
- LLMManager: Handles interactions with the Groq-based large language model.
- DatabaseManager: Manages operations with the SQLite database.
- State: Maintains application state.
- graph_instructions: Contains instructions for creating or formatting visual graphs.
- Configuration files like requirements.txt and licensing/documentation files.

a. Create a clear visual separation for the three layers:
b. In the Configuration Layer, include:
- User Data Upload and API Key Validation module.
- File processing for CSV (and future support for Excel, JSON, Parquet).
- SQLite database creation.
c. In the Streamlit Interface layer, illustrate:
- The Dashboard with data metrics, dynamic visualizations, and a chat interface.
- Pathways showing how user actions (upload, query) initiate flows.
d. In the Core Processing layer, outline the SQL Agent Workflow:
- Module steps: parse_question, get_unique_nouns, generate_sql, validate_and_fix_sql, execute_sql.
- Branch the execute_sql step into two paths showing text output (formatted via DataFormatter for results) and visualization selection (choose_visualization & graph_instructions).
- Response generation that passes the formatted result back to the chat interface and dashboard update.
e. Inter-module communication:
- Similarly, illustrate the flow back to the interface after processing.

##### Architectural Patterns & Design Principles
- Separation of Concerns: Each layer handles a distinct responsibility – configuration/setup, user interaction via UI, and backend query processing.
- Sequential Workflow: Within Core Processing, emphasize the step-by-step processing in the SQL Agent workflow.
- Modular Design: Highlight how each module (WorkflowManager, SQLAgent, LLMManager, etc.) plays a specialized role.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Configuration Layer
    subgraph "Configuration Layer"
        C1["User Data Upload & API Key Validation"]:::config
        C2["File Processing (CSV,Excel,JSON,Parquet)"]:::config
        C3["SQLite Database Creation"]:::config
        C4["Database Manager"]:::component
    end

    %% Streamlit Interface
    subgraph "Streamlit Interface"
        SI1["Main Application"]:::component
        SI2["Dashboard (Metrics,Chat,Visualizations)"]:::ui
    end

    %% Core Processing Layer
    subgraph "Core Processing"
        CP1["Workflow Manager"]:::component
        CP2["SQL Agent"]:::component
        CP3["LLM Manager"]:::component
        CP4["Application State"]:::component

        subgraph "SQL Agent Workflow"
            W1["parse_question"]:::step
            W2["get_unique_nouns"]:::step
            W3["generate_sql"]:::step
            W4["validate_and_fix_sql"]:::step
            W5["execute_sql"]:::step
            W6["Data Formatter"]:::processing
            W7["Graph Instructions"]:::processing
            W8["Response Generation"]:::step
        end
    end

    %% Connections between Configuration Layer nodes
    C1 -->|"processes"| C2
    C2 -->|"prepares"| C3
    C3 -->|"initializes DB"| CP1

    %% Connections in Streamlit Interface
    SI1 -->|"launches"| SI2
    SI2 -->|"submits query"| CP1

    %% Connections within Core Processing (between components)
    CP1 -->|"orchestrates"| CP2
    CP1 -->|"queries LLM"| CP3
    CP1 -->|"updates state"| CP4

    %% Connections for SQL Agent Workflow steps
    W1 -->|"next"| W2
    W2 -->|"next"| W3
    W3 -->|"next"| W4
    W4 -->|"next"| W5
    W5 -- "text_output" --> W6
    W5 -- "visualization" --> W7
    W6 -->|"merges"| W8
    W7 -->|"merges"| W8
    W8 -->|"returns result"| CP1
    CP1 -->|"updates dashboard"| SI2

    %% Click Events for component mapping
    click SI1 "https://github.com/yogeshvarm/data_analysis_assistant/blob/main/app.py"
    click CP1 "https://github.com/yogeshvarm/data_analysis_assistant/blob/main/langgraph/WorkflowManager.py"
    click CP2 "https://github.com/yogeshvarm/data_analysis_assistant/blob/main/langgraph/SQLAgent.py"
    click W6 "https://github.com/yogeshvarm/data_analysis_assistant/blob/main/langgraph/DataFormatter.py"
    click CP3 "https://github.com/yogeshvarm/data_analysis_assistant/blob/main/langgraph/LLMManager.py"
    click C4 "https://github.com/yogeshvarm/data_analysis_assistant/blob/main/langgraph/DatabaseManager.py"
    click CP4 "https://github.com/yogeshvarm/data_analysis_assistant/blob/main/langgraph/State.py"
    click W7 "https://github.com/yogeshvarm/data_analysis_assistant/blob/main/langgraph/graph_instructions.py"

    %% Styles
    classDef config fill:#FFECB3,stroke:#F57C00,stroke-width:2px;
    classDef ui fill:#C8E6C9,stroke:#388E3C,stroke-width:2px;
    classDef component fill:#BBDEFB,stroke:#1976D2,stroke-width:2px;
    classDef step fill:#D1C4E9,stroke:#7B1FA2,stroke-width:2px;
    classDef processing fill:#FFE0B2,stroke:#F57C00,stroke-width:2px;
```
