# wilson-zhelin/streamline-analyst architecture

> Architecture diagram of the GitHub repository wilson-zhelin/streamline-analyst, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/wilson-zhelin/streamline-analyst
- Repository: https://github.com/wilson-zhelin/streamline-analyst

## Overview

##### Main Components
- User Interface – Represented by the Streamlit app (app/app.py). This is where the user uploads a file and selects analysis modes.
- Data Ingestion & Preprocessing – Modules that handle data upload and cleaning (e.g., app/src/preprocess.py, handle_null_value.py). Include tasks like handling null values and encoding.
- LLM Service & Decision Engine – The integration with OpenAI’s API via the LLM service (e.g., app/src/llm_service.py) which drives target variable identification, model suggestions, and strategic choices.
- Model Service Layer – This includes the different modeling components:
▪ Regression Models (app/regression_model.py, app/src/regression_model.py)
▪ Clustering Models (app/cluster_model.py, app/src/cluster_model.py)
▪ Predictive Modeling (app/src/predictive_model.py)
- Visualization Engine – Responsible for generating plots and visualizations (app/visualization.py and app/src/plot.py).
- Data & Configuration Management – The configuration files (app/config/config.yaml and app/src/config/config.yaml) and utility modules (app/util.py, app/src/util.py) that facilitate modular configuration and reusability.
- Sample Data Repository – (sample_data folder) to support demo and testing scenarios.

##### Interactions & Relationships
- From Ingestion, lead into a branching process:
▪ One branch for Preprocessing and Data Cleaning (modules such as preprocess and handle_null_value).
▪ One branch signaling an LLM request (llm_service) to determine target variable, modeling strategies, and model selection.
- After decision making, indicate separate paths:
▪ To the Model Service Layer – invoking regression, clustering, or classification based on the LLM’s recommendation.
▪ To the Visualization Engine – for post-modeling visuals and user feedback.

##### Architectural Patterns and Design Principles
- Modular design: Note that similar functionalities (e.g., clustering vs. regression) are implemented in separate modules, enabling easy replacement or extension.
- Configuration-driven design: Represent the configuration modules feeding settings into each processing component.
- Pipeline architecture: Indicate a flow that starts from data ingestion, goes through preprocessing, decision-making with LLM support, model training, and concludes with visualization.

##### Technology & Framework References
- “Python 3.11.5” along with libraries like scikit-learn (implied by various model implementations) and potentially plotting libraries used in visualization.
- Denote external integration with OpenAI’s API in the LLM Service component.
- If applicable, mention any specific packages as noted in requirements.txt.

##### Final Assembly
- Every module from the code base (or at least the root ones like app/app.py, app/src directory modules, and configuration files) is represented.
- Do not hesitate to “over-segment” components if it improves clarity—each module (preprocessing, LLM service, modeling, visualization) should be a separate node.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Nodes
    UI["Streamlit Front-End"]:::frontend
    DI["Data Ingestion Module"]:::processing
    DP["Data Preprocessing Module"]:::processing
    LLM["LLM Service & Decision Engine"]:::processing
    CM["Data & Configuration Management"]:::config
    SD["Sample Data Repository"]:::external
    EO["OpenAI API"]:::external
    MD["Model Service Dispatcher"]:::model
    MO["Model Output Aggregator"]:::model
    VE["Visualization Engine"]:::processing

    %% Model Service Layer Subgraph
    subgraph "Model Service Layer"
        R["Regression Models"]:::model
        C["Clustering Models"]:::model
        P["Predictive Modeling"]:::model
    end

    %% Connections
    UI -->|"UploadFile"| DI
    DI -->|"RawData"| DP
    DI -->|"LLMRequest"| LLM

    %% Data & LLM feed into model dispatcher
    DP -->|"CleanData"| MD
    LLM -->|"ModelRec"| MD

    %% Dispatcher to Model Service Layer components
    MD -->|"Dispatch"| R
    MD -->|"Dispatch"| C
    MD -->|"Dispatch"| P

    %% Modeling components feed into Model Output
    R -->|"Result"| MO
    C -->|"Result"| MO
    P -->|"Result"| MO

    %% Model Output flows to Visualization Engine
    MO -->|"ModelResults"| VE

    %% Configuration management drives various modules
    CM -->|"ConfigDriven"| DP
    CM -->|"LLMSettings"| LLM
    CM -->|"ModelConfig"| MD
    CM -->|"VizSettings"| VE

    %% UI loads sample data
    UI -->|"LoadSampleData"| SD

    %% External OpenAI integration with LLM service (bidirectional dashed)
    LLM -.->|"LLMExchange"| EO
    EO -.->|"LLMExchange"| LLM

    %% Styles
    classDef frontend fill:#F9D6C1,stroke:#333,stroke-width:2px;
    classDef processing fill:#C1E1C1,stroke:#333,stroke-width:2px;
    classDef model fill:#E1C1F9,stroke:#333,stroke-width:2px;
    classDef config fill:#C1D6F9,stroke:#333,stroke-width:2px;
    classDef external fill:#F9F9C1,stroke:#333,stroke-width:2px;

    %% Click Events
    click UI "https://github.com/wilson-zhelin/streamline-analyst/blob/main/app/app.py"
    click DP "https://github.com/wilson-zhelin/streamline-analyst/blob/main/app/src/preprocess.py"
    click DP "https://github.com/wilson-zhelin/streamline-analyst/blob/main/app/src/handle_null_value.py"
    click LLM "https://github.com/wilson-zhelin/streamline-analyst/blob/main/app/src/llm_service.py"
    click R "https://github.com/wilson-zhelin/streamline-analyst/blob/main/app/regression_model.py"
    click R "https://github.com/wilson-zhelin/streamline-analyst/blob/main/app/src/regression_model.py"
    click C "https://github.com/wilson-zhelin/streamline-analyst/blob/main/app/cluster_model.py"
    click C "https://github.com/wilson-zhelin/streamline-analyst/blob/main/app/src/cluster_model.py"
    click P "https://github.com/wilson-zhelin/streamline-analyst/blob/main/app/src/predictive_model.py"
    click P "https://github.com/wilson-zhelin/streamline-analyst/blob/main/app/prediction_model.py"
    click VE "https://github.com/wilson-zhelin/streamline-analyst/blob/main/app/visualization.py"
    click VE "https://github.com/wilson-zhelin/streamline-analyst/blob/main/app/src/plot.py"
    click CM "https://github.com/wilson-zhelin/streamline-analyst/blob/main/app/config/config.yaml"
    click CM "https://github.com/wilson-zhelin/streamline-analyst/blob/main/app/src/config/config.yaml"
    click CM "https://github.com/wilson-zhelin/streamline-analyst/blob/main/app/util.py"
    click CM "https://github.com/wilson-zhelin/streamline-analyst/blob/main/app/src/util.py"
    click SD "https://github.com/wilson-zhelin/streamline-analyst/tree/main/sample_data"
```
