How wilson-zhelin/streamline-analyst works
Main Components
• User Interface – Represented by the Streamlit app (app/app.py). This is where the user uploads a file and selects analysis modes.
• Data Ingestion & Preprocessing – Modules that handle data upload and cleaning (e.g., app/src/preprocess.py, handle_null_value.py). Include tasks like handling null values and encoding.
• LLM Service & Decision Engine – The integration with OpenAI’s API via the LLM service (e.g., app/src/llm_service.py) which drives target variable identification, model suggestions, and strategic choices.
• Model Service Layer – This includes the different modeling components:
▪ Regression Models (app/regression_model.py, app/src/regression_model.py)
▪ Clustering Models (app/cluster_model.py, app/src/cluster_model.py)
▪ Predictive Modeling (app/src/predictive_model.py)
• Visualization Engine – Responsible for generating plots and visualizations (app/visualization.py and app/src/plot.py).
• Data & Configuration Management – The configuration files (app/config/config.yaml and app/src/config/config.yaml) and utility modules (app/util.py, app/src/util.py) that facilitate modular configuration and reusability.
• Sample Data Repository – (sample_data folder) to support demo and testing scenarios.
Interactions & Relationships
• From Ingestion, lead into a branching process:
▪ One branch for Preprocessing and Data Cleaning (modules such as preprocess and handle_null_value).
▪ One branch signaling an LLM request (llm_service) to determine target variable, modeling strategies, and model selection.
• After decision making, indicate separate paths:
▪ To the Model Service Layer – invoking regression, clustering, or classification based on the LLM’s recommendation.
▪ To the Visualization Engine – for post-modeling visuals and user feedback.
Architectural Patterns and Design Principles
• Modular design: Note that similar functionalities (e.g., clustering vs. regression) are implemented in separate modules, enabling easy replacement or extension.
• Configuration-driven design: Represent the configuration modules feeding settings into each processing component.
• Pipeline architecture: Indicate a flow that starts from data ingestion, goes through preprocessing, decision-making with LLM support, model training, and concludes with visualization.
Technology & Framework References
• “Python 3.11.5” along with libraries like scikit-learn (implied by various model implementations) and potentially plotting libraries used in visualization.
• Denote external integration with OpenAI’s API in the LLM Service component.
• If applicable, mention any specific packages as noted in requirements.txt.
Final Assembly
• Every module from the code base (or at least the root ones like app/app.py, app/src directory modules, and configuration files) is represented.
• Do not hesitate to “over-segment” components if it improves clarity—each module (preprocessing, LLM service, modeling, visualization) should be a separate node.