# gentlew1nd/pricespeculator architecture

> Architecture diagram of the GitHub repository gentlew1nd/pricespeculator, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/gentlew1nd/pricespeculator
- Repository: https://github.com/gentlew1nd/pricespeculator

## Overview

##### Main Components
- Data Pipeline Modules:
- Data Ingestion: Look at components like src/mlProject/components/data_ingestion.py and pipeline/stage_01_data_ingestion.py along with artifacts/data_ingestion.
- Data Validation: Represent the data_validation components in both src/mlProject/components/data_validation.py and stage_02_data_validation.py, plus artifacts/data_validation.
- Data Transformation: Depicted via src/mlProject/components/data_transformation.py and stage_03_data_transformation.py and the corresponding artifacts/data_transformation.
- Model Training: From src/mlProject/components/model_trainer.py and stage_04_model_trainer.py (also artifact/model_trainer), including tools like scaler and model.joblib.
- Model Evaluation: Through src/mlProject/components/model_evaluation.py and stage_05_model_evaluation.py, connecting to artifacts/model_evaluation.
- Configuration Management:
- Includes config files (config/config.yaml, params.yaml, schema.yaml) and the configuration management modules in src/mlProject/config.
- Web Application Layer:
- Backend: app.py, main.py that likely manage routing or orchestration.
- Frontend: The templates folder (index.html, results.html) and the static folder (assets, js, css) to render the UI.
- Research and Notebooks: Represent the research directory housing Jupyter notebooks for exploratory data analysis, trials, etc.
- Artifacts: A dedicated folder structure storing intermediate files, serialized objects (joblibs), and processed data which interface with the ML pipeline.

##### Relationships and Data Flow
- Pipeline Flow:
- Data Collection/Ingestion → Data Validation → Data Transformation → Model Training → Model Evaluation.
- Artifact creation at each stage (such as CSVs, joblib files, metrics.json).
- Integration with the Web App:
- Once the ML model is trained and evaluated, the prediction pipeline (src/mlProject/pipeline/prediction.py) interfaces with the web application (app.py) to generate and display results.
- Feedback/Updates:
- A feedback loop where outputs of training/evaluation might influence updates to configuration or further research iterations.

##### Architectural Patterns and Design Principles
- Modularity and Separation of Concerns: Each pipeline stage is a self-contained component. The src folder encapsulates the domain logic (components, configuration, pipeline, utilities).
- Layered Architecture: Divide the system into a backend service (ML processing) and a presentation layer (web UI). Clearly delineate how data moves from raw input in “artifacts” and “config” to processed predictions rendered for the end-user.
- Extensibility: Indicate how the design allows new components to be plugged in (for instance, incorporating new validation checks or transformation methods) without overhauling the system.

##### Specify Relevant Technologies/Frameworks
- Python: The core programming language.
- Joblib: Used for model and vectorizer serialization (seen in artifacts).
- YAML: For configuration and parameters.
- Jupyter Notebooks: Used for research and experimentation (research folder).
- Possibly Flask (or a similar lightweight framework): Deduce from the presence of app.py, templates, and static folders.

##### Elements
- Each major component (Data Ingestion, Data Validation, Data Transformation, Model Training, Model Evaluation, Prediction Pipeline, Configuration Manager, Web Frontend, Artifacts Storage).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Configuration Layer
    subgraph "Configuration Layer"
        conf_files["Configuration Files (config/config.yaml, params.yaml, schema.yaml)"]:::config
        conf_module["Configuration Module (src/mlProject/config/configuration.py)"]:::config
    end

    %% ML Pipeline
    subgraph "ML Pipeline"
        di_module["Data Ingestion Module (src/mlProject/components/data_ingestion.py)"]:::pipeline
        di_pipeline["Data Inestion Pipeline (src/mlProject/pipeline/stage_01_data_ingestion.py)"]:::pipeline
        di_artifact["Data Ingestion Artifacts (artifacts/data_ingestion)"]:::artifact

        dv_module["Data Validation Module (src/mlProject/components/data_validation.py)"]:::pipeline
        dv_pipeline["Data Validation Pipeline (src/mlProject/pipeline/stage_02_data_validation.py)"]:::pipeline
        dv_artifact["Data Validation Artifacts (artifacts/data_validation)"]:::artifact

        dt_module["Data Transformation Module (src/mlProject/components/data_transformation.py)"]:::pipeline
        dt_pipeline["Data Transformation Pipeline (src/mlProject/pipeline/stage_03_data_transformation.py)"]:::pipeline
        dt_artifact["Data Transformation Artifacts (artifacts/data_transformation)"]:::artifact

        mt_module["Model Training Module (src/mlProject/components/model_trainer.py)"]:::pipeline
        mt_pipeline["Model Training Pipeline (src/mlProject/pipeline/stage_04_model_trainer.py)"]:::pipeline
        mt_artifact["Model Training Artifacts (artifacts/model_trainer)"]:::artifact

        me_module["Model Evaluation Module (src/mlProject/components/model_evaluation.py)"]:::pipeline
        me_pipeline["Model Evaluation Pipeline (src/mlProject/pipeline/stage_05_model_evaluation.py)"]:::pipeline
        me_artifact["Model Evaluation Artifacts (artifacts/model_evaluation)"]:::artifact

        pred_pipeline["Prediction Pipeline (src/mlProject/pipeline/prediction.py)"]:::pipeline
    end

    %% Web Application Layer
    subgraph "Web Application Layer"
        web_backend["Web App Backend (app.py, main.py)"]:::web
        web_frontend["Web App Frontend (templates & static)"]:::web
    end

    %% Research
    research_node["Research & Notebooks (research)"]:::research

    %% Data Flow Connections
    conf_files -->|"params"| di_module
    conf_module -->|"params"| di_module

    di_module -->|"ingest_process"| di_pipeline
    di_pipeline -->|"data_file"| di_artifact

    di_artifact -->|"validated_input"| dv_module
    dv_module -->|"validation_process"| dv_pipeline
    dv_pipeline -->|"clean_data"| dv_artifact

    dv_artifact -->|"transformation_input"| dt_module
    dt_module -->|"transformation_process"| dt_pipeline
    dt_pipeline -->|"transformation_output"| dt_artifact

    dt_artifact -->|"training_input"| mt_module
    mt_module -->|"training_process"| mt_pipeline
    mt_pipeline -->|"trained_model"| mt_artifact

    mt_artifact -->|"evaluation_input"| me_module
    me_module -->|"evaluation_process"| me_pipeline
    me_pipeline -->|"evaluation_results"| me_artifact

    me_artifact -->|"ready_model"| pred_pipeline
    pred_pipeline -->|"prediction"| web_backend
    web_backend -->|"results_display"| web_frontend

    %% Feedback Loop
    me_artifact -- "analysis" --- research_node
    research_node -- "feedback" --- conf_module

    %% Click Events for Component Mapping
    click di_module "https://github.com/gentlew1nd/pricespeculator/blob/main/src/mlProject/components/data_ingestion.py"
    click di_pipeline "https://github.com/gentlew1nd/pricespeculator/blob/main/src/mlProject/pipeline/stage_01_data_ingestion.py"
    click di_artifact "https://github.com/gentlew1nd/pricespeculator/tree/main/artifacts/data_ingestion"

    click dv_module "https://github.com/gentlew1nd/pricespeculator/blob/main/src/mlProject/components/data_validation.py"
    click dv_pipeline "https://github.com/gentlew1nd/pricespeculator/blob/main/src/mlProject/pipeline/stage_02_data_validation.py"
    click dv_artifact "https://github.com/gentlew1nd/pricespeculator/tree/main/artifacts/data_validation"

    click dt_module "https://github.com/gentlew1nd/pricespeculator/blob/main/src/mlProject/components/data_transformation.py"
    click dt_pipeline "https://github.com/gentlew1nd/pricespeculator/blob/main/src/mlProject/pipeline/stage_03_data_transformation.py"
    click dt_artifact "https://github.com/gentlew1nd/pricespeculator/tree/main/artifacts/data_transformation"

    click mt_module "https://github.com/gentlew1nd/pricespeculator/blob/main/src/mlProject/components/model_trainer.py"
    click mt_pipeline "https://github.com/gentlew1nd/pricespeculator/blob/main/src/mlProject/pipeline/stage_04_model_trainer.py"
    click mt_artifact "https://github.com/gentlew1nd/pricespeculator/tree/main/artifacts/model_trainer"

    click me_module "https://github.com/gentlew1nd/pricespeculator/blob/main/src/mlProject/components/model_evaluation.py"
    click me_pipeline "https://github.com/gentlew1nd/pricespeculator/blob/main/src/mlProject/pipeline/stage_05_data_evaluation.py"
    click me_artifact "https://github.com/gentlew1nd/pricespeculator/tree/main/artifacts/model_evaluation"

    click conf_files "https://github.com/gentlew1nd/pricespeculator/blob/main/config/config.yaml"
    click conf_module "https://github.com/gentlew1nd/pricespeculator/blob/main/src/mlProject/config/configuration.py"

    click pred_pipeline "https://github.com/gentlew1nd/pricespeculator/blob/main/src/mlProject/pipeline/prediction.py"

    click web_backend "https://github.com/gentlew1nd/pricespeculator/blob/main/app.py"
    click web_frontend "https://github.com/gentlew1nd/pricespeculator/blob/main/templates/index.html"

    click research_node "https://github.com/gentlew1nd/pricespeculator/tree/main/research"

    %% Styles
    classDef config fill:#c8e6c9,stroke:#388e3c,stroke-width:2px;
    classDef pipeline fill:#bbdefb,stroke:#1976d2,stroke-width:2px;
    classDef artifact fill:#fff9c4,stroke:#fbc02d,stroke-width:2px;
    classDef web fill:#ffcdd2,stroke:#d32f2f,stroke-width:2px;
    classDef research fill:#d1c4e9,stroke:#7b1fa2,stroke-width:2px;
```
