# eugenio-mercuriali/ml-helpers architecture

> Architecture diagram of the GitHub repository eugenio-mercuriali/ml-helpers, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/eugenio-mercuriali/ml-helpers
- Repository: https://github.com/eugenio-mercuriali/ml-helpers

## Overview

##### Project Type and Purpose
- This is an open-source Python helper library for machine learning projects (“ML-helpers”).
- It provides reusable patterns, utilities, and pipeline components (encoders, outlier removers, cross-validators, builders, factories, models, predictors).

##### File-Structure Analysis
- Top-level packages:
- core/
– base/: abstract base classes (_BaseFeature, _Builder, _Encoder, _Factory, _Model, _Predictor, _OutlierRemover)
– cross_validation/: generic and time-series cross-validation strategies
– predictor_processor/: end-to-end pipeline orchestration
- functions/: standalone feature-engineering utilities (e.g., feature_crosses.py, target_encoder.py)
- patterns/: design pattern implementations (e.g., multiton)
- predictor_processor contains subpackages:
- builders/: constructs pipelines
- encoders/: label/one-hot/cross-fold encoding
- factories/: dynamic instantiation of encoders, models, outlier removers
- models/: concrete ML models (linear, NGB regressor)
- outlier_removers/: IQR, isolation forest
- predict/: prediction wrappers (quantile regressor)
- utils/: helper functions

##### Key Insights from README
- “Patterns, utils and trick for ML projects.”

a. Main Components
– User Data Input
– core.base (abstract classes)
– core.cross_validation
– predictor_processor pipeline (builders → factories → encoders/outlier_removers/models → predict)
– functions/ and patterns/ as side modules for specific tasks
– External ML libraries (scikit-learn, pandas, numpy)

b. Relationships & Data Flow
1. User loads data → passes to PredictorBuilder
2. Builder uses Factory to instantiate OutlierRemover(s) → applies to data
3. Data flows to Encoder(s) → transformed features
4. Model(s) instantiated via ModelFactory → trained or loaded
5. Predictor applies model to encoded data → outputs predictions
6. CrossValidation module can wrap the entire pipeline for evaluation

c. Architectural Patterns
– Factory pattern (dynamic instantiation)
– Builder pattern (pipeline assembly)
– Strategy pattern (plug-in for encoders, removers, cross-Validators)
– “Multiton” (in patterns/) for shared single-key instances

d. Technologies & Frameworks
– Pure Python modules and packages
– Dependencies implied: scikit-learn, pandas, numpy, optionally LightGBM/NGBoost
– No web or database layer—library focused

##### Essential Detail Checklist
- Every subpackage and its responsibility clearly labeled
- Base classes and extension points called out
- Relationship between predictor_processor and core modules shown

## Diagram (Mermaid)

```mermaid
graph TB
    %% Main flow
    UserInput["User Data Input"]:::external
    Output["Predictions Output"]:::external

    %% Core Base Classes
    subgraph "Core Base Classes"
        BaseFeature["_BaseFeature"]:::core
        BuilderBase["_Builder"]:::core
        EncoderBase["_Encoder"]:::core
        FactoryBase["_Factory"]:::core
        ModelBase["_Model"]:::core
        OutlierRemoverBase["_OutlierRemover"]:::core
        PredictorBase["_Predictor"]:::core
    end

    %% Cross Validation
    subgraph "Cross Validation"
        CVBase["BaseCrossValidator"]:::core
        CVFactory["CrossValidatorFactory"]:::core
        subgraph "Time-Series CV"
            TSCVBase["BaseTSCV"]:::core
            PreqBlocks["PrequentialBlocks"]:::core
            PreqGrow["PrequentialGrowingWindow"]:::core
        end
    end

    %% Predictor Processor
    subgraph "Predictor Processor"
        subgraph "Builders"
            PredictorBuilder["PredictorBuilder"]:::orchestration
        end
        subgraph "Factories"
            EncoderFactory["EncoderFactory"]:::orchestration
            ModelFactory["ModelFactory"]:::orchestration
            OutlierRemoverFactory["OutlierRemoverFactory"]:::orchestration
        end
        subgraph "Encoders"
            CrossFoldEncoder["CrossFoldTargetEncoder"]:::pipeline
            OneHotEncoder["OneHotEncoder"]:::pipeline
        end
        subgraph "Outlier Removers"
            IQRRemover["IQRRemover"]:::pipeline
            IsolationForestRemover["IsolationForestRemover"]:::pipeline
        end
        subgraph "Models"
            LinearRegressor["LinearRegressor"]:::pipeline
            NGBRegressor["NGBoostRegressor"]:::pipeline
        end
        subgraph "Predict"
            QuantileRegressor["QuantileRegressor"]:::pipeline
            PredictorConfig["PredictorConfig"]:::util
            PredictorConstants["PredictorConstants"]:::util
        end
        subgraph "Utils"
            PPUtils["Utils"]:::util
        end
    end

    %% Side Modules
    Functions["functions/ (Utilities)"]:::util
    Patterns["patterns/ (Design Patterns)"]:::util

    %% External Dependencies
    subgraph "External Libraries"
        SKLearn["scikit-learn"]:::external
        Pandas["pandas"]:::external
        NumPy["numpy"]:::external
    end

    %% Pipeline Flow
    UserInput -->|"raw DataFrame"| PredictorBuilder
    PredictorBuilder -->|"uses"| OutlierRemoverFactory
    OutlierRemoverFactory -->|"instantiates"| IQRRemover
    OutlierRemoverFactory -->|"instantiates"| IsolationForestRemover
    IQRRemover -->|"cleaned DataFrame"| EncoderFactory
    IsolationForestRemover -->|"cleaned DataFrame"| EncoderFactory
    EncoderFactory -->|"instantiates"| CrossFoldEncoder
    EncoderFactory -->|"instantiates"| OneHotEncoder
    CrossFoldEncoder -->|"encoded features"| ModelFactory
    OneHotEncoder -->|"encoded features"| ModelFactory
    ModelFactory -->|"instantiates"| LinearRegressor
    ModelFactory -->|"instantiates"| NGBRegressor
    LinearRegressor -->|"fitted model"| QuantileRegressor
    NGBRegressor -->|"fitted model"| QuantileRegressor
    QuantileRegressor -->|"predictions"| Output

    %% Cross Validation wrapping
    CVFactory -->|"wraps pipeline"| PredictorBuilder
    CVBase -->|"strategy for"| CVFactory
    TSCVBase -->|"implements"| CVFactory
    PreqBlocks -->|"implements"| CVFactory
    PreqGrow -->|"implements"| CVFactory

    %% Side module connections
    Functions -->|"provide utilities"| PredictorBuilder
    Patterns -->|"provide Multiton"| FactoryBase

    %% External usage
    SKLearn -->|"used by"| ModelFactory
    Pandas -->|"data structures"| UserInput
    NumPy -->|"arrays"| UserInput

    %% Click Events
    click BaseFeature "https://github.com/eugenio-mercuriali/ml-helpers/blob/main/core/base/_base_feature.py"
    click BuilderBase "https://github.com/eugenio-mercuriali/ml-helpers/blob/main/core/base/_builder.py"
    click EncoderBase "https://github.com/eugenio-mercuriali/ml-helpers/blob/main/core/base/_encoder.py"
    click FactoryBase "https://github.com/eugenio-mercuriali/ml-helpers/blob/main/core/base/_factory.py"
    click ModelBase "https://github.com/eugenio-mercuriali/ml-helpers/blob/main/core/base/_model.py"
    click OutlierRemoverBase "https://github.com/eugenio-mercuriali/ml-helpers/blob/main/core/base/_outlier_remover.py"
    click PredictorBase "https://github.com/eugenio-mercuriali/ml-helpers/blob/main/core/base/_predictor.py"
    click CVBase "https://github.com/eugenio-mercuriali/ml-helpers/blob/main/core/cross_validation/base_crossvalidator.py"
    click CVFactory "https://github.com/eugenio-mercuriali/ml-helpers/blob/main/core/cross_validation/crossvalidator_factory.py"
    click TSCVBase "https://github.com/eugenio-mercuriali/ml-helpers/blob/main/core/cross_validation/time_series/base_tscv.py"
    click PreqBlocks "https://github.com/eugenio-mercuriali/ml-helpers/blob/main/core/cross_validation/time_series/prequential_blocks.py"
    click PreqGrow "https://github.com/eugenio-mercuriali/ml-helpers/blob/main/core/cross_validation/time_series/prequential_growing_window.py"
    click PredictorBuilder "https://github.com/eugenio-mercuriali/ml-helpers/blob/main/core/predictor_processor/builders/predictor_builder.py"
    click EncoderFactory "https://github.com/eugenio-mercuriali/ml-helpers/blob/main/core/predictor_processor/factories/encoder_factory.py"
    click ModelFactory "https://github.com/eugenio-mercuriali/ml-helpers/blob/main/core/predictor_processor/factories/model_factory.py"
    click OutlierRemoverFactory "https://github.com/eugenio-mercuriali/ml-helpers/blob/main/core/predictor_processor/factories/outlier_remover_factory.py"
    click CrossFoldEncoder "https://github.com/eugenio-mercuriali/ml-helpers/blob/main/core/predictor_processor/encoders/crossfold_target.py"
    click OneHotEncoder "https://github.com/eugenio-mercuriali/ml-helpers/blob/main/core/predictor_processor/encoders/one_hot.py"
    click IQRRemover "https://github.com/eugenio-mercuriali/ml-helpers/blob/main/core/predictor_processor/outlier_removers/iqr.py"
    click IsolationForestRemover "https://github.com/eugenio-mercuriali/ml-helpers/blob/main/core/predictor_processor/outlier_removers/isolation_forest.py"
    click LinearRegressor "https://github.com/eugenio-mercuriali/ml-helpers/blob/main/core/predictor_processor/models/linear_regressor.py"
    click NGBRegressor "https://github.com/eugenio-mercuriali/ml-helpers/blob/main/core/predictor_processor/models/ngb_regressor.py"
    click QuantileRegressor "https://github.com/eugenio-mercuriali/ml-helpers/blob/main/core/predictor_processor/predict/quantile_regressor.py"
    click PredictorConfig "https://github.com/eugenio-mercuriali/ml-helpers/blob/main/core/predictor_processor/predictor_config.py"
    click PredictorConstants "https://github.com/eugenio-mercuriali/ml-helpers/blob/main/core/predictor_processor/predictor_constants.py"
    click PPUtils "https://github.com/eugenio-mercuriali/ml-helpers/blob/main/core/predictor_processor/utils/utils.py"
    click Functions "https://github.com/eugenio-mercuriali/ml-helpers/tree/main/functions/"
    click Patterns "https://github.com/eugenio-mercuriali/ml-helpers/blob/main/patterns/multiton.py"

    %% Styles
    classDef core fill:#cce5ff,stroke:#333,stroke-width:1px
    classDef pipeline fill:#d4edda,stroke:#333,stroke-width:1px
    classDef orchestration fill:#fff3cd,stroke:#333,stroke-width:1px
    classDef util fill:#e2e3e5,stroke:#333,stroke-width:1px
    classDef external fill:#f8d7da,stroke:#333,stroke-width:1px
```
