How eugenio-mercuriali/ml-helpers works
Project Type and Purpose
• This is an open-source Python helper library for machine learning projects (“ML-helpers”).
• It provides reusable patterns, utilities, and pipeline components (encoders, outlier removers, cross-validators, builders, factories, models, predictors).
File-Structure Analysis
• Top-level packages:
• core/
– base/: abstract base classes (_BaseFeature, _Builder, _Encoder, _Factory, _Model, _Predictor, _OutlierRemover)
– cross_validation/: generic and time-series cross-validation strategies
– predictor_processor/: end-to-end pipeline orchestration
• functions/: standalone feature-engineering utilities (e.g., feature_crosses.py, target_encoder.py)
• patterns/: design pattern implementations (e.g., multiton)
• predictor_processor contains subpackages:
• builders/: constructs pipelines
• encoders/: label/one-hot/cross-fold encoding
• factories/: dynamic instantiation of encoders, models, outlier removers
• models/: concrete ML models (linear, NGB regressor)
• outlier_removers/: IQR, isolation forest
• predict/: prediction wrappers (quantile regressor)
• utils/: helper functions
Key Insights from README
• “Patterns, utils and trick for ML projects.”
a. Main Components
– User Data Input
– core.base (abstract classes)
– core.cross_validation
– predictor_processor pipeline (builders → factories → encoders/outlier_removers/models → predict)
– functions/ and patterns/ as side modules for specific tasks
– External ML libraries (scikit-learn, pandas, numpy)
b. Relationships & Data Flow
1. User loads data → passes to PredictorBuilder
2. Builder uses Factory to instantiate OutlierRemover(s) → applies to data
3. Data flows to Encoder(s) → transformed features
4. Model(s) instantiated via ModelFactory → trained or loaded
5. Predictor applies model to encoded data → outputs predictions
6. CrossValidation module can wrap the entire pipeline for evaluation
c. Architectural Patterns
– Factory pattern (dynamic instantiation)
– Builder pattern (pipeline assembly)
– Strategy pattern (plug-in for encoders, removers, cross-Validators)
– “Multiton” (in patterns/) for shared single-key instances
d. Technologies & Frameworks
– Pure Python modules and packages
– Dependencies implied: scikit-learn, pandas, numpy, optionally LightGBM/NGBoost
– No web or database layer—library focused
Essential Detail Checklist
• Every subpackage and its responsibility clearly labeled
• Base classes and extension points called out
• Relationship between predictor_processor and core modules shown