How feature-engine/feature_engine works
Main Components
• Library Core – The heart of the package lies in the “feature_engine” directory. It contains modules for various feature engineering functionality:
• Data Transformation/Creation (e.g., math_features, relative_features, cyclical_features, decision_tree_features)
• Preprocessing (e.g., imputation, encoding, discretisation, outlier handling, transformation, scaling)
• Feature Selection (e.g., drop features, correlated features, recursive elimination, and more)
• Datetime processing and Time Series features
• Wrappers and pipelines for integrating with Scikit-learn
• Base Classes and Shared Functionalities – Look at the “_base_transformers”, “_check_init_parameters” and “_docstrings” modules that establish common behaviors across transformers.
• Testing Suite – The “tests” directory holds many detailed unit and integration tests. It is critical to show how testing is related to every component to maintain robustness.
• Documentation Infrastructure – The “docs” folder (with Sphinx-specific files and custom extensions) defines how the documentation is built and hosted on Read the Docs.
• CI/CD and Contribution Tools – Directories like “.circleci”, “.github”, configuration files (pytest.ini, tox.ini, etc.), and setup scripts indicate continuous integration, testing, and deployment workflows.
• External Resources – The README links to online courses, notebooks, and external dependencies (like Scikit-learn) implying that user examples and training materials are an important part of the ecosystem.
Relationships and Interactions
• The library is built around a common design pattern based on fit/transform methods, similar to Scikit-learn. Illustrate how each transformer (in encoding, imputation, discretisation, etc.) inherits or builds upon the base classes.
• How the Pipeline (and make_pipeline) integrates multiple transformers into a sequential processing flow.
• Flow from raw input data → preprocessing → transformation/encoding → feature selection → final modelling (conceptually, even if the library itself does not provide a full modelling layer).
• Testing layer that runs against each of these components, ensuring that each module complies with expected behavior.
• Documentation build process and CI/CD pipelines as part of the extended “ecosystem” that ensures quality and maintainability.
Architectural Patterns and Design Principles
• Modular Design: The project is clearly divided into functional modules (e.g., encoding, imputation, discretisation) allowing independent development and testing.
• Layered Architecture: Core logic (the transformers) is isolated from the user interface (documentation and usage examples) and deployment/integration (CI/CD configurations).
• Adherence to Interfaces: The project follows the Scikit-learn transformer paradigm (fit/transform), so showing how each module conforms to these standards is imperative.
• Emphasis on Reusability: Many submodules like the mixins, base classes, and common documentation modules are aimed at reusability across feature engineering components.
Relevant Technologies, Frameworks, and Tools
• Python and Scikit-learn APIs.
• Sphinx for documentation.
• Pytest for unit testing.
• CI/CD tools (CircleCI, GitHub Actions) and related configuration files.
• Packaging components (setup.py, requirements.txt) that outline dependency management and installation.