How zhangtp1996/openfe_reproduce works
1. Repository purpose
- This is a tabular machine-learning experimentation repo centered on OpenFE: automated feature engineering plus downstream model training and evaluation.
- The codebase is organized around reproducible experiments for multiple datasets, rather than a single application runtime.
2. Core feature-engineering engine
- OpenFE.py, FeatureGenerator.py, and utils.py at the repo root form the main OpenFE implementation layer.
- This layer is responsible for generating candidate features, ranking/selecting them, and producing transformed datasets that are passed to model training.
3. Two experiment families
- There are two major execution paths:
- runs/: the general benchmark framework for “other experiments” in the paper.
- run_BNP/ and run_IEEE/: dataset-specific pipelines for the Kaggle-style BNP and IEEE experiments.
- These are separate boundaries because they have different data formats, evaluation steps, and output locations.
4. General benchmark pipeline (runs/)
- runs/FE_first_order.py and runs/FE_high_order.py generate first-order and higher-order features.
- runs/eval.py trains and scores models on the engineered features.
- runs/run_all.py and runs/shell_inst/*.sh provide orchestration for full sweeps over datasets.
- runs/clear.sh removes generated outputs, indicating the repo treats experiment outputs as disposable artifacts.
5. Data and model abstraction layer
- runs/lib/ contains reusable infrastructure for experiment execution:
- data.py for dataset loading/splitting,
- metrics.py for evaluation,
- env.py and util.py for runtime/config handling,
- deep.py plus node/ for neural/tabular model support.
- This layer isolates model-agnostic experiment logic from dataset- and model-specific runners.
6. Model zoo / training backends
- runs/bin/ contains wrappers for a range of baseline and comparison models: LightGBM, XGBoost, CatBoost, MLP, ResNet, TabNet, FT-Transformer, DCN2, AutoInt, NODE, GrowNet, SNN, and ensembles.
- Architecturally, these are interchangeable downstream consumers of the same tabular inputs and tuned parameter files.
7. Baselines are self-contained
- runs/baseline/ houses reproduced baseline methods such as SAFE, FCTree, AutoCross, myAutoFeat, and DCN2.
- Each baseline has its own subfolder and runner script, meaning baseline execution is intentionally separated from the OpenFE pipeline to keep their dependencies and behavior isolated.
8. Hyperparameter tuning and experiment state
- runs/tune_parameter.py and the runs/tuned_parameters/ tree store tuning configurations and run artifacts.
- The many default/, tuned/, tuning/, and *_ensemble/ directories show that the workflow relies on persisted trial state, best configs, checkpoints, metrics, and saved predictions.
- These folders are output/state boundaries rather than source logic.
9. Dataset-specific Kaggle pipelines
- run_BNP/ and run_IEEE/ are custom pipelines for the BNP and IEEE Kaggle datasets.
- They include separate feature-generation scripts (FE_first_order.py, FE_high_order.py) and evaluation scripts (eval_first_order.py, eval_high_order.py), reflecting dataset-specific preprocessing and submission generation.
- run_IEEE/main.py and IEEE_utils.py are the IEEE orchestration entry points, while BNP.sh and IEEE.sh are the top-level launchers.
10. External data dependencies
- The repository depends on external datasets in ./data/.
- Two input classes matter:
- Kaggle CSV/ZIP datasets for BNP and IEEE,
- .npy-based datasets for the remaining benchmarks.
- This means the code is compute-heavy and file-driven, with no network service or database component.
11. Artifact flow
- Typical flow is: raw dataset → feature generation/OpenFE → model training/evaluation → saved predictions/metrics/artifacts.
- Saved artifacts include stats.json, result, prediction arrays (p_train.npy, p_val.npy, p_test.npy), feature importance files, checkpoints, and submission CSVs for Kaggle tasks.
12. Operational model
- Execution is primarily via Python scripts and shell wrappers under Anaconda/Python 3.8.
- The repo is optimized for reproducible batch experiments on a local or cluster environment, with environment variables and LD_LIBRARY_PATH configured for native ML dependencies.
Main entry points: eval.py Eval · OpenFE.py OpenFE · run_all.py Run All · run_BNP BNP
Components 19
OpenFE Core
Main feature engineering implementation and shared utilities.
- OpenFE · Top-level OpenFE implementation and feature search entrypoint.
OpenFE.py - Feature Gen · Candidate feature construction and transformation logic.
FeatureGenerator.py - Utils · Shared helpers used by OpenFE and experiment runners.
utils.py
General Benchmarks
Reusable benchmark pipeline for tabular datasets and model comparison.
- FE First · First-order feature generation for benchmark experiments.
runs/FE_first_order.py - FE High · Higher-order feature generation for benchmark experiments.
runs/FE_high_order.py - Eval · Trains and scores downstream models on engineered features.
runs/eval.py - Run All · Batch execution driver for multi-dataset sweeps.
runs/run_all.py - Shell Scripts · Per-dataset shell launchers for benchmark runs.
runs/shell_inst - Data · Dataset loading, splitting, and input preparation.
runs/lib/data.py - Metrics · Metric calculation and score aggregation.
runs/lib/metrics.py - Env · Runtime and configuration handling for experiments.
runs/lib/env.py - Deep · Neural/tabular model support shared across runners.
runs/lib/deep.py - Model Zoo · Wrappers for LightGBM, XGBoost, CatBoost, MLP, ResNet, TabNet, FT-Transformer, NODE, and ensembles.
runs/bin - Baselines · Self-contained reproduced baseline methods such as SAFE, FCTree, AutoCross, myAutoFeat, and DCN2.
runs/baseline
Kaggle Pipelines
Dataset-specific BNP and IEEE execution paths.
Run State
Persisted tuning outputs, metrics, checkpoints, and predictions.
- Tuning · Parameter tuning entrypoints and experiment-state management.
runs/tune_parameter.py - Tuned Params · Stored configurations, checkpoints, metrics, and predictions for benchmark runs.
runs/tuned_parameters - Data Dir · External datasets consumed from the repository data directory.
Connections 24
- OpenFE → Feature Gen: uses · OpenFE delegates feature construction to generator logic.
- OpenFE → Utils: uses · Core feature engineering relies on shared utilities.
- FE First → OpenFE: invokes · Benchmark first-order runs call into the OpenFE engine.
- FE High → OpenFE: invokes · Higher-order runs call into the OpenFE engine.
- FE First → Data: loads · Feature generation reads dataset splits and inputs.
- FE High → Data: loads · Higher-order feature generation reads dataset splits and inputs.
- Eval → Metrics: scores · Evaluation uses shared metric computation.
- Eval → Model Zoo: trains · Evaluation fits downstream tabular model backends.
- Eval → Env: configures · Evaluation uses runtime and experiment configuration.
- Model Zoo → Deep: shares · Deep tabular models reuse the shared deep utility layer.
- Baselines → Data: loads · Baseline runners consume the same tabular datasets.
- Baselines → Metrics: compares · Baselines report results through shared metrics.
- Run All → FE First: orchestrates · Batch runner triggers first-order feature generation.
- Run All → FE High: orchestrates · Batch runner triggers higher-order feature generation.
- Run All → Eval: orchestrates · Batch runner launches model evaluation.
- Shell Scripts → Run All: wraps · Shell scripts provide dataset-specific launch wrappers.
- Tuning → Tuned Params: writes · Tuning persists best configs and trial artifacts.
- Eval → Tuned Params: reads · Evaluation consumes stored tuned parameter sets.
- BNP → Data Dir: reads · BNP pipeline consumes Kaggle input files from data.
- IEEE → Data Dir: reads · IEEE pipeline consumes Kaggle input files from data.
- BNP → OpenFE: uses · BNP feature generation reuses the OpenFE engine.
- IEEE → OpenFE: uses · IEEE feature generation reuses the OpenFE engine.
- BNP → Eval: evaluates · BNP-specific evaluation produces competition outputs.
- IEEE → Eval: evaluates · IEEE-specific evaluation produces competition outputs.