How vcherepanova/tabular-feature-selection works
Project Type and Purpose
• This is a research-oriented machine-learning pipeline for benchmarking and evaluating feature selection methods on tabular deep learning models.
• It provides end-to-end support for:
• Data ingestion and augmentation (adding noise/extraneous features)
• Classical and deep-learning-based feature selection
• Downstream model training (MLP, FT-Transformer, ResNet)
• Hyperparameter tuning for both feature selector and downstream model
• Multi-seed experiment orchestration
A. Configuration Manager
• Hydra configs under config/ (dataset specs, model specs, tuning specs)
• Top-level config files: train_model.yaml, tune_config.yaml, etc.
B. Data Layer
• External data store (/data folder, populated as per README)
• Data loader & augmentation in deep_tabular/utils/data_tools.py
C. Feature Selection Layer
1. Classical FS (RandomForest, XGBoost, Lasso, Univariate)
– Entry point: train_classical.py mode=feature_selection
– Hypers from config/hyp_for_* and config/model/*.yaml
2. Deep FS (Deep Lasso, First-Layer Lasso, Attention Map)
– Entry point: train_deep_model.py mode=feature_selection
– Regularizers implemented in deep_lasso.py and deep_tabular/models/*.py
D. Downstream Training Layer
• Entry points:
– train_deep_model.py mode=downstream (MLP, FT-Transformer, ResNet)
– train_classical.py mode=downstream (classical models)
• Uses selected features (via importance_path + topk) or raw features
E. Hyperparameter Tuning Layer
1. Baseline tuner: tune_baseline.py
2. Full-pipeline tuner: tune_full_pipeline.py (joint FS + downstream)
3. Outputs best configs (best_config.json) and trial stats (trials.csv)
F. Orchestration / Multi-Seed Runner
• run_full_pipeline.py --multirun ...
• Loops over seeds, loads best configs, runs FS + downstream per seed
G. Launch Scripts
• Examples in launch/*.sh showing real job invocations
H. Results & Artifacts
• feature_importances.pt, stats.json, best_stats.json, final_stats.json
Component Interactions and Data Flow
1. User invokes a script (train_*/tune_*/run_*) with a Hydra config.
2. Hydra resolves configs → Configuration Manager feeds parameters to modules.
3. Data Layer reads raw datasets → optionally augments with extraneous features.
4. Depending on mode:
a. Feature Selection Layer consumes the data → outputs feature_importances.pt.
b. Downstream Training Layer consumes data (optionally filtered by FS) → outputs stats.json.
5. Tuning Layer wraps FS and/or downstream training in optimizer loops → outputs best configs & metrics.
6. Multi-Seed Runner iterates final pipeline over seeds using stored best configs → produces final_stats.json per seed.
Architectural Patterns & Principles
• Configuration-driven design (Hydra) separates code from parameters.
• Modular layers for data ingestion, FS, training, tuning, orchestration.
• Clear separation of classical vs. deep-learning FS components.
• Single responsibility: each top-level script focuses on one job type.
• Reusable utilities in deep_tabular/utils.
Essential Details Not to Omit
• Hydra’s central role in wiring parameters to each script.
• The dual-path FS design (classical vs. deep)
• The optional “importance_path + topk” link between FS and downstream training.
• Tuning layer that can optimize FS alone (tune_baseline), downstream alone, or both jointly (tune_full_pipeline).
• Multi-seed orchestration pulling best configs from tuning outputs.
• The utility modules (deep_tabular/utils and deep_lasso.py) as shared code dependencies.