# vcherepanova/tabular-feature-selection architecture

> Architecture diagram of the GitHub repository vcherepanova/tabular-feature-selection, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/vcherepanova/tabular-feature-selection
- Repository: https://github.com/vcherepanova/tabular-feature-selection

## Overview

##### Project Type and Purpose
- This is a research-oriented machine-learning pipeline for benchmarking and evaluating feature selection methods on tabular deep learning models.
- It provides end-to-end support for:
- Data ingestion and augmentation (adding noise/extraneous features)
- Classical and deep-learning-based feature selection
- Downstream model training (MLP, FT-Transformer, ResNet)
- Hyperparameter tuning for both feature selector and downstream model
- Multi-seed experiment orchestration

A. Configuration Manager
- Hydra configs under `config/` (dataset specs, model specs, tuning specs)
- Top-level config files: `train_model.yaml`, `tune_config.yaml`, etc.
B. Data Layer
- External data store (`/data` folder, populated as per README)
- Data loader & augmentation in `deep_tabular/utils/data_tools.py`
C. Feature Selection Layer
1. Classical FS (RandomForest, XGBoost, Lasso, Univariate)
– Entry point: `train_classical.py mode=feature_selection`
– Hypers from `config/hyp_for_*` and `config/model/*.yaml`
2. Deep FS (Deep Lasso, First-Layer Lasso, Attention Map)
– Entry point: `train_deep_model.py mode=feature_selection`
– Regularizers implemented in `deep_lasso.py` and `deep_tabular/models/*.py`
D. Downstream Training Layer
- Entry points:
– `train_deep_model.py mode=downstream` (MLP, FT-Transformer, ResNet)
– `train_classical.py mode=downstream` (classical models)
- Uses selected features (via `importance_path` + `topk`) or raw features
E. Hyperparameter Tuning Layer
1. Baseline tuner: `tune_baseline.py`
2. Full-pipeline tuner: `tune_full_pipeline.py` (joint FS + downstream)
3. Outputs best configs (`best_config.json`) and trial stats (`trials.csv`)
F. Orchestration / Multi-Seed Runner
- `run_full_pipeline.py --multirun ...`
- Loops over seeds, loads best configs, runs FS + downstream per seed
G. Launch Scripts
- Examples in `launch/*.sh` showing real job invocations
H. Results & Artifacts
- `feature_importances.pt`, `stats.json`, `best_stats.json`, `final_stats.json`

##### Component Interactions and Data Flow
1. User invokes a script (train_*/tune_*/run_*) with a Hydra config.
2. Hydra resolves configs → Configuration Manager feeds parameters to modules.
3. Data Layer reads raw datasets → optionally augments with extraneous features.
4. Depending on mode:
a. Feature Selection Layer consumes the data → outputs `feature_importances.pt`.
b. Downstream Training Layer consumes data (optionally filtered by FS) → outputs `stats.json`.
5. Tuning Layer wraps FS and/or downstream training in optimizer loops → outputs best configs & metrics.
6. Multi-Seed Runner iterates final pipeline over seeds using stored best configs → produces `final_stats.json` per seed.

##### Architectural Patterns & Principles
- Configuration-driven design (Hydra) separates code from parameters.
- Modular layers for data ingestion, FS, training, tuning, orchestration.
- Clear separation of classical vs. deep-learning FS components.
- Single responsibility: each top-level script focuses on one job type.
- Reusable utilities in `deep_tabular/utils`.

##### Essential Details Not to Omit
- Hydra’s central role in wiring parameters to each script.
- The dual-path FS design (classical vs. deep)
- The optional “importance_path + topk” link between FS and downstream training.
- Tuning layer that can optimize FS alone (`tune_baseline`), downstream alone, or both jointly (`tune_full_pipeline`).
- Multi-seed orchestration pulling best configs from tuning outputs.
- The utility modules (`deep_tabular/utils` and `deep_lasso.py`) as shared code dependencies.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Configuration Layer
    CFG["Configuration Manager\n(config/)"]:::config
    HYD["Hydra Resolve"]:::config

    %% Data Layer
    DATASTORE["External Data Store\n(/data/)"]:::data
    DATA["Data Loader & Augmentation\n(deep_tabular/utils/data_tools.py)"]:::data

    %% Feature Selection Layer
    subgraph "Feature Selection Layer"
        CLFS["Classical FS\n(train_classical.py)"]:::compute
        DLFS["Deep FS\n(train_deep_model.py)"]:::compute
    end

    %% Downstream Training Layer
    subgraph "Downstream Training Layer"
        CDS["Classical Downstream\n(train_classical.py)"]:::compute
        DDS["Deep Downstream\n(train_deep_model.py)"]:::compute
    end

    %% Hyperparameter Tuning Layer
    subgraph "Hyperparameter Tuning Layer"
        TBASE["Baseline Tuner\n(tune_baseline.py)"]:::compute
        TFULL["Full-Pipeline Tuner\n(tune_full_pipeline.py)"]:::compute
    end

    %% Orchestration & Launch
    RUN["Multi-Seed Runner\n(run_full_pipeline.py)"]:::orchestration
    LAUNCH["Launch Scripts\n(launch/)"]:::orchestration

    %% Results & Artifacts
    RES["Results & Artifacts\n(feature_importances.pt, stats.json,\nbest_stats.json, final_stats.json)"]:::results

    %% Data and Config Flow
    CFG -->|provides configs| HYD
    HYD -->|parameters| DATA
    DATASTORE --> DATA

    %% FS Flow
    DATA -->|raw data| CLFS
    DATA -->|raw data| DLFS
    HYD -->|FS params| CLFS
    HYD -->|FS params| DLFS
    CLFS -->|feature_importances.pt| RES
    DLFS -->|feature_importances.pt| RES

    %% Downstream Flow
    DATA -->|raw data| CDS
    CLFS -->|importance_path+topk| CDS
    DLFS -->|importance_path+topk| DDS
    DATA -->|raw data| DDS
    HYD -->|downstream params| CDS
    HYD -->|downstream params| DDS
    CDS -->|stats.json| RES
    DDS -->|stats.json| RES

    %% Tuning Flow
    CLFS -->|reports metrics| TBASE
    CDS -->|reports metrics| TBASE
    HYD -->|tuning configs| TBASE
    TBASE -->|best_config.json & trials.csv| RES

    CLFS -->|reports metrics| TFULL
    DLFS -->|reports metrics| TFULL
    CDS -->|reports metrics| TFULL
    DDS -->|reports metrics| TFULL
    HYD -->|tuning configs| TFULL
    TFULL -->|best_config.json & trials.csv| RES

    %% Multi-Seed Orchestration
    TFULL -->|best configs| RUN
    RUN -->|final_stats.json| RES
    HYD -->|orchestration configs| RUN

    %% Launch Scripts
    LAUNCH --> HYD

    %% Styles
    classDef config fill:#d0e1f9,stroke:#1f4e79,color:#1f4e79;
    classDef data fill:#d4efdf,stroke:#1e8449,color:#1e8449;
    classDef compute fill:#fdebd0,stroke:#d35400,color:#d35400;
    classDef orchestration fill:#ebedef,stroke:#7f8c8d,color:#7f8c8d;
    classDef results fill:#e8daef,stroke:#7d3c98,color:#7d3c98;

    %% Click Events
    click CFG "https://github.com/vcherepanova/tabular-feature-selection/tree/main/config/"
    click DATA "https://github.com/vcherepanova/tabular-feature-selection/blob/main/deep_tabular/utils/data_tools.py"
    click DATASTORE "https://github.com/vcherepanova/tabular-feature-selection/tree/main//data/"
    click CLFS "https://github.com/vcherepanova/tabular-feature-selection/blob/main/train_classical.py"
    click CLFS "https://github.com/vcherepanova/tabular-feature-selection/tree/main/config/hyp/"
    click CLFS "https://github.com/vcherepanova/tabular-feature-selection/tree/main/config/model/"
    click DLFS "https://github.com/vcherepanova/tabular-feature-selection/blob/main/train_deep_model.py"
    click DLFS "https://github.com/vcherepanova/tabular-feature-selection/blob/main/deep_lasso.py"
    click DLFS "https://github.com/vcherepanova/tabular-feature-selection/tree/main/deep_tabular/models/"
    click CDS "https://github.com/vcherepanova/tabular-feature-selection/blob/main/train_classical.py"
    click DDS "https://github.com/vcherepanova/tabular-feature-selection/blob/main/train_deep_model.py"
    click DDS "https://github.com/vcherepanova/tabular-feature-selection/tree/main/config/model_downstream/"
    click TBASE "https://github.com/vcherepanova/tabular-feature-selection/blob/main/tune_baseline.py"
    click TBASE "https://github.com/vcherepanova/tabular-feature-selection/blob/main/config/tune_config.yaml"
    click TFULL "https://github.com/vcherepanova/tabular-feature-selection/blob/main/tune_full_pipeline.py"
    click TFULL "https://github.com/vcherepanova/tabular-feature-selection/blob/main/config/tune_full_pipeline_config.yaml"
    click RUN "https://github.com/vcherepanova/tabular-feature-selection/blob/main/run_full_pipeline.py"
    click RUN "https://github.com/vcherepanova/tabular-feature-selection/blob/main/config/run_full_pipeline_config.yaml"
    click LAUNCH "https://github.com/vcherepanova/tabular-feature-selection/tree/main/launch/"
```
