How inwonakng/tdbench works
Project type and purpose
• This is a research‑oriented machine‑learning benchmarking framework (“TDBench”) for tabular data distillation and downstream classification.
• It is not a web or full‑stack application but rather a modular pipeline: data ingestion → preprocessing → distillation → representation learning → classifier training/HPO → result aggregation/visualization.
File structure
• Top‑level:
• scripts & Q*.py files: experiment entry points for different research questions
• config/: Hydra config hierarchy for data, encoder, distill, classifier, tuning
• tabdd/: application code (subpackages: config, data, distill, models, results, dashboard, scripts, tune, utils)
• figures/: generated plots
• requirements_*.txt: environment definitions
• Key patterns:
• Hydra for configuration management (config/ + @hydra.main scripts)
• Modular ML components under tabdd/models and tabdd/distill
• Data loader & preprocessing under tabdd/data
• Results loaders & plotters under tabdd/results and tabdd/dashboard
• CLI orchestration via python -m tabdd.[scripts|tune].*
README insights
• Entry points must be run with -m tabdd.xxx in a conda‑managed environment
• Hydra config overrides drive experiments (e.g. data.datasets, distill.methods)
• Extensibility points: add new YAML under config, new DatasetConfig, new DistillConfig, new Encoder subclasses
a. Main components
1. CLI / Experiment Launcher
– Entry scripts (Q*.py, tabdd/scripts/, tabdd/tune/)
2. Config Manager
– Hydra + YAML files under config/
3. Pipeline Orchestrator
– PipelineConfig + main() functions that wire subcomponents
– Ray integration for distributed execution/HPO
4. Data Module
– tabdd/data/load (OpenML/Kaggle), tabdd/data/preprocessing, TabularDataModule
– Outputs preprocessed batches or cached HDF5
5. Distillation Engine
– tabdd/distill methods (kmeans, random_sample, graph_cut, etc.)
– Produces distilled datasets or embeddings
6. Encoder & Classifier Trainers
– tabdd/models/encoder (autoencoders, multihead, GNN)
– tabdd/models/classifier (MLP, XGB, ResNet, Transformer)
– Wrapped in Hydra‑driven tuning (Ray Tune)
7. Results Store
– Filesystem directories (RAY_RESULT_DIR, results/), CSV/HDF5 records
8. Analysis & Visualization
– tabdd/results/load & tabdd/results/plot modules
– tabdd/dashboard for interactive dashboards
– figures/ output
b. Relationships & data flow
• CLI → Config Manager loads YAML → Pipeline Orchestrator
• Orchestrator invokes Data Module → raw data → preprocessing → cached features
• Orchestrator forks two paths:
1. Distillation Engine on raw or encoded features → distilled set
2. Encoder training on raw data → latent representations
• Classifier Trainer consumes either raw, distilled or encoded data → model training/HPO via Ray → metrics/results
• Results Store collects experiment outputs → passed to Analysis & Visualization → plots/figures
c. Architectural patterns & design principles
• Modular plugin pattern: new distill methods, encoders, classifiers drop into their folders and register via Hydra config
• Separation of concerns: data ingestion, transformation, model logic, result analysis
• Config‑driven execution: Hydra for reproducibility and easy parameter sweeps
• Distributed/HPO orchestration: Ray for parallelism
d. Relevant technologies
• Python 3.10, PyTorch, torch_geometric
• Hydra for configuration
• Ray Tune for HPO and distributed experiments
• OpenML/Kaggle APIs for data ingestion
• Matplotlib/Plotly for plotting, Jinja2 (?) for dashboard