How yuchendoudou/tode works
Project type and purpose
• This is a research‐oriented deep learning pipeline in PyTorch for transparent object depth estimation using a Transformer backbone (TODE-Trans).
• Key use cases: dataset preparation, model training, evaluation on multiple datasets (ClearGrasp, Omniverse, TransCG), checkpoint management, and sample inference.
File structure
• Top‑level scripts:
• train.py – orchestrates training loops via config files
• inference.py / sample_inference.py – perform one‐off inference for evaluation or demos
• test.py – runs validation/testing protocols
• Directories by function:
• configs/ – YAML files defining hyperparameters, dataset splits, training/inference modes
• datasets/ – PyTorch Dataset wrappers for each source (cleargrasp, omniverse, transcg, mixed)
• models/ – network definitions (swin_transformer.py, Tode.py)
• utils/ – builder, data preparation, loss (criterion), metrics, logging, helper functions
• checkpoints/ – pretrained model weights for inference/testing
README for additional insights
• External dependencies: HDF5, OpenEXR, PyTorch 1.11.0
• Datasets must be downloaded and arranged under a root directory
• Commands show how configs drive different training/test scenarios
1. Dataset Management
• ClearGrasp Dataset Loader
• Omniverse Dataset Loader
• TransCG Dataset Loader
• MixedDataset wrapper
2. Configuration Engine
• YAML config parser (default.yaml, train_*.yaml, inference.yaml)
3. Model Builder
• Swin Transformer backbone
• TODE‐Trans head (depth estimation)
4. Training Pipeline
• DataLoader → Model → Loss (criterion) → Optimizer → Scheduler
• Checkpointer (save/load weights)
5. Evaluation / Metrics
• Depth metrics (e.g., abs error, rmse) computed in utils/metrics.py
• Logger (console + file) in utils/logger.py
6. Inference / Sample Inference
• Inference script using inference.yaml
• Output visualization / depth map export
7. External Dependencies
• GPU / CUDA
• HDF5 / OpenEXR libraries
• Config Engine → Training Pipeline & Inference
• Dataset Management → Training Pipeline (DataLoader)
• Training Pipeline → Model Builder
• Model Builder → Compute Graph (forward/backward)
• Training Pipeline → Checkpointer (save)
• Training Pipeline → Evaluation (metrics, logger)
• Inference Script → Model Builder (load weights) → Dataset → Output Visualizer
c. Architectural patterns & design principles:
• Modular separation: clear boundaries between data loading, model definition, training logic, and utilities
• Config-driven execution: YAML files allow reproducible experiments without code edits
• Checkpoint abstraction: copycheckpoint.py handles weight transfers; training scripts use checkpoint API
• Extensibility: new datasets/models can be added by extending datasets/ or models/ directories
d. Key technologies/frameworks:
• PyTorch 1.11.0 (torch, torchvision)
• HDF5, OpenEXR for data formats
• Python 3.x ecosystem (numpy, yaml, tqdm, etc.)
Final recommendations
• Both control flow (script → module) and data flow (images → depth maps)