How rsenatorov/ridev1 works
Project Type and Purpose
• This is a research-grade deep-learning framework for single-image depth estimation.
• It comprises a data pipeline (loading, augmentation), a two-stage neural network (ResNeXt encoder + custom RIDE decoder), training infrastructure (losses, optimizer, scheduler, logging), and multiple inference/visualization tools.
File Structure
Top-level components:
• dataset/ ← contains test set (used by data loader)
• logs/ ← training metrics and ASPP feature maps
• src/ ← all Python source: data loader, model definition, training & inference scripts
• src/network/ ← core model code (encoder, decoder, building blocks, loss)
• extract_aspp_feature_maps.py ← feature-map visualization utility
• train.py ← orchestrates training
• inference_example.py, inference_webcam.py ← offline and real-time inference
README for Architectural Insights
• Encoder: ResNeXt-101, pretrained, extracts multi-scale features.
• Decoder: combines upsampling, ASPP+strip-pooling, hybrid skip, LPG, auxiliary refinement.
• Loss: composite L1 + SSIM with learnable weights.
• Data Pipeline: RIDEDataset handles loading, resizing/cropping, augmentations, normalization.
• Inference & Visualization modules hook into model to produce depth maps and ASPP feature images.
Main Components & Their Relationships
A. Data Layer
– RIDEDataset (dataset.py)
– Preprocessing & Augmentations
B. Model Layer
– Encoder (ResNeXt-101)
– Decoder (RIDE)
• ASPPModule
• UpsampleConv
• HybridSkipBlock
• EdgeAttentionBlock
• LPG
• AuxRefinementBlock
– Loss Function (CompositeLoss)
C. Training Pipeline
– train.py
– Optimizer & Scheduler
– Logging & Checkpoints → logs/ & checkpoints/
D. Inference Pipeline
– inference_example.py (batch)
– inference_webcam.py (real-time)
– Post-processing & Visualization
E. Feature Visualization
– extract_aspp_feature_maps.py → logs/features/
F. External Dependencies
– PyTorch, torchvision, OpenCV, torchmetrics, matplotlib, tqdm
• Data → Model (Encoder+Decoder)
• Model + Loss → Training Loop → Checkpoints & Logs
• Saved Checkpoint → Inference Scripts → Depth Map Output
• Model Hooks → Feature-Map Extraction Utility → PNGs
Architectural Patterns
• Encoder–Decoder with skip connections
• Atrous Spatial Pyramid Pooling + strip convolutions for multi-scale context
• Attention gating & SE blocks for feature fusion
• Composite multi-term loss with learnable weights
Detailed Capturing
– Show multi-scale side outputs feeding into intermediate supervision.