How keyang8/witt works
Project Type and Purpose
• This is a PyTorch‐based ML research pipeline implementing a Wireless Image Transmission Transformer (WITT).
• Core goal: train and evaluate an end‐to‐end semantic communication model (encoder → noisy channel → decoder) on image datasets under varying channel conditions.
File Structure
• data/
– datasets.py: defines PyTorch Dataset classes and data‐loading logic for DIV2K, CIFAR10, Kodak, etc.
• loss/
– distortion.py: implements distortion metrics (MSE, MS‐SSIM).
• net/
– encoder.py & decoder.py: Transformer‐based modules for encoding/decoding image latents.
– channel.py: simulates wireless channels (AWGN, Rayleigh).
– modules.py: spatial modulation module (scales latents by CSI).
– network.py: orchestrates encoder + channel + decoder into the full WITT model.
• train.py: CLI entry point for training and testing loops.
• utils.py: shared utilities (logging, checkpointing, metric computations).
• results/: holds evaluation outputs and visualization scripts.
Extract Key Architecture Details from README
• Model variations: WITT (with spatial modulation) and WITT_W/O (without).
• External building blocks: Swin Transformer backbone, BPG + LDPC or capacity‐achieving channel codes.
• CLI parameters select dataset, model variant, channel type, SNR settings, distortion metric.
Main Components and Interactions
a. Data Ingestion
– Input: image datasets (DIV2K, CIFAR10, Kodak, etc.)
– Loader: data/datasets.py → DataLoader → minibatches
b. Model Forward Pass
– Encoder (net/encoder.py): Swin‐based feature extraction → bottleneck representation
– Spatial Modulation (net/modules.py): scale by CSI if WITT variant
– Channel Simulator (net/channel.py): apply AWGN or Rayleigh noise
– Decoder (net/decoder.py): reconstruct image from noisy latents
c. Loss Computation
– Distortion (loss/distortion.py): MSE or MS‐SSIM between output and ground truth
d. Training Loop (train.py)
– Optimizer & scheduler → backpropagate loss → update network (net/network.py)
– Logging and checkpointing via utils.py
e. Evaluation & Results
– Run forward pass on test set → save metrics and visual comparisons in results/
Architectural Patterns & Design Principles
• Modular separation: data, model, loss, utilities, results
• Single‐entry CLI (train.py) controlling both train and test pipelines
• Clear plug‐in points: choose channel type, distortion metric, or model variant via flags
• Reusable building blocks: encoder/decoder follow same interface, channel module is swappable