How 1981rupal/lssn_research_model works
LSSN is a research implementation for aligning text- and image-conditioned latent trajectories during diffusion-model training. The workflow samples a timestep and noise, corrupts a clean latent, then trains a U-Net on fused conditioning while comparing intermediate features from separate text-only and image-only passes with an invariance loss. The README also describes symmetric cross-attention and dual-path conditioning; these are represented alongside the denoising workflow. The implementation uses dummy batches rather than a production data pipeline. Edges reflect the supplied excerpts and README; unsampled behavior is not inferred.
Main entry points: train_lssn.py Dual-path trainer · lssn_model.py LSSN U-Net · lssn_modules.py Transformer block · noise_schedule.py Noise schedule
Components 14
Training workflow
- Dummy batches
train_lssn.py - Dual-path trainer
train_lssn.py - Training losses
Diffusion model
- Noise schedule
noise_schedule.py - LSSN U-Net
lssn_model.py - Time conditioning
lssn_model.py - Residual blocks
lssn_model.py - Spatial transformer
lssn_model.py - Scale sampling blocks
lssn_model.py
Synchronization layers
- Transformer block
lssn_modules.py - Synchronization attention
lssn_modules.py - Feed-forward layer
lssn_modules.py
Training objectives
- Invariance loss
loss.py
Other components
- Researcher
Connections 16
- Researcher → Dummy batches: starts training
README.md - Dummy batches → Dual-path trainer: supplies batches
train_lssn.py - Dual-path trainer → Noise schedule: corrupts latents
train_lssn.py - Dual-path trainer → LSSN U-Net: runs three passes
train_lssn.py - Noise schedule → Dual-path trainer: returns noisy latent
train_lssn.py - LSSN U-Net → Dual-path trainer: returns prediction and features
train_lssn.py - Dual-path trainer → Invariance loss: compares path features
train_lssn.py - Invariance loss → Dual-path trainer: returns alignment loss
train_lssn.py - Dual-path trainer → Training losses: reports losses
train_lssn.py - LSSN U-Net → Time conditioning: embeds timesteps
lssn_model.py - LSSN U-Net → Residual blocks: applies residual blocks
lssn_model.py - LSSN U-Net → Spatial transformer: applies attention layers
lssn_model.py - LSSN U-Net → Scale sampling blocks: scales feature maps
lssn_model.py - Spatial transformer → Transformer block: runs transformer layers
lssn_model.py - Transformer block → Synchronization attention: applies cross-attention
lssn_modules.py - Transformer block → Feed-forward layer: applies feed-forward
lssn_modules.py