How songjiexie/feature-alignment-toc works
This repository implements task-oriented image communication: an encoder sends features through a noisy channel to a task decoder, and feature-alignment methods support inference across independently trained, different-provider models. The README describes independent-system pretraining, server-based one-shot alignment, and on-device zero-shot alignment. The graph includes the shared training and evaluation path, model families, and anchor-based alignment. Some entry points and model files were not excerpted, so their documented capabilities are shown without speculative internal connections.
Main entry points: main_original.py Independent training · train.py Training loop · server_based_align.py Server alignment · utils.py Model selection
Components 18
System training
- Independent training
main_original.py - Training loop
train.py - Model checkpoints
utils.py
Feature models
- Image encoders
models/ResN_VGG_models.py - Mixed models
models/models.py - Task decoders
models/ResN_VGG_models.py - Neural building blocks
models/modules.py
Feature alignment
- Server alignment
server_based_align.py - Alignment methods
train.py - Aligner networks
models/align_models.py - Zero-shot encoder
models/align_models.py - Anchor features
utils.py - On-device alignment
main_on_device_alignment.py
Data and runtime
- Dataset loading
dataloader.py - Noisy channel
models/channel_model.py - Model selection
utils.py
Other components
- Researcher
- Image datasets
Connections 24
- Researcher → Independent training: starts training
README.md - Researcher → Server alignment: starts alignment
README.md - Researcher → On-device alignment: starts training
README.md - Independent training → Training loop: uses trainer
main_original.py - Independent training → Dataset loading: loads data
main_original.py - Independent training → Noisy channel: configures channel
main_original.py - Independent training → Model selection: selects models
main_original.py - Image datasets → Dataset loading: provides images
- Dataset loading → Training loop: supplies batches
train.py - Training loop → Noisy channel: transmits features
train.py - Training loop → Model checkpoints: saves models
train.py - Training loop → Aligner networks: uses aligners
train.py - Model selection → Image encoders: builds encoder
utils.py - Model selection → Task decoders: builds decoder
utils.py - Model selection → Mixed models: builds mixed models
utils.py - Server alignment → Model checkpoints: loads trained models
README.md - Server alignment → Alignment methods: fits alignment
README.md - Alignment methods → Aligner networks: uses learned aligners
train.py - Server alignment → Task decoders: cross-model inference
README.md - On-device alignment → Zero-shot encoder: uses zero-shot alignment
README.md - Zero-shot encoder → Image encoders: builds encoder
models/align_models.py - Zero-shot encoder → Anchor features: compares features
models/align_models.py - Aligner networks → Mixed models: supports mixed models
models/align_models.py - Neural building blocks → Image encoders: building blocks