# songjiexie/feature-alignment-toc architecture

> Architecture diagram of the GitHub repository songjiexie/feature-alignment-toc, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/songjiexie/feature-alignment-toc
- Repository: https://github.com/songjiexie/feature-alignment-toc

## Overview

This repository implements task-oriented image communication: an encoder sends features through a noisy channel to a task decoder, and feature-alignment methods support inference across independently trained, different-provider models. The README describes independent-system pretraining, server-based one-shot alignment, and on-device zero-shot alignment. The graph includes the shared training and evaluation path, model families, and anchor-based alignment. Some entry points and model files were not excerpted, so their documented capabilities are shown without speculative internal connections.

Main entry points: [`main_original.py`](https://github.com/songjiexie/feature-alignment-toc/blob/main/main_original.py) (Independent training), [`train.py`](https://github.com/songjiexie/feature-alignment-toc/blob/main/train.py) (Training loop), [`server_based_align.py`](https://github.com/songjiexie/feature-alignment-toc/blob/main/server_based_align.py) (Server alignment), [`utils.py`](https://github.com/songjiexie/feature-alignment-toc/blob/main/utils.py) (Model selection)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_training["System training"]
  node_pretrain["Independent training<br/>[main_original.py]"]
  node_trainer["Training loop<br/>[train.py]"]
  node_checkpoint[("Model checkpoints<br/>[utils.py]")]
end

subgraph group_models["Feature models"]
  node_encoder["Image encoders<br/>[ResN_VGG_models.py]"]
  node_mixed_models["Mixed models<br/>[models.py]"]
  node_decoder["Task decoders<br/>[ResN_VGG_models.py]"]
  node_modules["Neural building blocks<br/>[modules.py]"]
end

subgraph group_alignment["Feature alignment"]
  node_server_align["Server alignment"]
  node_align_fit["Alignment methods<br/>[train.py]"]
  node_aligner_models["Aligner networks<br/>[align_models.py]"]
  node_zero_shot["Zero-shot encoder<br/>[align_models.py]"]
  node_anchors["Anchor features<br/>[utils.py]"]
  node_on_device["On-device alignment"]
end

subgraph group_runtime["Data and runtime"]
  node_dataset_loader["Dataset loading<br/>[dataloader.py]"]
  node_channel["Noisy channel<br/>[channel_model.py]"]
  node_model_factory["Model selection<br/>[utils.py]"]
end

node_engineer(("Researcher"))
node_datasets[("Image datasets")]

node_engineer -->|"starts training"| node_pretrain
node_engineer -->|"starts alignment"| node_server_align
node_engineer -->|"starts training"| node_on_device
node_pretrain -->|"uses trainer"| node_trainer
node_pretrain -->|"loads data"| node_dataset_loader
node_pretrain -->|"configures channel"| node_channel
node_pretrain -->|"selects models"| node_model_factory
node_datasets -.->|"provides images"| node_dataset_loader
node_dataset_loader -->|"supplies batches"| node_trainer
node_trainer -->|"transmits features"| node_channel
node_trainer -->|"saves models"| node_checkpoint
node_trainer -->|"uses aligners"| node_aligner_models
node_model_factory -->|"builds encoder"| node_encoder
node_model_factory -->|"builds decoder"| node_decoder
node_model_factory -->|"builds mixed models"| node_mixed_models
node_server_align -->|"loads trained models"| node_checkpoint
node_server_align -->|"fits alignment"| node_align_fit
node_align_fit -->|"uses learned aligners"| node_aligner_models
node_server_align -->|"cross-model inference"| node_decoder
node_on_device -->|"uses zero-shot alignment"| node_zero_shot
node_zero_shot -->|"builds encoder"| node_encoder
node_zero_shot -->|"compares features"| node_anchors
node_aligner_models -->|"supports mixed models"| node_mixed_models
node_modules -.->|"building blocks"| node_encoder

click node_pretrain "https://github.com/songjiexie/feature-alignment-toc/blob/main/main_original.py"
click node_trainer "https://github.com/songjiexie/feature-alignment-toc/blob/main/train.py"
click node_checkpoint "https://github.com/songjiexie/feature-alignment-toc/blob/main/utils.py"
click node_dataset_loader "https://github.com/songjiexie/feature-alignment-toc/blob/main/dataloader.py"
click node_encoder "https://github.com/songjiexie/feature-alignment-toc/blob/main/models/ResN_VGG_models.py"
click node_mixed_models "https://github.com/songjiexie/feature-alignment-toc/blob/main/models/models.py"
click node_decoder "https://github.com/songjiexie/feature-alignment-toc/blob/main/models/ResN_VGG_models.py"
click node_channel "https://github.com/songjiexie/feature-alignment-toc/blob/main/models/channel_model.py"
click node_model_factory "https://github.com/songjiexie/feature-alignment-toc/blob/main/utils.py"
click node_server_align "https://github.com/songjiexie/feature-alignment-toc/blob/main/server_based_align.py"
click node_align_fit "https://github.com/songjiexie/feature-alignment-toc/blob/main/train.py"
click node_aligner_models "https://github.com/songjiexie/feature-alignment-toc/blob/main/models/align_models.py"
click node_zero_shot "https://github.com/songjiexie/feature-alignment-toc/blob/main/models/align_models.py"
click node_anchors "https://github.com/songjiexie/feature-alignment-toc/blob/main/utils.py"
click node_on_device "https://github.com/songjiexie/feature-alignment-toc/blob/main/main_on_device_alignment.py"
click node_modules "https://github.com/songjiexie/feature-alignment-toc/blob/main/models/modules.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_pretrain,node_trainer,node_checkpoint toneBlue
class node_encoder,node_mixed_models,node_decoder,node_modules,node_datasets toneAmber
class node_server_align,node_align_fit,node_aligner_models,node_zero_shot,node_anchors,node_on_device toneMint
class node_dataset_loader,node_channel,node_model_factory toneRose
class node_engineer toneIndigo
```

## Components

### System training

- [Independent training](https://github.com/songjiexie/feature-alignment-toc/blob/main/main_original.py): `main_original.py`
- [Training loop](https://github.com/songjiexie/feature-alignment-toc/blob/main/train.py): `train.py`
- [Model checkpoints](https://github.com/songjiexie/feature-alignment-toc/blob/main/utils.py): `utils.py`

### Feature models

- [Image encoders](https://github.com/songjiexie/feature-alignment-toc/blob/main/models/ResN_VGG_models.py): `models/ResN_VGG_models.py`
- [Mixed models](https://github.com/songjiexie/feature-alignment-toc/blob/main/models/models.py): `models/models.py`
- [Task decoders](https://github.com/songjiexie/feature-alignment-toc/blob/main/models/ResN_VGG_models.py): `models/ResN_VGG_models.py`
- [Neural building blocks](https://github.com/songjiexie/feature-alignment-toc/blob/main/models/modules.py): `models/modules.py`

### Feature alignment

- [Server alignment](https://github.com/songjiexie/feature-alignment-toc/blob/main/server_based_align.py): `server_based_align.py`
- [Alignment methods](https://github.com/songjiexie/feature-alignment-toc/blob/main/train.py): `train.py`
- [Aligner networks](https://github.com/songjiexie/feature-alignment-toc/blob/main/models/align_models.py): `models/align_models.py`
- [Zero-shot encoder](https://github.com/songjiexie/feature-alignment-toc/blob/main/models/align_models.py): `models/align_models.py`
- [Anchor features](https://github.com/songjiexie/feature-alignment-toc/blob/main/utils.py): `utils.py`
- [On-device alignment](https://github.com/songjiexie/feature-alignment-toc/blob/main/main_on_device_alignment.py): `main_on_device_alignment.py`

### Data and runtime

- [Dataset loading](https://github.com/songjiexie/feature-alignment-toc/blob/main/dataloader.py): `dataloader.py`
- [Noisy channel](https://github.com/songjiexie/feature-alignment-toc/blob/main/models/channel_model.py): `models/channel_model.py`
- [Model selection](https://github.com/songjiexie/feature-alignment-toc/blob/main/utils.py): `utils.py`

### Other components

- **Researcher**
- **Image datasets**

## Connections

- Researcher → Independent training: starts training (evidence: `README.md`)
- Researcher → Server alignment: starts alignment (evidence: `README.md`)
- Researcher → On-device alignment: starts training (evidence: `README.md`)
- Independent training → Training loop: uses trainer (evidence: `main_original.py`)
- Independent training → Dataset loading: loads data (evidence: `main_original.py`)
- Independent training → Noisy channel: configures channel (evidence: `main_original.py`)
- Independent training → Model selection: selects models (evidence: `main_original.py`)
- Image datasets → Dataset loading: provides images
- Dataset loading → Training loop: supplies batches (evidence: `train.py`)
- Training loop → Noisy channel: transmits features (evidence: `train.py`)
- Training loop → Model checkpoints: saves models (evidence: `train.py`)
- Training loop → Aligner networks: uses aligners (evidence: `train.py`)
- Model selection → Image encoders: builds encoder (evidence: `utils.py`)
- Model selection → Task decoders: builds decoder (evidence: `utils.py`)
- Model selection → Mixed models: builds mixed models (evidence: `utils.py`)
- Server alignment → Model checkpoints: loads trained models (evidence: `README.md`)
- Server alignment → Alignment methods: fits alignment (evidence: `README.md`)
- Alignment methods → Aligner networks: uses learned aligners (evidence: `train.py`)
- Server alignment → Task decoders: cross-model inference (evidence: `README.md`)
- On-device alignment → Zero-shot encoder: uses zero-shot alignment (evidence: `README.md`)
- Zero-shot encoder → Image encoders: builds encoder (evidence: `models/align_models.py`)
- Zero-shot encoder → Anchor features: compares features (evidence: `models/align_models.py`)
- Aligner networks → Mixed models: supports mixed models (evidence: `models/align_models.py`)
- Neural building blocks → Image encoders: building blocks
