# facebookresearch/fairseq architecture

> Architecture diagram of the GitHub repository facebookresearch/fairseq, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/facebookresearch/fairseq
- Repository: https://github.com/facebookresearch/fairseq

## Overview

Fairseq is a sequence-modeling toolkit for training and using models in translation, language modeling, summarization, and speech. A typical workflow configures a task and model, loads task-specific data, trains with a criterion and optimizer, saves checkpoints, then generates or scores outputs. The repository also includes substantial multilingual, non-autoregressive, and speech capabilities. Relationships below are limited to README statements and source references visible in the supplied index or excerpts; unsampled implementation details are not asserted.

Main entry points: [`fairseq/trainer.py`](https://github.com/facebookresearch/fairseq/blob/main/fairseq/trainer.py) (Training engine), [`fairseq/tasks/__init__.py`](https://github.com/facebookresearch/fairseq/blob/main/fairseq/tasks/__init__.py) (Task registry), [`fairseq/models/__init__.py`](https://github.com/facebookresearch/fairseq/blob/main/fairseq/models/__init__.py) (Model registry), [`fairseq/dataclass/configs.py`](https://github.com/facebookresearch/fairseq/blob/main/fairseq/dataclass/configs.py) (Config system)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_workflow["Configuration and workflow"]
  node_config["Config system<br/>[configs.py]"]
  node_task["Task registry<br/>[__init__.py]"]
  node_checkpoint["Checkpoint management"]
  node_metrics["Training metrics<br/>[metrics.py]"]
end

subgraph group_data["Data preparation"]
  node_preprocess["Binarization<br/>[binarizer.py]"]
  node_datautils["Data utilities<br/>[data_utils.py]"]
  node_indexed[("Indexed datasets<br/>[indexed_dataset.py]")]
  node_iterator["Batch iterators<br/>[iterators.py]"]
end

subgraph group_learning["Modeling and training"]
  node_models["Model registry<br/>[__init__.py]"]
  node_transformer["Transformer models"]
  node_attention["Attention module"]
  node_criteria["Training criteria"]
  node_trainer["Training engine<br/>[trainer.py]"]
  node_optim["Optimizers"]
  node_distributed["Distributed runtime<br/>[utils.py]"]
end

subgraph group_inference["Generation and evaluation"]
  node_generation["Sequence generation"]
  node_search["Decoding search<br/>[search.py]"]
  node_scoring["Output scoring<br/>[bleu.py]"]
end

subgraph group_speech["Speech and multilingual"]
  node_multilingual["Multilingual data"]
  node_speechmodels["Speech models<br/>[s2s_conformer.py]"]
  node_speechdata["Speech datasets"]
end

node_researcher(("Researcher"))

node_researcher -.->|"configures"| node_config
node_researcher -.->|"prepares data"| node_preprocess
node_config -->|"sets task"| node_task
node_task -.->|"loads multilingual data"| node_multilingual
node_preprocess -.->|"writes datasets"| node_indexed
node_datautils -->|"uses datasets"| node_indexed
node_multilingual -->|"uses data utilities"| node_datautils
node_task -.->|"selects model"| node_models
node_models -->|"registers models"| node_transformer
node_transformer -.->|"uses attention"| node_attention
node_task -.->|"loads speech data"| node_speechdata
node_speechmodels -.->|"uses attention"| node_attention
node_researcher -.->|"trains models"| node_trainer
node_trainer -->|"saves and loads"| node_checkpoint
node_trainer -->|"uses models"| node_models
node_trainer -->|"uses optimizer"| node_optim
node_trainer -->|"uses distributed runtime"| node_distributed
node_trainer -->|"logs metrics"| node_metrics
node_trainer -.->|"consumes batches"| node_iterator
node_researcher -.->|"generates outputs"| node_generation
node_generation -->|"uses search"| node_search
node_researcher -.->|"scores outputs"| node_scoring

click node_config "https://github.com/facebookresearch/fairseq/blob/main/fairseq/dataclass/configs.py"
click node_task "https://github.com/facebookresearch/fairseq/blob/main/fairseq/tasks/__init__.py"
click node_preprocess "https://github.com/facebookresearch/fairseq/blob/main/fairseq/binarizer.py"
click node_datautils "https://github.com/facebookresearch/fairseq/blob/main/fairseq/data/data_utils.py"
click node_indexed "https://github.com/facebookresearch/fairseq/blob/main/fairseq/data/indexed_dataset.py"
click node_iterator "https://github.com/facebookresearch/fairseq/blob/main/fairseq/data/iterators.py"
click node_multilingual "https://github.com/facebookresearch/fairseq/blob/main/fairseq/data/multilingual/multilingual_data_manager.py"
click node_models "https://github.com/facebookresearch/fairseq/blob/main/fairseq/models/__init__.py"
click node_transformer "https://github.com/facebookresearch/fairseq/blob/main/fairseq/models/transformer/transformer_base.py"
click node_attention "https://github.com/facebookresearch/fairseq/blob/main/fairseq/modules/multihead_attention.py"
click node_criteria "https://github.com/facebookresearch/fairseq/blob/main/fairseq/criterions/fairseq_criterion.py"
click node_trainer "https://github.com/facebookresearch/fairseq/blob/main/fairseq/trainer.py"
click node_optim "https://github.com/facebookresearch/fairseq/blob/main/fairseq/optim/fairseq_optimizer.py"
click node_distributed "https://github.com/facebookresearch/fairseq/blob/main/fairseq/distributed/utils.py"
click node_checkpoint "https://github.com/facebookresearch/fairseq/blob/main/fairseq/checkpoint_utils.py"
click node_generation "https://github.com/facebookresearch/fairseq/blob/main/fairseq/sequence_generator.py"
click node_search "https://github.com/facebookresearch/fairseq/blob/main/fairseq/search.py"
click node_scoring "https://github.com/facebookresearch/fairseq/blob/main/fairseq/scoring/bleu.py"
click node_metrics "https://github.com/facebookresearch/fairseq/blob/main/fairseq/logging/metrics.py"
click node_speechmodels "https://github.com/facebookresearch/fairseq/blob/main/fairseq/models/speech_to_speech/s2s_conformer.py"
click node_speechdata "https://github.com/facebookresearch/fairseq/blob/main/fairseq/data/audio/speech_to_text_dataset.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_config,node_task,node_checkpoint,node_metrics toneBlue
class node_preprocess,node_datautils,node_indexed,node_iterator toneAmber
class node_models,node_transformer,node_attention,node_criteria,node_trainer,node_optim,node_distributed toneMint
class node_generation,node_search,node_scoring toneRose
class node_multilingual,node_speechmodels,node_speechdata,node_researcher toneIndigo
```

## Components

### Configuration and workflow

- [Config system](https://github.com/facebookresearch/fairseq/blob/main/fairseq/dataclass/configs.py): `fairseq/dataclass/configs.py`
- [Task registry](https://github.com/facebookresearch/fairseq/blob/main/fairseq/tasks/__init__.py): `fairseq/tasks/__init__.py`
- [Checkpoint management](https://github.com/facebookresearch/fairseq/blob/main/fairseq/checkpoint_utils.py): `fairseq/checkpoint_utils.py`
- [Training metrics](https://github.com/facebookresearch/fairseq/blob/main/fairseq/logging/metrics.py): `fairseq/logging/metrics.py`

### Data preparation

- [Binarization](https://github.com/facebookresearch/fairseq/blob/main/fairseq/binarizer.py): `fairseq/binarizer.py`
- [Data utilities](https://github.com/facebookresearch/fairseq/blob/main/fairseq/data/data_utils.py): `fairseq/data/data_utils.py`
- [Indexed datasets](https://github.com/facebookresearch/fairseq/blob/main/fairseq/data/indexed_dataset.py): `fairseq/data/indexed_dataset.py`
- [Batch iterators](https://github.com/facebookresearch/fairseq/blob/main/fairseq/data/iterators.py): `fairseq/data/iterators.py`

### Modeling and training

- [Model registry](https://github.com/facebookresearch/fairseq/blob/main/fairseq/models/__init__.py): `fairseq/models/__init__.py`
- [Transformer models](https://github.com/facebookresearch/fairseq/blob/main/fairseq/models/transformer/transformer_base.py): `fairseq/models/transformer/transformer_base.py`
- [Attention module](https://github.com/facebookresearch/fairseq/blob/main/fairseq/modules/multihead_attention.py): `fairseq/modules/multihead_attention.py`
- [Training criteria](https://github.com/facebookresearch/fairseq/blob/main/fairseq/criterions/fairseq_criterion.py): `fairseq/criterions/fairseq_criterion.py`
- [Training engine](https://github.com/facebookresearch/fairseq/blob/main/fairseq/trainer.py): `fairseq/trainer.py`
- [Optimizers](https://github.com/facebookresearch/fairseq/blob/main/fairseq/optim/fairseq_optimizer.py): `fairseq/optim/fairseq_optimizer.py`
- [Distributed runtime](https://github.com/facebookresearch/fairseq/blob/main/fairseq/distributed/utils.py): `fairseq/distributed/utils.py`

### Generation and evaluation

- [Sequence generation](https://github.com/facebookresearch/fairseq/blob/main/fairseq/sequence_generator.py): `fairseq/sequence_generator.py`
- [Decoding search](https://github.com/facebookresearch/fairseq/blob/main/fairseq/search.py): `fairseq/search.py`
- [Output scoring](https://github.com/facebookresearch/fairseq/blob/main/fairseq/scoring/bleu.py): `fairseq/scoring/bleu.py`

### Speech and multilingual

- [Multilingual data](https://github.com/facebookresearch/fairseq/blob/main/fairseq/data/multilingual/multilingual_data_manager.py): `fairseq/data/multilingual/multilingual_data_manager.py`
- [Speech models](https://github.com/facebookresearch/fairseq/blob/main/fairseq/models/speech_to_speech/s2s_conformer.py): `fairseq/models/speech_to_speech/s2s_conformer.py`
- [Speech datasets](https://github.com/facebookresearch/fairseq/blob/main/fairseq/data/audio/speech_to_text_dataset.py): `fairseq/data/audio/speech_to_text_dataset.py`

### Other components

- **Researcher**

## Connections

- Researcher → Config system: configures
- Researcher → Binarization: prepares data
- Config system → Task registry: sets task (evidence: `fairseq/dataclass/initialize.py`)
- Task registry → Multilingual data: loads multilingual data
- Binarization → Indexed datasets: writes datasets
- Data utilities → Indexed datasets: uses datasets (evidence: `fairseq/data/data_utils.py`)
- Multilingual data → Data utilities: uses data utilities (evidence: `fairseq/data/multilingual/multilingual_data_manager.py`)
- Task registry → Model registry: selects model
- Model registry → Transformer models: registers models (evidence: `fairseq/models/__init__.py`)
- Transformer models → Attention module: uses attention
- Task registry → Speech datasets: loads speech data
- Speech models → Attention module: uses attention
- Researcher → Training engine: trains models
- Training engine → Checkpoint management: saves and loads (evidence: `fairseq/trainer.py`)
- Training engine → Model registry: uses models (evidence: `fairseq/trainer.py`)
- Training engine → Optimizers: uses optimizer (evidence: `fairseq/trainer.py`)
- Training engine → Distributed runtime: uses distributed runtime (evidence: `fairseq/trainer.py`)
- Training engine → Training metrics: logs metrics (evidence: `fairseq/trainer.py`)
- Training engine → Batch iterators: consumes batches
- Researcher → Sequence generation: generates outputs
- Sequence generation → Decoding search: uses search (evidence: `fairseq/search.py`)
- Researcher → Output scoring: scores outputs
