How facebookresearch/fairseq works
Fairseq is a sequence-modeling toolkit for training and using models in translation, language modeling, summarization, and speech. A typical workflow configures a task and model, loads task-specific data, trains with a criterion and optimizer, saves checkpoints, then generates or scores outputs. The repository also includes substantial multilingual, non-autoregressive, and speech capabilities. Relationships below are limited to README statements and source references visible in the supplied index or excerpts; unsampled implementation details are not asserted.
Main entry points: trainer.py Training engine · tasks/__init__.py Task registry · models/__init__.py Model registry · configs.py Config system
Components 22
Configuration and workflow
- Config system
fairseq/dataclass/configs.py - Task registry
fairseq/tasks/__init__.py - Checkpoint management
fairseq/checkpoint_utils.py - Training metrics
fairseq/logging/metrics.py
Data preparation
- Binarization
fairseq/binarizer.py - Data utilities
fairseq/data/data_utils.py - Indexed datasets
fairseq/data/indexed_dataset.py - Batch iterators
fairseq/data/iterators.py
Modeling and training
- Model registry
fairseq/models/__init__.py - Transformer models
fairseq/models/transformer/transformer_base.py - Attention module
fairseq/modules/multihead_attention.py - Training criteria
fairseq/criterions/fairseq_criterion.py - Training engine
fairseq/trainer.py - Optimizers
fairseq/optim/fairseq_optimizer.py - Distributed runtime
fairseq/distributed/utils.py
Generation and evaluation
- Sequence generation
fairseq/sequence_generator.py - Decoding search
fairseq/search.py - Output scoring
fairseq/scoring/bleu.py
Speech and multilingual
- Multilingual data
fairseq/data/multilingual/multilingual_data_manager.py - Speech models
fairseq/models/speech_to_speech/s2s_conformer.py - Speech datasets
fairseq/data/audio/speech_to_text_dataset.py
Other components
- Researcher
Connections 22
- Researcher → Config system: configures
- Researcher → Binarization: prepares data
- Config system → Task registry: sets task
fairseq/dataclass/initialize.py - Task registry → Multilingual data: loads multilingual data
- Binarization → Indexed datasets: writes datasets
- Data utilities → Indexed datasets: uses datasets
fairseq/data/data_utils.py - Multilingual data → Data utilities: uses data utilities
fairseq/data/multilingual/multilingual_data_manager.py - Task registry → Model registry: selects model
- Model registry → Transformer models: registers models
fairseq/models/__init__.py - Transformer models → Attention module: uses attention
- Task registry → Speech datasets: loads speech data
- Speech models → Attention module: uses attention
- Researcher → Training engine: trains models
- Training engine → Checkpoint management: saves and loads
fairseq/trainer.py - Training engine → Model registry: uses models
fairseq/trainer.py - Training engine → Optimizers: uses optimizer
fairseq/trainer.py - Training engine → Distributed runtime: uses distributed runtime
fairseq/trainer.py - Training engine → Training metrics: logs metrics
fairseq/trainer.py - Training engine → Batch iterators: consumes batches
- Researcher → Sequence generation: generates outputs
- Sequence generation → Decoding search: uses search
fairseq/search.py - Researcher → Output scoring: scores outputs