# ntt123/light-speed architecture

> Architecture diagram of the GitHub repository ntt123/light-speed, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-08-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ntt123/light-speed
- Repository: https://github.com/ntt123/light-speed

## Overview

#### Purpose and entry points
Light Speed is a VITS-derived text-to-speech training and inference repository. Its distinguishing training path uses externally aligned phoneme durations to expand phoneme representations to acoustic frame resolution before synthesis.

Primary entry points:
- Single- or multi-GPU training: `train.py`
- Speech generation workflow: `inference.ipynb`
- Dataset preparation: `prepare_ljs_tfdata.ipynb`, `prepare_vbx_tfdata.ipynb`
- Duration-model training: `train_duration_model.ipynb`

#### Data preparation and alignment boundary
Training data preparation is notebook-driven and produces TensorFlow-oriented dataset artifacts consumed by the loader layer:
- `prepare_ljs_tfdata.ipynb`
- `prepare_vbx_tfdata.ipynb`
- `tfloader.py`

An external forced aligner, such as Montreal Forced Aligner, is required upstream to obtain phoneme-duration ground truth. This is a material boundary: alignment quality and phoneme/token conventions must remain compatible with the prepared dataset and model inputs.

#### Training orchestration
`train.py` is the training control plane. It coordinates data loading, model construction, losses, optimization/checkpoint behavior, and distributed execution. The documented runtime supports standard Python execution and PyTorch distributed launch via `torchrun`.

Key supporting modules:
- `train.py`
- `tfloader.py`
- `losses.py`
- `config.json`

`config.json` is the likely shared source for model/training dimensions and runtime parameters; changes must remain consistent with dataset feature shapes and model submodules.

#### Acoustic model and duration-conditioned expansion
The central neural architecture is organized in:
- `models.py`
- `modules.py`
- `attentions.py`

The model consumes phoneme-level information and aligned durations, then upsamples the phoneme sequence to frame-level conditioning. This replaces or reduces reliance on learned duration alignment during the primary TTS training path, favoring robustness over some potential speech-quality tradeoff.

Attention and reusable neural building blocks are separated from top-level model assembly, making `models.py` the integration boundary and `modules.py` / `attentions.py` the primary reusable architecture layers.

#### Latent flow and waveform/acoustic transformations
The VITS-style latent transformation path is represented by:
- `flow.py`
- `modules.py`
- `models.py`

The repository specifically modifies the WaveNet flow receptive field with dilated convolutions to capture longer-range interactions. Flow construction and its coupling with encoders/decoders should therefore be treated as a coordinated architecture: tensor channel counts, sequence masks, and latent dimensions must agree across these files.

Mel-spectrogram and related audio feature processing are isolated in:
- `mel_processing.py`
- `commons.py`

This forms the boundary between raw/audio-derived training targets and the neural model’s acoustic inputs or losses.

#### Inference and deployment dependencies
Inference is demonstrated rather than exposed as a standalone service:
- `inference.ipynb`
- `models.py`
- `mel_processing.py`

A deployed demo/checkpoint consumer must reproduce preprocessing, phoneme representation, duration prediction or supplied durations, and checkpoint configuration used during training. Pretrained voice demos are hosted externally on Hugging Face Spaces; the repository itself contains training and notebook-based inference workflows rather than service infrastructure.

Main entry points: [`models.py`](https://github.com/ntt123/light-speed/blob/main/models.py) (VITS model assembly), [`train.py`](https://github.com/ntt123/light-speed/blob/main/train.py) (Training control plane), [`inference.ipynb`](https://github.com/ntt123/light-speed/blob/main/inference.ipynb) (Speech generation), [`config.json`](https://github.com/ntt123/light-speed/blob/main/config.json) (Shared configuration)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Data & alignment"]
  node_aligner{{"External forced aligner<br/>alignment boundary"}}
  node_prep_ljs["LJS dataset preparation<br/>notebook"]
  node_prep_vbx["VBX dataset preparation<br/>notebook"]
  node_dataset[("Prepared TF datasets<br/>dataset artifacts")]
  node_loader["Dataset loader<br/>[tfloader.py]"]
end

subgraph group_training["Training"]
  node_config["Shared configuration<br/>[config.json]"]
  node_train["Training control plane<br/>training entry point<br/>[train.py]"]
  node_distributed["PyTorch distributed launch<br/>runtime"]
  node_losses["Training losses<br/>loss functions<br/>[losses.py]"]
end

subgraph group_model["Duration-conditioned VITS"]
  node_audio["Mel &amp; audio features<br/>audio processing<br/>[mel_processing.py]"]
  node_commons["Shared tensor utilities<br/>neural utilities<br/>[commons.py]"]
  node_models["VITS model assembly<br/>model integration<br/>[models.py]"]
  node_duration_expansion["Aligned duration expansion<br/>sequence expansion"]
  node_flow["Latent flow<br/>normalizing flow<br/>[flow.py]"]
  node_modules["Reusable neural modules<br/>neural layers<br/>[modules.py]"]
  node_attention["Attention layers<br/>attention components<br/>[attentions.py]"]
end

subgraph group_inference["Inference"]
  node_duration_training["Duration-model training<br/>notebook"]
  node_inference["Speech generation<br/>inference notebook<br/>[inference.ipynb]"]
end

node_aligner -->|"aligned durations"| node_prep_ljs
node_aligner -->|"aligned durations"| node_prep_vbx
node_prep_ljs -->|"writes"| node_dataset
node_prep_vbx -->|"writes"| node_dataset
node_dataset -->|"consumed by"| node_loader
node_loader -->|"batches"| node_train
node_config -->|"runtime settings"| node_train
node_distributed -.->|"launches"| node_train
node_train -->|"builds and optimizes"| node_models
node_audio -->|"acoustic targets"| node_train
node_losses -->|"objective"| node_train
node_config -->|"model dimensions"| node_models
node_models -->|"uses"| node_duration_expansion
node_duration_expansion -->|"frame-level conditioning"| node_flow
node_flow -->|"built from"| node_modules
node_models -->|"composes"| node_modules
node_models -->|"composes"| node_attention
node_commons -->|"shared operations"| node_audio
node_duration_training -.->|"duration predictor"| node_inference
node_models -->|"checkpointed model"| node_inference
node_audio -->|"preprocessing"| node_inference
node_config -->|"matching configuration"| node_inference

click node_prep_ljs "https://github.com/ntt123/light-speed/blob/main/prepare_ljs_tfdata.ipynb"
click node_prep_vbx "https://github.com/ntt123/light-speed/blob/main/prepare_vbx_tfdata.ipynb"
click node_loader "https://github.com/ntt123/light-speed/blob/main/tfloader.py"
click node_config "https://github.com/ntt123/light-speed/blob/main/config.json"
click node_train "https://github.com/ntt123/light-speed/blob/main/train.py"
click node_losses "https://github.com/ntt123/light-speed/blob/main/losses.py"
click node_audio "https://github.com/ntt123/light-speed/blob/main/mel_processing.py"
click node_commons "https://github.com/ntt123/light-speed/blob/main/commons.py"
click node_models "https://github.com/ntt123/light-speed/blob/main/models.py"
click node_flow "https://github.com/ntt123/light-speed/blob/main/flow.py"
click node_modules "https://github.com/ntt123/light-speed/blob/main/modules.py"
click node_attention "https://github.com/ntt123/light-speed/blob/main/attentions.py"
click node_duration_training "https://github.com/ntt123/light-speed/blob/main/train_duration_model.ipynb"
click node_inference "https://github.com/ntt123/light-speed/blob/main/inference.ipynb"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_aligner,node_prep_ljs,node_prep_vbx,node_dataset,node_loader toneBlue
class node_config,node_train,node_distributed,node_losses toneAmber
class node_audio,node_commons,node_models,node_duration_expansion,node_flow,node_modules,node_attention toneMint
class node_duration_training,node_inference toneRose
```

## Components

### Data & alignment

- **External forced aligner**: alignment boundary · Produces phoneme-duration ground truth compatible with dataset tokens.
- [LJS dataset preparation](https://github.com/ntt123/light-speed/blob/main/prepare_ljs_tfdata.ipynb): notebook · `prepare_ljs_tfdata.ipynb`
- [VBX dataset preparation](https://github.com/ntt123/light-speed/blob/main/prepare_vbx_tfdata.ipynb): notebook · `prepare_vbx_tfdata.ipynb`
- **Prepared TF datasets**: dataset artifacts · Phoneme, duration, and acoustic training features.
- [Dataset loader](https://github.com/ntt123/light-speed/blob/main/tfloader.py): loader · `tfloader.py`

### Training

- [Shared configuration](https://github.com/ntt123/light-speed/blob/main/config.json): configuration · Dimensions and runtime settings must match data and model shapes. · `config.json`
- [Training control plane](https://github.com/ntt123/light-speed/blob/main/train.py): training entry point · Coordinates loading, model construction, losses, optimization, checkpoints, and distributed execution. · `train.py`
- **PyTorch distributed launch**: runtime · Optional multi-GPU execution via torchrun.
- [Training losses](https://github.com/ntt123/light-speed/blob/main/losses.py): loss functions · `losses.py`

### Duration-conditioned VITS

- [Mel & audio features](https://github.com/ntt123/light-speed/blob/main/mel_processing.py): audio processing · Converts audio-derived targets and acoustic features for model training. · `mel_processing.py`
- [Shared tensor utilities](https://github.com/ntt123/light-speed/blob/main/commons.py): neural utilities · `commons.py`
- [VITS model assembly](https://github.com/ntt123/light-speed/blob/main/models.py): model integration · Connects encoders, duration-conditioned expansion, flow, and decoder. · `models.py`
- **Aligned duration expansion**: sequence expansion · Upsamples phoneme representations to acoustic frame resolution using aligned durations.
- [Latent flow](https://github.com/ntt123/light-speed/blob/main/flow.py): normalizing flow · WaveNet-style dilated-convolution flow captures longer-range interactions. · `flow.py`
- [Reusable neural modules](https://github.com/ntt123/light-speed/blob/main/modules.py): neural layers · `modules.py`
- [Attention layers](https://github.com/ntt123/light-speed/blob/main/attentions.py): attention components · `attentions.py`

### Inference

- [Duration-model training](https://github.com/ntt123/light-speed/blob/main/train_duration_model.ipynb): notebook · Trains duration prediction for inference when durations are not supplied. · `train_duration_model.ipynb`
- [Speech generation](https://github.com/ntt123/light-speed/blob/main/inference.ipynb): inference notebook · Reproduces training preprocessing, phonemes, durations, and checkpoint configuration. · `inference.ipynb`

## Connections

- External forced aligner → LJS dataset preparation: aligned durations
- External forced aligner → VBX dataset preparation: aligned durations
- LJS dataset preparation → Prepared TF datasets: writes
- VBX dataset preparation → Prepared TF datasets: writes
- Prepared TF datasets → Dataset loader: consumed by
- Dataset loader → Training control plane: batches
- Shared configuration → Training control plane: runtime settings
- PyTorch distributed launch → Training control plane: launches
- Training control plane → VITS model assembly: builds and optimizes
- Mel & audio features → Training control plane: acoustic targets
- Training losses → Training control plane: objective
- Shared configuration → VITS model assembly: model dimensions
- VITS model assembly → Aligned duration expansion: uses
- Aligned duration expansion → Latent flow: frame-level conditioning
- Latent flow → Reusable neural modules: built from
- VITS model assembly → Reusable neural modules: composes
- VITS model assembly → Attention layers: composes
- Shared tensor utilities → Mel & audio features: shared operations
- Duration-model training → Speech generation: duration predictor
- VITS model assembly → Speech generation: checkpointed model
- Mel & audio features → Speech generation: preprocessing
- Shared configuration → Speech generation: matching configuration