# nvidia/nemo-text-processing architecture

> Architecture diagram of the GitHub repository nvidia/nemo-text-processing, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nvidia/nemo-text-processing
- Repository: https://github.com/nvidia/nemo-text-processing

## Overview

NeMo Text Processing provides weighted finite-state grammar-based text normalization (TN) and inverse text normalization (ITN), with grammars for many languages and semiotic classes. The user-facing workflow converts text between spoken-form words and written forms; ITN’s sampled entry point selects language-specific classification and verbalization grammars and uses a token parser. The repository also includes hybrid WFST/language-model rescoring and FST alignment utilities. The tree and README support these capabilities, but most TN and language-grammar implementations were not sampled, so their internal wiring is summarized.

Main entry points: [`nemo_text_processing/inverse_text_normalization/inverse_normalize.py`](https://github.com/nvidia/nemo-text-processing/blob/main/nemo_text_processing/inverse_text_normalization/inverse_normalize.py) (ITN entry point), [`nemo_text_processing/hybrid/wfst_lm_rescoring.py`](https://github.com/nvidia/nemo-text-processing/blob/main/nemo_text_processing/hybrid/wfst_lm_rescoring.py) (WFST-LM rescoring), [`nemo_text_processing/text_normalization`](https://github.com/nvidia/nemo-text-processing/tree/main/nemo_text_processing/text_normalization) (TN grammars), [`nemo_text_processing/inverse_text_normalization`](https://github.com/nvidia/nemo-text-processing/tree/main/nemo_text_processing/inverse_text_normalization) (ITN classifiers)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_normalization["Text normalization"]
  node_tn["TN grammars"]
  node_tn_classes["TN class rules"]
end

subgraph group_inverse["Inverse normalization"]
  node_itn["ITN entry point"]
  node_itn_classes["ITN classifiers"]
  node_token_parser["Token parser<br/>[token_parser.py]"]
  node_itn_verbalizers["ITN verbalizers"]
end

subgraph group_hybrid["Hybrid rescoring"]
  node_hybrid_rescore["WFST-LM rescoring"]
  node_hybrid_utils["Scoring utilities<br/>[utils.py]"]
  node_lm_scorer["MLM scorer<br/>[mlm_scorer.py]"]
end

subgraph group_utilities["FST utilities"]
  node_alignment["FST alignment<br/>[alignment.py]"]
end

node_user(("Text user"))
node_spoken["Spoken-form text"]
node_written["Written-form text"]

node_user -->|"submits text"| node_tn
node_tn -->|"applies grammars"| node_tn_classes
node_tn_classes -->|"produces text"| node_written
node_user -->|"submits text"| node_itn
node_itn -->|"selects language rules"| node_itn_classes
node_itn -->|"parses tokens"| node_token_parser
node_itn -->|"verbalizes tokens"| node_itn_verbalizers
node_itn -->|"returns text"| node_spoken
node_written -.->|"provides candidates"| node_hybrid_rescore
node_hybrid_rescore -->|"filters candidates"| node_hybrid_utils
node_hybrid_rescore -->|"scores candidates"| node_lm_scorer
node_user -.->|"requests alignment"| node_alignment

click node_tn "https://github.com/nvidia/nemo-text-processing/tree/main/nemo_text_processing/text_normalization"
click node_tn_classes "https://github.com/nvidia/nemo-text-processing/tree/main/nemo_text_processing/text_normalization"
click node_itn "https://github.com/nvidia/nemo-text-processing/blob/main/nemo_text_processing/inverse_text_normalization/inverse_normalize.py"
click node_itn_classes "https://github.com/nvidia/nemo-text-processing/tree/main/nemo_text_processing/inverse_text_normalization"
click node_token_parser "https://github.com/nvidia/nemo-text-processing/blob/main/nemo_text_processing/text_normalization/token_parser.py"
click node_itn_verbalizers "https://github.com/nvidia/nemo-text-processing/tree/main/nemo_text_processing/inverse_text_normalization"
click node_hybrid_rescore "https://github.com/nvidia/nemo-text-processing/blob/main/nemo_text_processing/hybrid/wfst_lm_rescoring.py"
click node_hybrid_utils "https://github.com/nvidia/nemo-text-processing/blob/main/nemo_text_processing/hybrid/utils.py"
click node_lm_scorer "https://github.com/nvidia/nemo-text-processing/blob/main/nemo_text_processing/hybrid/mlm_scorer.py"
click node_alignment "https://github.com/nvidia/nemo-text-processing/blob/main/nemo_text_processing/fst_alignment/alignment.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_tn,node_tn_classes,node_user toneBlue
class node_itn,node_itn_classes,node_token_parser,node_itn_verbalizers toneAmber
class node_hybrid_rescore,node_hybrid_utils,node_lm_scorer toneMint
class node_alignment toneRose
class node_spoken,node_written toneIndigo
```

## Components

### Text normalization

- [TN grammars](https://github.com/nvidia/nemo-text-processing/tree/main/nemo_text_processing/text_normalization): `nemo_text_processing/text_normalization`
- [TN class rules](https://github.com/nvidia/nemo-text-processing/tree/main/nemo_text_processing/text_normalization): `nemo_text_processing/text_normalization`

### Inverse normalization

- [ITN entry point](https://github.com/nvidia/nemo-text-processing/blob/main/nemo_text_processing/inverse_text_normalization/inverse_normalize.py): `nemo_text_processing/inverse_text_normalization/inverse_normalize.py`
- [ITN classifiers](https://github.com/nvidia/nemo-text-processing/tree/main/nemo_text_processing/inverse_text_normalization): `nemo_text_processing/inverse_text_normalization`
- [Token parser](https://github.com/nvidia/nemo-text-processing/blob/main/nemo_text_processing/text_normalization/token_parser.py): `nemo_text_processing/text_normalization/token_parser.py`
- [ITN verbalizers](https://github.com/nvidia/nemo-text-processing/tree/main/nemo_text_processing/inverse_text_normalization): `nemo_text_processing/inverse_text_normalization`

### Hybrid rescoring

- [WFST-LM rescoring](https://github.com/nvidia/nemo-text-processing/blob/main/nemo_text_processing/hybrid/wfst_lm_rescoring.py): `nemo_text_processing/hybrid/wfst_lm_rescoring.py`
- [Scoring utilities](https://github.com/nvidia/nemo-text-processing/blob/main/nemo_text_processing/hybrid/utils.py): `nemo_text_processing/hybrid/utils.py`
- [MLM scorer](https://github.com/nvidia/nemo-text-processing/blob/main/nemo_text_processing/hybrid/mlm_scorer.py): `nemo_text_processing/hybrid/mlm_scorer.py`

### FST utilities

- [FST alignment](https://github.com/nvidia/nemo-text-processing/blob/main/nemo_text_processing/fst_alignment/alignment.py): `nemo_text_processing/fst_alignment/alignment.py`

### Other components

- **Text user**
- **Spoken-form text**
- **Written-form text**

## Connections

- Text user → TN grammars: submits text
- TN grammars → TN class rules: applies grammars
- TN class rules → Written-form text: produces text
- Text user → ITN entry point: submits text
- ITN entry point → ITN classifiers: selects language rules
- ITN entry point → Token parser: parses tokens
- ITN entry point → ITN verbalizers: verbalizes tokens
- ITN entry point → Spoken-form text: returns text
- Written-form text → WFST-LM rescoring: provides candidates
- WFST-LM rescoring → Scoring utilities: filters candidates
- WFST-LM rescoring → MLM scorer: scores candidates
- Text user → FST alignment: requests alignment