# ai4bharat/indicxlit architecture

> Architecture diagram of the GitHub repository ai4bharat/indicxlit, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ai4bharat/indicxlit
- Repository: https://github.com/ai4bharat/indicxlit

## Overview

IndicXlit is a multilingual transliteration product for converting between Roman and Indic scripts, with word-level inference exposed through Python, command-line, and hosted API interfaces. Its broader workflow prepares and filters parallel datasets, trains models, generates candidates, and evaluates results; the repository also documents mining and finetuning. Runtime excerpts support the inference and API relationships shown here. Training, mining, and some evaluation components are represented by their existing directories because their source wiring was not sampled.

Main entry points: [`app/ai4bharat/transliteration/rnn/engine.py`](https://github.com/ai4bharat/indicxlit/blob/master/app/ai4bharat/transliteration/rnn/engine.py) (RNN engine), [`Dataset_Format/Create_Aksharantar_JSONL.py`](https://github.com/ai4bharat/indicxlit/blob/master/Dataset_Format/Create_Aksharantar_JSONL.py) (Aksharantar formatting), [`inference/python/xlit_translit.py`](https://github.com/ai4bharat/indicxlit/blob/master/inference/python/xlit_translit.py) (Python interface), [`inference/cli/en-indic/generate_result_files.py`](https://github.com/ai4bharat/indicxlit/blob/master/inference/cli/en-indic/generate_result_files.py) (CLI inference)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime["Transliteration runtime"]
  node_python["Python interface<br/>[xlit_translit.py]"]
  node_cli["CLI inference"]
  node_api["HTTP API<br/>[api_expose.py]"]
  node_server["Server routes<br/>[xlit_server.py]"]
  node_rnn["RNN engine<br/>[engine.py]"]
  node_rnnmodel["RNN inference<br/>[core.py]"]
  node_transformer["Transformer engine<br/>[base_engine.py]"]
  node_transformerinfer["Transformer inference"]
end

subgraph group_data["Data preparation"]
  node_corpus["Corpus processing"]
  node_dataset["Aksharantar formatting"]
  node_ulca["ULCA formatting<br/>[ULCA_dataset.py]"]
  node_filtration["Data filtration"]
end

subgraph group_training["Model training"]
  node_training["Training workflow"]
end

subgraph group_evaluation["Evaluation and mining"]
  node_generation["Candidate generation"]
  node_evaluation["Rescoring evaluation"]
  node_mining["Transliteration mining"]
end

node_user(("User"))

node_user -->|"submits words"| node_python
node_python -->|"requests transliteration"| node_rnn
node_user -->|"provides input"| node_cli
node_cli -->|"requests inference"| node_rnn
node_user -->|"sends request"| node_api
node_api -->|"gets app"| node_server
node_server -->|"routes inference"| node_rnn
node_rnn -->|"runs inference"| node_rnnmodel
node_transformer -->|"initializes"| node_transformerinfer
node_corpus -->|"prepares corpus"| node_dataset
node_dataset -->|"formats pairs"| node_filtration
node_filtration -->|"prepares splits"| node_training
node_training -->|"trained model"| node_generation
node_generation -->|"produces candidates"| node_evaluation
node_mining -.->|"mined pairs"| node_dataset

click node_python "https://github.com/ai4bharat/indicxlit/blob/master/inference/python/xlit_translit.py"
click node_cli "https://github.com/ai4bharat/indicxlit/blob/master/inference/cli/en-indic/generate_result_files.py"
click node_api "https://github.com/ai4bharat/indicxlit/blob/master/app/api_expose.py"
click node_server "https://github.com/ai4bharat/indicxlit/blob/master/app/ai4bharat/transliteration/xlit_server.py"
click node_rnn "https://github.com/ai4bharat/indicxlit/blob/master/app/ai4bharat/transliteration/rnn/engine.py"
click node_rnnmodel "https://github.com/ai4bharat/indicxlit/blob/master/app/ai4bharat/transliteration/rnn/core.py"
click node_transformer "https://github.com/ai4bharat/indicxlit/blob/master/app/ai4bharat/transliteration/transformer/base_engine.py"
click node_transformerinfer "https://github.com/ai4bharat/indicxlit/blob/master/app/ai4bharat/transliteration/transformer/custom_interactive.py"
click node_corpus "https://github.com/ai4bharat/indicxlit/blob/master/transliterate_corpus/data_tok_norm/working/tokenize_and_normalize.py"
click node_dataset "https://github.com/ai4bharat/indicxlit/blob/master/Dataset_Format/Create_Aksharantar_JSONL.py"
click node_ulca "https://github.com/ai4bharat/indicxlit/blob/master/ULCA_Format/ULCA_dataset.py"
click node_filtration "https://github.com/ai4bharat/indicxlit/blob/master/model_training_scripts/data_filtration/refresh_data_train_all_test_valid.py"
click node_training "https://github.com/ai4bharat/indicxlit/tree/master/model_training_scripts"
click node_generation "https://github.com/ai4bharat/indicxlit/blob/master/model_training_scripts/generation/generate_result_files.py"
click node_evaluation "https://github.com/ai4bharat/indicxlit/blob/master/model_training_scripts/evaluate/evaluate_result_with_rescore_option.py"
click node_mining "https://github.com/ai4bharat/indicxlit/tree/master/data_mining/transliteration_mining_samanantar"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_python,node_cli,node_api,node_server,node_rnn,node_rnnmodel,node_transformer,node_transformerinfer,node_user toneBlue
class node_corpus,node_dataset,node_ulca,node_filtration toneAmber
class node_training toneMint
class node_generation,node_evaluation,node_mining toneRose
```

## Components

### Transliteration runtime

- [Python interface](https://github.com/ai4bharat/indicxlit/blob/master/inference/python/xlit_translit.py): `inference/python/xlit_translit.py`
- [CLI inference](https://github.com/ai4bharat/indicxlit/blob/master/inference/cli/en-indic/generate_result_files.py): `inference/cli/en-indic/generate_result_files.py`
- [HTTP API](https://github.com/ai4bharat/indicxlit/blob/master/app/api_expose.py): `app/api_expose.py`
- [Server routes](https://github.com/ai4bharat/indicxlit/blob/master/app/ai4bharat/transliteration/xlit_server.py): `app/ai4bharat/transliteration/xlit_server.py`
- [RNN engine](https://github.com/ai4bharat/indicxlit/blob/master/app/ai4bharat/transliteration/rnn/engine.py): `app/ai4bharat/transliteration/rnn/engine.py`
- [RNN inference](https://github.com/ai4bharat/indicxlit/blob/master/app/ai4bharat/transliteration/rnn/core.py): `app/ai4bharat/transliteration/rnn/core.py`
- [Transformer engine](https://github.com/ai4bharat/indicxlit/blob/master/app/ai4bharat/transliteration/transformer/base_engine.py): `app/ai4bharat/transliteration/transformer/base_engine.py`
- [Transformer inference](https://github.com/ai4bharat/indicxlit/blob/master/app/ai4bharat/transliteration/transformer/custom_interactive.py): `app/ai4bharat/transliteration/transformer/custom_interactive.py`

### Data preparation

- [Corpus processing](https://github.com/ai4bharat/indicxlit/blob/master/transliterate_corpus/data_tok_norm/working/tokenize_and_normalize.py): `transliterate_corpus/data_tok_norm/working/tokenize_and_normalize.py`
- [Aksharantar formatting](https://github.com/ai4bharat/indicxlit/blob/master/Dataset_Format/Create_Aksharantar_JSONL.py): `Dataset_Format/Create_Aksharantar_JSONL.py`
- [ULCA formatting](https://github.com/ai4bharat/indicxlit/blob/master/ULCA_Format/ULCA_dataset.py): `ULCA_Format/ULCA_dataset.py`
- [Data filtration](https://github.com/ai4bharat/indicxlit/blob/master/model_training_scripts/data_filtration/refresh_data_train_all_test_valid.py): `model_training_scripts/data_filtration/refresh_data_train_all_test_valid.py`

### Model training

- [Training workflow](https://github.com/ai4bharat/indicxlit/tree/master/model_training_scripts): `model_training_scripts`

### Evaluation and mining

- [Candidate generation](https://github.com/ai4bharat/indicxlit/blob/master/model_training_scripts/generation/generate_result_files.py): `model_training_scripts/generation/generate_result_files.py`
- [Rescoring evaluation](https://github.com/ai4bharat/indicxlit/blob/master/model_training_scripts/evaluate/evaluate_result_with_rescore_option.py): `model_training_scripts/evaluate/evaluate_result_with_rescore_option.py`
- [Transliteration mining](https://github.com/ai4bharat/indicxlit/tree/master/data_mining/transliteration_mining_samanantar): `data_mining/transliteration_mining_samanantar`

### Other components

- **User**

## Connections

- User → Python interface: submits words
- Python interface → RNN engine: requests transliteration
- User → CLI inference: provides input
- CLI inference → RNN engine: requests inference
- User → HTTP API: sends request
- HTTP API → Server routes: gets app
- Server routes → RNN engine: routes inference
- RNN engine → RNN inference: runs inference
- Transformer engine → Transformer inference: initializes
- Corpus processing → Aksharantar formatting: prepares corpus
- Aksharantar formatting → Data filtration: formats pairs
- Data filtration → Training workflow: prepares splits
- Training workflow → Candidate generation: trained model
- Candidate generation → Rescoring evaluation: produces candidates
- Transliteration mining → Aksharantar formatting: mined pairs