# kartikeyagrawal2007/punarvaani architecture

> Architecture diagram of the GitHub repository kartikeyagrawal2007/punarvaani, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kartikeyagrawal2007/punarvaani
- Repository: https://github.com/kartikeyagrawal2007/punarvaani

## Overview

Punarvaani is a Hindi visual-speech recognition system: silent video is cropped to mouth frames, a CTC vision model predicts Hindi phonemes, and a Sarvam language model reconstructs Devanagari text. Its runtime entry points support inference as well as dataset training and evaluation; data preparation, synthetic-data generation, and readiness checks are additional product capabilities. The graph combines sampled source references with README-described stages; unsampled model and preprocessing internals are not asserted beyond those documented relationships.

Main entry points: [`inference.py`](https://github.com/kartikeyagrawal2007/punarvaani/blob/main/inference.py) (Inference API), [`main.py`](https://github.com/kartikeyagrawal2007/punarvaani/blob/main/main.py) (CLI workflow), [`Data/dataset.py`](https://github.com/kartikeyagrawal2007/punarvaani/blob/main/Data/dataset.py) (Video dataset), [`face_cropper.py`](https://github.com/kartikeyagrawal2007/punarvaani/blob/main/face_cropper.py) (Face and mouth cropper)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime["Runtime and orchestration"]
  node_main["CLI workflow<br/>[main.py]"]
  node_api["Inference API<br/>[inference.py]"]
  node_vocab["HPI configuration<br/>[config.py]"]
end

subgraph group_vision["Video and visual recognition"]
  node_crop["Face and mouth cropper<br/>[face_cropper.py]"]
  node_geometry["Mouth geometry<br/>[mouth_geometry.py]"]
  node_preprocess["Batch lip preprocessing<br/>[preprocess_lips.py]"]
  node_visionmodel["ViT CTC recognizer<br/>[ML_VALLR.py]"]
  node_classic["Classic VALLR reference<br/>[VALLR.py]"]
end

subgraph group_language["Hindi language processing"]
  node_sarvam["Sarvam translator<br/>[Sarvam.py]"]
  node_g2p["Hindi G2P<br/>[hindi_g2p.py]"]
end

subgraph group_data["Data and readiness"]
  node_dataset["Video dataset<br/>[dataset.py]"]
  node_metrics["Evaluation metrics<br/>[metrics.py]"]
  node_synthetic["Synthetic dataset generator<br/>[synthetic.py]"]
  node_preflight["Dataset and adapter checks<br/>[preflight.py]"]
end

node_user(("Video user"))
node_transcript["Hindi transcript"]

node_user -->|"submits video"| node_api
node_api -->|"extracts mouth frames"| node_crop
node_crop -->|"computes crop boxes"| node_geometry
node_api -->|"runs recognition"| node_visionmodel
node_visionmodel -->|"passes phonemes"| node_sarvam
node_api -->|"translates phonemes"| node_sarvam
node_api -->|"loads Hindi inventory"| node_vocab
node_api -->|"returns transcription"| node_transcript
node_main -->|"builds datasets"| node_dataset
node_main -->|"trains or evaluates"| node_visionmodel
node_main -->|"computes error rates"| node_metrics
node_dataset -->|"converts transcripts"| node_g2p
node_dataset -->|"uses frame cropping"| node_crop
node_preprocess -->|"uses cropper"| node_crop
node_synthetic -->|"writes dataset format"| node_dataset
node_preflight -->|"checks phoneme handling"| node_g2p
node_preflight -->|"checks adapter"| node_sarvam
node_main -->|"loads phoneme inventory"| node_vocab
node_sarvam -->|"reconstructs Hindi text"| node_transcript

click node_main "https://github.com/kartikeyagrawal2007/punarvaani/blob/main/main.py"
click node_api "https://github.com/kartikeyagrawal2007/punarvaani/blob/main/inference.py"
click node_vocab "https://github.com/kartikeyagrawal2007/punarvaani/blob/main/config.py"
click node_dataset "https://github.com/kartikeyagrawal2007/punarvaani/blob/main/Data/dataset.py"
click node_crop "https://github.com/kartikeyagrawal2007/punarvaani/blob/main/face_cropper.py"
click node_geometry "https://github.com/kartikeyagrawal2007/punarvaani/blob/main/mouth_geometry.py"
click node_preprocess "https://github.com/kartikeyagrawal2007/punarvaani/blob/main/scripts/preprocess_lips.py"
click node_visionmodel "https://github.com/kartikeyagrawal2007/punarvaani/blob/main/Models/ML_VALLR.py"
click node_sarvam "https://github.com/kartikeyagrawal2007/punarvaani/blob/main/Models/Sarvam.py"
click node_g2p "https://github.com/kartikeyagrawal2007/punarvaani/blob/main/Data/hindi_g2p.py"
click node_metrics "https://github.com/kartikeyagrawal2007/punarvaani/blob/main/Data/metrics.py"
click node_synthetic "https://github.com/kartikeyagrawal2007/punarvaani/blob/main/Data/synthetic.py"
click node_preflight "https://github.com/kartikeyagrawal2007/punarvaani/blob/main/scripts/preflight.py"
click node_classic "https://github.com/kartikeyagrawal2007/punarvaani/blob/main/Models/VALLR.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_main,node_api,node_vocab,node_user toneBlue
class node_crop,node_geometry,node_preprocess,node_visionmodel,node_classic toneAmber
class node_sarvam,node_g2p toneMint
class node_dataset,node_metrics,node_synthetic,node_preflight toneRose
class node_transcript toneIndigo
```

## Components

### Runtime and orchestration

- [CLI workflow](https://github.com/kartikeyagrawal2007/punarvaani/blob/main/main.py): `main.py`
- [Inference API](https://github.com/kartikeyagrawal2007/punarvaani/blob/main/inference.py): `inference.py`
- [HPI configuration](https://github.com/kartikeyagrawal2007/punarvaani/blob/main/config.py): `config.py`

### Video and visual recognition

- [Face and mouth cropper](https://github.com/kartikeyagrawal2007/punarvaani/blob/main/face_cropper.py): `face_cropper.py`
- [Mouth geometry](https://github.com/kartikeyagrawal2007/punarvaani/blob/main/mouth_geometry.py): `mouth_geometry.py`
- [Batch lip preprocessing](https://github.com/kartikeyagrawal2007/punarvaani/blob/main/scripts/preprocess_lips.py): `scripts/preprocess_lips.py`
- [ViT CTC recognizer](https://github.com/kartikeyagrawal2007/punarvaani/blob/main/Models/ML_VALLR.py): `Models/ML_VALLR.py`
- [Classic VALLR reference](https://github.com/kartikeyagrawal2007/punarvaani/blob/main/Models/VALLR.py): `Models/VALLR.py`

### Hindi language processing

- [Sarvam translator](https://github.com/kartikeyagrawal2007/punarvaani/blob/main/Models/Sarvam.py): `Models/Sarvam.py`
- [Hindi G2P](https://github.com/kartikeyagrawal2007/punarvaani/blob/main/Data/hindi_g2p.py): `Data/hindi_g2p.py`

### Data and readiness

- [Video dataset](https://github.com/kartikeyagrawal2007/punarvaani/blob/main/Data/dataset.py): `Data/dataset.py`
- [Evaluation metrics](https://github.com/kartikeyagrawal2007/punarvaani/blob/main/Data/metrics.py): `Data/metrics.py`
- [Synthetic dataset generator](https://github.com/kartikeyagrawal2007/punarvaani/blob/main/Data/synthetic.py): `Data/synthetic.py`
- [Dataset and adapter checks](https://github.com/kartikeyagrawal2007/punarvaani/blob/main/scripts/preflight.py): `scripts/preflight.py`

### Other components

- **Video user**
- **Hindi transcript**

## Connections

- Video user → Inference API: submits video (evidence: `README.md`)
- Inference API → Face and mouth cropper: extracts mouth frames (evidence: `inference.py`)
- Face and mouth cropper → Mouth geometry: computes crop boxes (evidence: `face_cropper.py`)
- Inference API → ViT CTC recognizer: runs recognition (evidence: `inference.py`)
- ViT CTC recognizer → Sarvam translator: passes phonemes (evidence: `README.md`)
- Inference API → Sarvam translator: translates phonemes (evidence: `inference.py`)
- Inference API → HPI configuration: loads Hindi inventory (evidence: `inference.py`)
- Inference API → Hindi transcript: returns transcription (evidence: `README.md`)
- CLI workflow → Video dataset: builds datasets (evidence: `main.py`)
- CLI workflow → ViT CTC recognizer: trains or evaluates (evidence: `main.py`)
- CLI workflow → Evaluation metrics: computes error rates (evidence: `main.py`)
- Video dataset → Hindi G2P: converts transcripts (evidence: `Data/dataset.py`)
- Video dataset → Face and mouth cropper: uses frame cropping (evidence: `Data/dataset.py`)
- Batch lip preprocessing → Face and mouth cropper: uses cropper (evidence: `scripts/preprocess_lips.py`)
- Synthetic dataset generator → Video dataset: writes dataset format (evidence: `Data/synthetic.py`)
- Dataset and adapter checks → Hindi G2P: checks phoneme handling (evidence: `scripts/preflight.py`)
- Dataset and adapter checks → Sarvam translator: checks adapter (evidence: `scripts/preflight.py`)
- CLI workflow → HPI configuration: loads phoneme inventory (evidence: `main.py`)
- Sarvam translator → Hindi transcript: reconstructs Hindi text (evidence: `README.md`)
