# fawad-laal/qalb-urdu architecture

> Architecture diagram of the GitHub repository fawad-laal/qalb-urdu, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/fawad-laal/qalb-urdu
- Repository: https://github.com/fawad-laal/qalb-urdu

## Overview

This repository is an independent bilingual evaluation framework for Qalb, rather than the model itself. The primary workflow runs Urdu-script and Roman Urdu benchmark cases against Qalb, scores the responses, and saves results for analysis and reporting. Its documented scope includes eight task categories, iterative assessment rounds, performance analysis, and PDF and GPT-assisted Markdown reports. The README documents the runner’s role and outputs; the runner implementation was not sampled, so internal execution and scoring links are shown only at that documented level.

Main entry points: [`scripts/test_runner.py`](https://github.com/fawad-laal/qalb-urdu/blob/main/scripts/test_runner.py) (Test runner), [`scripts/generate_final_report.py`](https://github.com/fawad-laal/qalb-urdu/blob/main/scripts/generate_final_report.py) (GPT Markdown report), [`scripts/analyze_round4.py`](https://github.com/fawad-laal/qalb-urdu/blob/main/scripts/analyze_round4.py) (Round analysis), [`scripts/analyze_low_performers.py`](https://github.com/fawad-laal/qalb-urdu/blob/main/scripts/analyze_low_performers.py) (Failure analysis)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_evaluation["Evaluation workflow"]
  node_runner["Test runner<br/>[test_runner.py]"]
  node_qalb{{"Qalb model"}}
  node_scoring["Response scoring<br/>[test_runner.py]"]
  node_results["Run results"]
end

subgraph group_benchmarks["Benchmark inputs"]
  node_urdu_cases["Urdu cases"]
  node_roman_cases["Roman Urdu cases"]
  node_categories["Task categories<br/>[test_cases.py]"]
end

subgraph group_analysis["Analysis"]
  node_round_analysis["Round analysis<br/>[analyze_round4.py]"]
  node_low_analysis["Failure analysis"]
  node_round2_analysis["Round 2 analysis<br/>[analyze_round2.py]"]
end

subgraph group_reporting["Reporting"]
  node_collector["Data collector<br/>[data_collector.py]"]
  node_report_data["Report data"]
  node_pdf_report["PDF report"]
  node_academic_pdf["Academic report"]
  node_markdown_report["GPT Markdown report"]
end

node_evaluator(("Evaluator"))
node_openai{{"OpenAI service"}}

node_evaluator -->|"starts run"| node_runner
node_runner -->|"loads cases"| node_urdu_cases
node_runner -->|"loads cases"| node_roman_cases
node_runner -->|"uses categories"| node_categories
node_runner -->|"sends prompts"| node_qalb
node_runner -->|"scores responses"| node_scoring
node_scoring -->|"records scores"| node_results
node_round_analysis -->|"analyzes results"| node_results
node_low_analysis -->|"analyzes failures"| node_results
node_round2_analysis -->|"analyzes results"| node_results
node_collector -->|"aggregates and saves"| node_report_data
node_report_data -->|"loads report data"| node_pdf_report
node_results -->|"provides results"| node_markdown_report
node_markdown_report -.->|"requests analysis"| node_openai
node_report_data -.->|"supplies evaluation content"| node_academic_pdf

click node_runner "https://github.com/fawad-laal/qalb-urdu/blob/main/scripts/test_runner.py"
click node_scoring "https://github.com/fawad-laal/qalb-urdu/blob/main/scripts/test_runner.py"
click node_urdu_cases "https://github.com/fawad-laal/qalb-urdu/blob/main/tests/baseline/urdu_script_tests.json"
click node_roman_cases "https://github.com/fawad-laal/qalb-urdu/blob/main/tests/baseline/roman_urdu_tests.json"
click node_categories "https://github.com/fawad-laal/qalb-urdu/blob/main/tests/test_cases.py"
click node_round_analysis "https://github.com/fawad-laal/qalb-urdu/blob/main/scripts/analyze_round4.py"
click node_low_analysis "https://github.com/fawad-laal/qalb-urdu/blob/main/scripts/analyze_low_performers.py"
click node_round2_analysis "https://github.com/fawad-laal/qalb-urdu/blob/main/scripts/analyze_round2.py"
click node_collector "https://github.com/fawad-laal/qalb-urdu/blob/main/scripts/data_collector.py"
click node_pdf_report "https://github.com/fawad-laal/qalb-urdu/blob/main/scripts/generate_pdf_report.py"
click node_academic_pdf "https://github.com/fawad-laal/qalb-urdu/blob/main/scripts/generate_academic_pdf.py"
click node_markdown_report "https://github.com/fawad-laal/qalb-urdu/blob/main/scripts/generate_final_report.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_runner,node_qalb,node_scoring,node_results toneBlue
class node_urdu_cases,node_roman_cases,node_categories toneAmber
class node_round_analysis,node_low_analysis,node_round2_analysis toneMint
class node_collector,node_report_data,node_pdf_report,node_academic_pdf,node_markdown_report toneRose
class node_evaluator,node_openai toneIndigo
```

## Components

### Evaluation workflow

- [Test runner](https://github.com/fawad-laal/qalb-urdu/blob/main/scripts/test_runner.py): `scripts/test_runner.py`
- **Qalb model**
- [Response scoring](https://github.com/fawad-laal/qalb-urdu/blob/main/scripts/test_runner.py): `scripts/test_runner.py`
- **Run results**

### Benchmark inputs

- [Urdu cases](https://github.com/fawad-laal/qalb-urdu/blob/main/tests/baseline/urdu_script_tests.json): `tests/baseline/urdu_script_tests.json`
- [Roman Urdu cases](https://github.com/fawad-laal/qalb-urdu/blob/main/tests/baseline/roman_urdu_tests.json): `tests/baseline/roman_urdu_tests.json`
- [Task categories](https://github.com/fawad-laal/qalb-urdu/blob/main/tests/test_cases.py): `tests/test_cases.py`

### Analysis

- [Round analysis](https://github.com/fawad-laal/qalb-urdu/blob/main/scripts/analyze_round4.py): `scripts/analyze_round4.py`
- [Failure analysis](https://github.com/fawad-laal/qalb-urdu/blob/main/scripts/analyze_low_performers.py): `scripts/analyze_low_performers.py`
- [Round 2 analysis](https://github.com/fawad-laal/qalb-urdu/blob/main/scripts/analyze_round2.py): `scripts/analyze_round2.py`

### Reporting

- [Data collector](https://github.com/fawad-laal/qalb-urdu/blob/main/scripts/data_collector.py): `scripts/data_collector.py`
- **Report data**
- [PDF report](https://github.com/fawad-laal/qalb-urdu/blob/main/scripts/generate_pdf_report.py): `scripts/generate_pdf_report.py`
- [Academic report](https://github.com/fawad-laal/qalb-urdu/blob/main/scripts/generate_academic_pdf.py): `scripts/generate_academic_pdf.py`
- [GPT Markdown report](https://github.com/fawad-laal/qalb-urdu/blob/main/scripts/generate_final_report.py): `scripts/generate_final_report.py`

### Other components

- **Evaluator**
- **OpenAI service**

## Connections

- Evaluator → Test runner: starts run
- Test runner → Urdu cases: loads cases
- Test runner → Roman Urdu cases: loads cases
- Test runner → Task categories: uses categories
- Test runner → Qalb model: sends prompts
- Test runner → Response scoring: scores responses
- Response scoring → Run results: records scores
- Round analysis → Run results: analyzes results
- Failure analysis → Run results: analyzes failures
- Round 2 analysis → Run results: analyzes results
- Data collector → Report data: aggregates and saves
- Report data → PDF report: loads report data
- Run results → GPT Markdown report: provides results
- GPT Markdown report → OpenAI service: requests analysis
- Report data → Academic report: supplies evaluation content