# khiloni/multilingual-ocr-pipeline-architecture architecture

> Architecture diagram of the GitHub repository khiloni/multilingual-ocr-pipeline-architecture, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/khiloni/multilingual-ocr-pipeline-architecture
- Repository: https://github.com/khiloni/multilingual-ocr-pipeline-architecture

## Overview

DocScribe accepts PDFs through its dashboard/API, queues OCR work, and returns searchable structured results and exports. The graph captures the documented page-processing stages and major capabilities; several pipeline internals and API routes are represented at subsystem level because their source was not excerpted. Edges are limited to relationships explicit in excerpts, the SOURCE INDEX, or README.

Main entry points: [`backend/worker/tasks.py`](https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/worker/tasks.py) (OCR task orchestration), [`frontend/src/App.tsx`](https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/frontend/src/App.tsx) (Dashboard), [`backend/app/main.py`](https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/app/main.py) (FastAPI application), [`frontend/src/api/client.ts`](https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/frontend/src/api/client.ts) (API client)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ui["Dashboard"]
  node_dashboard["Dashboard<br/>[App.tsx]"]
  node_upload["Upload form<br/>[UploadForm.tsx]"]
  node_api_client["API client<br/>[client.ts]"]
  node_status["Job status<br/>[JobStatus.tsx]"]
  node_preview["Page preview<br/>[PagePreview.tsx]"]
  node_viewer["Result viewer<br/>[ResultViewer.tsx]"]
  node_documents_ui["Document list<br/>[DocumentList.tsx]"]
  node_search_ui["Search box<br/>[SearchBox.tsx]"]
end

subgraph group_api["API and persistence"]
  node_api["FastAPI application<br/>[main.py]"]
  node_job_routes["Job routes<br/>[jobs.py]"]
  node_document_routes["Document routes<br/>[documents.py]"]
  node_search_routes["Search routes<br/>[search.py]"]
  node_webhook_routes["Webhook routes<br/>[webhooks.py]"]
  node_queue["Task queue<br/>[queue.py]"]
  node_database[("PostgreSQL records<br/>[models.py]")]
  node_config["Runtime configuration<br/>[config.py]"]
end

subgraph group_worker["OCR processing"]
  node_worker_task["OCR task orchestration<br/>[tasks.py]"]
  node_preprocess["Rasterize and preprocess<br/>[preprocess.py]"]
  node_router["Engine routing and retry<br/>[router.py]"]
  node_paddle["PaddleOCR engine<br/>[paddle_engine.py]"]
  node_surya["Surya fallback<br/>[surya_engine.py]"]
  node_structure["Table and figure detection"]
  node_postprocess["Block post-processing<br/>[postprocess.py]"]
  node_normalize["Schema normalization<br/>[normalize.py]"]
  node_correction["Text correction<br/>[correction.py]"]
  node_spellcheck["Spell checking<br/>[spellcheck.py]"]
  node_quality["Page quality scoring<br/>[quality.py]"]
end

subgraph group_results["Results and search"]
  node_storage[("Object storage<br/>[storage.py]")]
  node_exports["PDF exports<br/>[export_pdf.py]"]
end

node_user(("Document user"))
node_actor_search(("Search user"))
node_minio[("MinIO / S3")]
node_postgres[("PostgreSQL")]
node_celery["Celery worker<br/>[celery_app.py]"]

node_user -->|"uses"| node_dashboard
node_dashboard -->|"renders"| node_upload
node_dashboard -->|"renders"| node_status
node_dashboard -->|"renders"| node_preview
node_dashboard -->|"renders"| node_viewer
node_dashboard -->|"renders"| node_documents_ui
node_dashboard -->|"renders"| node_search_ui
node_upload -->|"uploads through"| node_api_client
node_status -->|"polls through"| node_api_client
node_viewer -->|"downloads through"| node_api_client
node_documents_ui -->|"retrieves through"| node_api_client
node_search_ui -->|"searches through"| node_api_client
node_api_client -->|"HTTP requests"| node_api
node_api -->|"routes to"| node_job_routes
node_api -->|"routes to"| node_document_routes
node_api -->|"routes to"| node_search_routes
node_api -->|"routes to"| node_webhook_routes
node_job_routes -->|"enqueues"| node_queue
node_job_routes -->|"reads and writes"| node_database
node_document_routes -->|"reads and writes"| node_database
node_search_routes -->|"searches"| node_database
node_webhook_routes -->|"updates"| node_database
node_job_routes -->|"retrieves artifacts"| node_storage
node_document_routes -->|"manages files"| node_storage
node_queue -->|"dispatches task"| node_celery
node_celery -->|"executes"| node_worker_task
node_worker_task -->|"prepares pages"| node_preprocess
node_worker_task -->|"routes pages"| node_router
node_router -->|"runs primary OCR"| node_paddle
node_router -->|"falls back to"| node_surya
node_worker_task -->|"detects layout"| node_structure
node_worker_task -->|"post-processes"| node_postprocess
node_worker_task -->|"normalizes"| node_normalize
node_worker_task -->|"scores pages"| node_quality
node_normalize -->|"applies correction"| node_correction
node_normalize -->|"uses spellcheck"| node_spellcheck
node_worker_task -->|"persists records"| node_database
node_worker_task -->|"uploads artifacts"| node_storage
node_worker_task -.->|"generates PDF exports"| node_exports
node_storage -.->|"stores objects"| node_minio
node_database -.->|"persists rows"| node_postgres
node_config -.->|"sets thresholds"| node_router
node_actor_search -->|"searches documents"| node_dashboard
node_api -->|"loads settings"| node_config

click node_dashboard "https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/frontend/src/App.tsx"
click node_upload "https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/frontend/src/components/UploadForm.tsx"
click node_api_client "https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/frontend/src/api/client.ts"
click node_status "https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/frontend/src/components/JobStatus.tsx"
click node_preview "https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/frontend/src/components/PagePreview.tsx"
click node_viewer "https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/frontend/src/components/ResultViewer.tsx"
click node_documents_ui "https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/frontend/src/components/DocumentList.tsx"
click node_search_ui "https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/frontend/src/components/SearchBox.tsx"
click node_api "https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/app/main.py"
click node_job_routes "https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/app/api/routes/jobs.py"
click node_document_routes "https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/app/api/routes/documents.py"
click node_search_routes "https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/app/api/routes/search.py"
click node_webhook_routes "https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/app/api/routes/webhooks.py"
click node_queue "https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/app/services/queue.py"
click node_database "https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/app/models/models.py"
click node_storage "https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/app/services/storage.py"
click node_worker_task "https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/worker/tasks.py"
click node_preprocess "https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/worker/pipeline/preprocess.py"
click node_router "https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/worker/pipeline/router.py"
click node_paddle "https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/worker/pipeline/engines/paddle_engine.py"
click node_surya "https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/worker/pipeline/engines/surya_engine.py"
click node_structure "https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/worker/pipeline/engines/structure_engine.py"
click node_postprocess "https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/worker/pipeline/postprocess.py"
click node_normalize "https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/worker/pipeline/normalize.py"
click node_correction "https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/worker/pipeline/correction.py"
click node_spellcheck "https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/worker/pipeline/spellcheck.py"
click node_quality "https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/worker/pipeline/quality.py"
click node_exports "https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/worker/pipeline/export_pdf.py"
click node_config "https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/app/core/config.py"
click node_celery "https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/worker/celery_app.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_dashboard,node_upload,node_api_client,node_status,node_preview,node_viewer,node_documents_ui,node_search_ui,node_user,node_actor_search toneBlue
class node_api,node_job_routes,node_document_routes,node_search_routes,node_webhook_routes,node_queue,node_database,node_config,node_minio,node_postgres toneAmber
class node_worker_task,node_preprocess,node_router,node_paddle,node_surya,node_structure,node_postprocess,node_normalize,node_correction,node_spellcheck,node_quality toneMint
class node_storage,node_exports,node_celery toneRose
```

## Components

### Dashboard

- [Dashboard](https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/frontend/src/App.tsx): `frontend/src/App.tsx`
- [Upload form](https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/frontend/src/components/UploadForm.tsx): `frontend/src/components/UploadForm.tsx`
- [API client](https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/frontend/src/api/client.ts): `frontend/src/api/client.ts`
- [Job status](https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/frontend/src/components/JobStatus.tsx): `frontend/src/components/JobStatus.tsx`
- [Page preview](https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/frontend/src/components/PagePreview.tsx): `frontend/src/components/PagePreview.tsx`
- [Result viewer](https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/frontend/src/components/ResultViewer.tsx): `frontend/src/components/ResultViewer.tsx`
- [Document list](https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/frontend/src/components/DocumentList.tsx): `frontend/src/components/DocumentList.tsx`
- [Search box](https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/frontend/src/components/SearchBox.tsx): `frontend/src/components/SearchBox.tsx`

### API and persistence

- [FastAPI application](https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/app/main.py): `backend/app/main.py`
- [Job routes](https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/app/api/routes/jobs.py): `backend/app/api/routes/jobs.py`
- [Document routes](https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/app/api/routes/documents.py): `backend/app/api/routes/documents.py`
- [Search routes](https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/app/api/routes/search.py): `backend/app/api/routes/search.py`
- [Webhook routes](https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/app/api/routes/webhooks.py): `backend/app/api/routes/webhooks.py`
- [Task queue](https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/app/services/queue.py): `backend/app/services/queue.py`
- [PostgreSQL records](https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/app/models/models.py): `backend/app/models/models.py`
- [Runtime configuration](https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/app/core/config.py): `backend/app/core/config.py`

### OCR processing

- [OCR task orchestration](https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/worker/tasks.py): `backend/worker/tasks.py`
- [Rasterize and preprocess](https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/worker/pipeline/preprocess.py): `backend/worker/pipeline/preprocess.py`
- [Engine routing and retry](https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/worker/pipeline/router.py): `backend/worker/pipeline/router.py`
- [PaddleOCR engine](https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/worker/pipeline/engines/paddle_engine.py): `backend/worker/pipeline/engines/paddle_engine.py`
- [Surya fallback](https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/worker/pipeline/engines/surya_engine.py): `backend/worker/pipeline/engines/surya_engine.py`
- [Table and figure detection](https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/worker/pipeline/engines/structure_engine.py): `backend/worker/pipeline/engines/structure_engine.py`
- [Block post-processing](https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/worker/pipeline/postprocess.py): `backend/worker/pipeline/postprocess.py`
- [Schema normalization](https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/worker/pipeline/normalize.py): `backend/worker/pipeline/normalize.py`
- [Text correction](https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/worker/pipeline/correction.py): `backend/worker/pipeline/correction.py`
- [Spell checking](https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/worker/pipeline/spellcheck.py): `backend/worker/pipeline/spellcheck.py`
- [Page quality scoring](https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/worker/pipeline/quality.py): `backend/worker/pipeline/quality.py`

### Results and search

- [Object storage](https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/app/services/storage.py): `backend/app/services/storage.py`
- [PDF exports](https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/worker/pipeline/export_pdf.py): `backend/worker/pipeline/export_pdf.py`

### Other components

- **Document user**
- **Search user**
- **MinIO / S3**
- **PostgreSQL**
- [Celery worker](https://github.com/khiloni/multilingual-ocr-pipeline-architecture/blob/main/backend/worker/celery_app.py): `backend/worker/celery_app.py`

## Connections

- Document user → Dashboard: uses (evidence: `README.md`)
- Dashboard → Upload form: renders (evidence: `frontend/src/App.tsx`)
- Dashboard → Job status: renders (evidence: `frontend/src/App.tsx`)
- Dashboard → Page preview: renders (evidence: `frontend/src/App.tsx`)
- Dashboard → Result viewer: renders (evidence: `frontend/src/App.tsx`)
- Dashboard → Document list: renders (evidence: `frontend/src/App.tsx`)
- Dashboard → Search box: renders (evidence: `frontend/src/App.tsx`)
- Upload form → API client: uploads through (evidence: `frontend/src/components/UploadForm.tsx`)
- Job status → API client: polls through (evidence: `frontend/src/components/JobStatus.tsx`)
- Result viewer → API client: downloads through (evidence: `frontend/src/components/ResultViewer.tsx`)
- Document list → API client: retrieves through (evidence: `frontend/src/components/DocumentList.tsx`)
- Search box → API client: searches through (evidence: `frontend/src/components/SearchBox.tsx`)
- API client → FastAPI application: HTTP requests (evidence: `frontend/src/api/client.ts`)
- FastAPI application → Job routes: routes to (evidence: `backend/app/main.py`)
- FastAPI application → Document routes: routes to (evidence: `backend/app/main.py`)
- FastAPI application → Search routes: routes to (evidence: `backend/app/main.py`)
- FastAPI application → Webhook routes: routes to (evidence: `backend/app/main.py`)
- Job routes → Task queue: enqueues (evidence: `backend/app/api/routes/jobs.py`)
- Job routes → PostgreSQL records: reads and writes (evidence: `backend/app/api/routes/jobs.py`)
- Document routes → PostgreSQL records: reads and writes (evidence: `backend/app/api/routes/documents.py`)
- Search routes → PostgreSQL records: searches (evidence: `backend/app/api/routes/search.py`)
- Webhook routes → PostgreSQL records: updates (evidence: `backend/app/api/routes/webhooks.py`)
- Job routes → Object storage: retrieves artifacts (evidence: `backend/app/api/routes/jobs.py`)
- Document routes → Object storage: manages files (evidence: `backend/app/api/routes/documents.py`)
- Task queue → Celery worker: dispatches task (evidence: `backend/app/services/queue.py`)
- Celery worker → OCR task orchestration: executes (evidence: `backend/worker/tasks.py`)
- OCR task orchestration → Rasterize and preprocess: prepares pages (evidence: `backend/worker/tasks.py`)
- OCR task orchestration → Engine routing and retry: routes pages (evidence: `backend/worker/tasks.py`)
- Engine routing and retry → PaddleOCR engine: runs primary OCR (evidence: `backend/worker/pipeline/router.py`)
- Engine routing and retry → Surya fallback: falls back to (evidence: `backend/worker/pipeline/router.py`)
- OCR task orchestration → Table and figure detection: detects layout (evidence: `backend/worker/tasks.py`)
- OCR task orchestration → Block post-processing: post-processes (evidence: `backend/worker/tasks.py`)
- OCR task orchestration → Schema normalization: normalizes (evidence: `backend/worker/tasks.py`)
- OCR task orchestration → Page quality scoring: scores pages (evidence: `backend/worker/tasks.py`)
- Schema normalization → Text correction: applies correction (evidence: `backend/worker/pipeline/normalize.py`)
- Schema normalization → Spell checking: uses spellcheck (evidence: `backend/worker/pipeline/normalize.py`)
- OCR task orchestration → PostgreSQL records: persists records (evidence: `backend/worker/tasks.py`)
- OCR task orchestration → Object storage: uploads artifacts (evidence: `backend/worker/tasks.py`)
- OCR task orchestration → PDF exports: generates PDF exports (evidence: `README.md`)
- Object storage → MinIO / S3: stores objects (evidence: `README.md`)
- PostgreSQL records → PostgreSQL: persists rows (evidence: `README.md`)
- Runtime configuration → Engine routing and retry: sets thresholds (evidence: `backend/worker/pipeline/router.py`)
- Search user → Dashboard: searches documents (evidence: `README.md`)
- FastAPI application → Runtime configuration: loads settings (evidence: `backend/app/main.py`)
