How khiloni/multilingual-ocr-pipeline-architecture works
DocScribe accepts PDFs through its dashboard/API, queues OCR work, and returns searchable structured results and exports. The graph captures the documented page-processing stages and major capabilities; several pipeline internals and API routes are represented at subsystem level because their source was not excerpted. Edges are limited to relationships explicit in excerpts, the SOURCE INDEX, or README.
Main entry points: tasks.py OCR task orchestration · App.tsx Dashboard · main.py FastAPI application · client.ts API client
Components 34
Dashboard
- Dashboard
frontend/src/App.tsx - Upload form
frontend/src/components/UploadForm.tsx - API client
frontend/src/api/client.ts - Job status
frontend/src/components/JobStatus.tsx - Page preview
frontend/src/components/PagePreview.tsx - Result viewer
frontend/src/components/ResultViewer.tsx - Document list
frontend/src/components/DocumentList.tsx - Search box
frontend/src/components/SearchBox.tsx
API and persistence
- FastAPI application
backend/app/main.py - Job routes
backend/app/api/routes/jobs.py - Document routes
backend/app/api/routes/documents.py - Search routes
backend/app/api/routes/search.py - Webhook routes
backend/app/api/routes/webhooks.py - Task queue
backend/app/services/queue.py - PostgreSQL records
backend/app/models/models.py - Runtime configuration
backend/app/core/config.py
OCR processing
- OCR task orchestration
backend/worker/tasks.py - Rasterize and preprocess
backend/worker/pipeline/preprocess.py - Engine routing and retry
backend/worker/pipeline/router.py - PaddleOCR engine
backend/worker/pipeline/engines/paddle_engine.py - Surya fallback
backend/worker/pipeline/engines/surya_engine.py - Table and figure detection
backend/worker/pipeline/engines/structure_engine.py - Block post-processing
backend/worker/pipeline/postprocess.py - Schema normalization
backend/worker/pipeline/normalize.py - Text correction
backend/worker/pipeline/correction.py - Spell checking
backend/worker/pipeline/spellcheck.py - Page quality scoring
backend/worker/pipeline/quality.py
Results and search
- Object storage
backend/app/services/storage.py - PDF exports
backend/worker/pipeline/export_pdf.py
Other components
- Document user
- Search user
- MinIO / S3
- PostgreSQL
- Celery worker
backend/worker/celery_app.py
Connections 44
- Document user → Dashboard: uses
README.md - Dashboard → Upload form: renders
frontend/src/App.tsx - Dashboard → Job status: renders
frontend/src/App.tsx - Dashboard → Page preview: renders
frontend/src/App.tsx - Dashboard → Result viewer: renders
frontend/src/App.tsx - Dashboard → Document list: renders
frontend/src/App.tsx - Dashboard → Search box: renders
frontend/src/App.tsx - Upload form → API client: uploads through
frontend/src/components/UploadForm.tsx - Job status → API client: polls through
frontend/src/components/JobStatus.tsx - Result viewer → API client: downloads through
frontend/src/components/ResultViewer.tsx - Document list → API client: retrieves through
frontend/src/components/DocumentList.tsx - Search box → API client: searches through
frontend/src/components/SearchBox.tsx - API client → FastAPI application: HTTP requests
frontend/src/api/client.ts - FastAPI application → Job routes: routes to
backend/app/main.py - FastAPI application → Document routes: routes to
backend/app/main.py - FastAPI application → Search routes: routes to
backend/app/main.py - FastAPI application → Webhook routes: routes to
backend/app/main.py - Job routes → Task queue: enqueues
backend/app/api/routes/jobs.py - Job routes → PostgreSQL records: reads and writes
backend/app/api/routes/jobs.py - Document routes → PostgreSQL records: reads and writes
backend/app/api/routes/documents.py - Search routes → PostgreSQL records: searches
backend/app/api/routes/search.py - Webhook routes → PostgreSQL records: updates
backend/app/api/routes/webhooks.py - Job routes → Object storage: retrieves artifacts
backend/app/api/routes/jobs.py - Document routes → Object storage: manages files
backend/app/api/routes/documents.py - Task queue → Celery worker: dispatches task
backend/app/services/queue.py - Celery worker → OCR task orchestration: executes
backend/worker/tasks.py - OCR task orchestration → Rasterize and preprocess: prepares pages
backend/worker/tasks.py - OCR task orchestration → Engine routing and retry: routes pages
backend/worker/tasks.py - Engine routing and retry → PaddleOCR engine: runs primary OCR
backend/worker/pipeline/router.py - Engine routing and retry → Surya fallback: falls back to
backend/worker/pipeline/router.py - OCR task orchestration → Table and figure detection: detects layout
backend/worker/tasks.py - OCR task orchestration → Block post-processing: post-processes
backend/worker/tasks.py - OCR task orchestration → Schema normalization: normalizes
backend/worker/tasks.py - OCR task orchestration → Page quality scoring: scores pages
backend/worker/tasks.py - Schema normalization → Text correction: applies correction
backend/worker/pipeline/normalize.py - Schema normalization → Spell checking: uses spellcheck
backend/worker/pipeline/normalize.py - OCR task orchestration → PostgreSQL records: persists records
backend/worker/tasks.py - OCR task orchestration → Object storage: uploads artifacts
backend/worker/tasks.py - OCR task orchestration → PDF exports: generates PDF exports
README.md - Object storage → MinIO / S3: stores objects
README.md - PostgreSQL records → PostgreSQL: persists rows
README.md - Runtime configuration → Engine routing and retry: sets thresholds
backend/worker/pipeline/router.py - Search user → Dashboard: searches documents
README.md - FastAPI application → Runtime configuration: loads settings
backend/app/main.py