How abinash123hg/document-understanding-agent works
This local document-QA application turns uploaded files into document-scoped, cited answers, with a per-page choice between embedded text and handwriting recognition. The API coordinates ingestion and chat; processing covers extraction through chunking, then hybrid retrieval and verified local-model answering. Storage combines JSON chunks with a vector index. README documents API, isolation, refusal and verification behavior; detailed internal calls are shown only where the excerpts support them, so unsampled retrieval/indexing and UI links are omitted.
Main entry points: main.py HTTP API · processor.py Extract and chunk · retriever.py Hybrid retrieval · storage.py Chunk store
Components 12
API and interface
- Browser UI
frontend - HTTP API
backend/main.py - Settings
backend/config.py
Document ingestion
- Extract and chunk
backend/processor.py - Image preparation
backend/imaging.py - Handwriting recognition
backend/handwriting.py
Grounded Q&A
- Hybrid retrieval
backend/retriever.py - Answer and verification
backend/llm.py
Index and state
- Chunk store
backend/storage.py - Vector index
backend/retriever.py
Other components
- User
- Local Ollama
Connections 14
- User → Browser UI: uses
README.md - Browser UI → HTTP API: HTTP requests
README.md - HTTP API → Extract and chunk: processes upload
backend/main.py - Extract and chunk → Image preparation: prepares scanned pages
backend/processor.py - Extract and chunk → Handwriting recognition: recognizes handwriting
backend/processor.py - Image preparation → Handwriting recognition: supplies line crops
backend/handwriting.py - Extract and chunk → Chunk store: stores chunks
backend/processor.py - HTTP API → Hybrid retrieval: retrieves for chat
backend/main.py - Hybrid retrieval → Chunk store: loads document chunks
backend/retriever.py - Hybrid retrieval → Vector index: dense search
README.md - HTTP API → Answer and verification: generates and verifies
backend/main.py - Answer and verification → Local Ollama: requests model output
backend/llm.py - HTTP API → Chunk store: lists documents
backend/main.py - HTTP API → Settings: uses configuration
backend/main.py