How adityasingh0912/pdf-document-analyzer works
Project Type & Purpose
• This is a modular, full-stack-ish Python application with a Streamlit front end and a series of back-end processing modules (PDF extraction, math detection, NER, summarization, optional LLM synthesis).
• It’s containerized via Docker, designed for scientific/technical PDF analysis.
High-Level Architecture Components
a. Frontend (Streamlit UI)
b. Orchestrator (analyzer.py)
c. Processing Modules:
• document_processor.py (PyMuPDF text extraction)
• math_detector.py (regex-based math detection)
• ner_processor.py (spaCy + Transformers NER)
• summary_generator.py (BERT extractive summarizer + optional Llama-3 Groq API)
d. External Services & Libraries:
• Groq LPU™ Inference Engine (LLM synthesis)
• Local ML frameworks (PyTorch, spaCy models)
• PyMuPDF
e. Deployment: Docker container
Data Flow & Interactions
• Streamlit UI → invokes analyzer.py → in sequence:
1. Text extraction → 2. Math detection → 3. NER → 4. Extractive summarization → (optional) 5. Abstractive LLM synthesis
• Each module returns results to the orchestrator, which passes aggregated data back to the UI for rendering (with highlights, summaries).
• If LLM synthesis is enabled, analyzer.py calls Groq SDK over HTTPS, passing extracted text+entities and receiving enhanced report.
• @startuml
• title AI Document Analyzer Architecture
• skinparam rectangle {
• BackgroundColor<<Module>> LightBlue
• BackgroundColor<<Service>> LightGreen
• BackgroundColor<<External>> LightOrange
• }
• rectangle "Streamlit UI\n(app.py)" <<Module>> as UI
• rectangle "Analyzer\n(analyzer.py)" <<Module>> as Orchestrator
• rectangle "PDF Extraction\n(document_processor.py)" <<Module>> as DocProc
• rectangle "Math Detector\n(math_detector.py)" <<Module>> as MathDet
• rectangle "NER Processor\n(ner_processor.py)" <<Module>> as NER
• rectangle "Summarizer\n(summary_generator.py)" <<Module>> as SumGen
• rectangle "Groq Inference\n(Groq SDK)" <<External>> as Groq
• UI --> Orchestrator : user request (PDF)
• Orchestrator --> DocProc : extract text
• Orchestrator --> MathDet : find math
• Orchestrator --> NER : detect entities
• Orchestrator --> SumGen : generate summary
• SumGen --> Groq : optional LLM synthesis
• Groq --> SumGen : enhanced report
• Orchestrator --> UI : JSON payload (text, math, entities, summaries)
• @enduml