How shivamgoyal03/privydoc works
Project Type & Purpose
• This is an open-source, on-device document analysis tool (“PrivyDoc”) that processes PDF and DOCX files locally using Microsoft Foundry Local. It provides both a Chainlit-based web UI and a CLI interface, orchestrates a multi-agent AI workflow, and stores results in a local JSON file for audit and export.
Main Architectural Components
a. User Interfaces
• Chainlit Web UI (HTTP server on localhost:8000)
• Command-Line Interface (CLI entrypoint)
b. Document Processor
• pdfplumber for PDF extraction
• python-docx for DOCX extraction
• Normalization & text cleaning module
c. Agent Orchestration / AI Engine
• Agent Framework coordinating multiple agents
• Foundry Local SDK executing models (qwen2.5, phi-3.5, phi-4)
• Sub-agents:
– Section Extraction Agent
– Entity Recognition Agent
– Summarization & Sentiment Agent
d. Storage Layer
• Local JSON database (analysis_history.json)
• Document fingerprinting & processing logs
e. Export Module
• Markdown, JSON, CSV serializers
f. External Dependency
• Microsoft Foundry Local runtime & models
Component Interactions & Data Flow
• User uploads file via Web UI or invokes CLI → Document Processor extracts raw text → Passes to Agent Orchestration →
1) Section Extraction → 2) Entity Recognition → 3) Summarization & Sentiment →
Results aggregated → Written to analysis_history.json → Web UI retrieves and displays results, offering export.
• In CLI mode, skip Web UI retrieval: Agent results go directly to JSON.
Architectural Patterns & Principles
• Pipeline pattern: sequential multi-stage processing.
• Modular agents: separation of concerns (extraction, NER, summarization).
• Local-first, zero-trust: all data stays on disk; no external calls.
• Extensibility: add new agents or export formats by plugging into the framework.
Technologies & Frameworks
• Python 3.10+
• pdfplumber, python-docx
• Microsoft Foundry Local SDK (foundrylocal)
• Chainlit for UI
• Custom Agent Framework
• JSON file for storage
Diagramming Recommendations
• Rounded rectangles for UI components (blue)
• Rectangles for processing modules/agents (green)
User → Processor → Agents → Storage → UI/Export
• Technologies next to each component.
• Agent stages in a container labeled “AI Engine / Agent Framework.”