How genieincodebottle/parsemypdf works
Main Components
• Input & File Management:
• Input PDFs are placed in the “input/” directory.
• There is an “images/” folder and a “converted_images/” folder used to manage images (possibly for OCR processing).
• Core Parsing Engine:
• The “parser/” directory holds multiple modules, each representing a different parsing strategy.
▪ API-based parsing methods (e.g., Amazon Textract, Claude via the “amazon-textract” and “claude” folders).
▪ Open weight / local methods (e.g., Camelot, pdfminer, PyPdf, docling, markitdown, etc.).
• Each parser submodule encapsulates code to handle specific PDF structures (tables, images, multi-column texts, etc.).
• Utility Layer:
• “utils/” provides common functionality such as converting PDFs to images and miscellaneous helper functions.
• OCR & Multimodal Models:
• The “vlm_ocr/” directory includes code to leverage local and API-driven OCR modules for handling multimodal data.
• Application Frontend:
• “pdf_parser_app.py” (and “vlm_ocr_app.py”) suggests a Streamlit-based UI for testing and running different parsing methods.
• Configuration & Environment:
• The “.env.example” file and requirements indicate how API keys and dependencies integrate into the overall system.
Relationships and Data Flow
• Orchestration layer (i.e., the main application scripts such as “pdf_parser_app.py”) that coordinates:
▪ Determining which parser to execute based on PDF characteristics.
▪ Routing the PDF data to one of the many parser modules.
• Detail the parser layer:
▪ Split the layer into API-based parsers (Amazon Textract, OpenAI, Anthropic, Gemini, etc.) and local parsers (Camelot, pdfminer, PyPdf, etc.).
▪ Indicate that these modules process the PDF (either in its native form or after conversion) and output results like CSV files placed in the “output/” directory.
• “vlm_ocr/” branch—a parallel module that uses multimodal models for OCR, operating similarly by taking in images/PDFs and outputting extracted text/data.
Architectural Patterns and Technologies
• Modular Design: Each parser is its own module that is loosely coupled yet orchestrated by central scripts.
• Separation of Concerns: Clear separation between input handling, pre-processing (e.g., image conversion), core parsing logic, and the UI/presentation layer.
• External Integration: There is a layer of integration with various external APIs and models (Amazon Textract, Anthropic, OpenAI, etc.) as well as local libraries (Camelot, pdfminer, etc.).
• Dependency Injection / Config Driven: Using .env for runtime configuration of API keys.