# genieincodebottle/parsemypdf architecture

> Architecture diagram of the GitHub repository genieincodebottle/parsemypdf, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/genieincodebottle/parsemypdf
- Repository: https://github.com/genieincodebottle/parsemypdf

## Overview

##### Main Components
- Input & File Management:
- Input PDFs are placed in the “input/” directory.
- There is an “images/” folder and a “converted_images/” folder used to manage images (possibly for OCR processing).
- Core Parsing Engine:
- The “parser/” directory holds multiple modules, each representing a different parsing strategy.
▪ API-based parsing methods (e.g., Amazon Textract, Claude via the “amazon-textract” and “claude” folders).
▪ Open weight / local methods (e.g., Camelot, pdfminer, PyPdf, docling, markitdown, etc.).
- Each parser submodule encapsulates code to handle specific PDF structures (tables, images, multi-column texts, etc.).
- Utility Layer:
- “utils/” provides common functionality such as converting PDFs to images and miscellaneous helper functions.
- OCR & Multimodal Models:
- The “vlm_ocr/” directory includes code to leverage local and API-driven OCR modules for handling multimodal data.
- Application Frontend:
- “pdf_parser_app.py” (and “vlm_ocr_app.py”) suggests a Streamlit-based UI for testing and running different parsing methods.
- Configuration & Environment:
- The “.env.example” file and requirements indicate how API keys and dependencies integrate into the overall system.

##### Relationships and Data Flow
- Orchestration layer (i.e., the main application scripts such as “pdf_parser_app.py”) that coordinates:
▪ Determining which parser to execute based on PDF characteristics.
▪ Routing the PDF data to one of the many parser modules.
- Detail the parser layer:
▪ Split the layer into API-based parsers (Amazon Textract, OpenAI, Anthropic, Gemini, etc.) and local parsers (Camelot, pdfminer, PyPdf, etc.).
▪ Indicate that these modules process the PDF (either in its native form or after conversion) and output results like CSV files placed in the “output/” directory.
- “vlm_ocr/” branch—a parallel module that uses multimodal models for OCR, operating similarly by taking in images/PDFs and outputting extracted text/data.

##### Architectural Patterns and Technologies
- Modular Design: Each parser is its own module that is loosely coupled yet orchestrated by central scripts.
- Separation of Concerns: Clear separation between input handling, pre-processing (e.g., image conversion), core parsing logic, and the UI/presentation layer.
- External Integration: There is a layer of integration with various external APIs and models (Amazon Textract, Anthropic, OpenAI, etc.) as well as local libraries (Camelot, pdfminer, etc.).
- Dependency Injection / Config Driven: Using .env for runtime configuration of API keys.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% File Management Layer
    subgraph "File Management"
        A["Input PDFs"]:::internal
        B["Images Folder"]:::internal
        C["Converted Images Folder"]:::internal
        D["Output Files"]:::internal
    end

    %% Utility Layer
    subgraph "Utilities"
        U["PDF to Image Converter"]:::internal
        V["Common Utils"]:::internal
    end

    %% Core Parsing Engine
    subgraph "Core Parsing Engine"
        subgraph "API Based Parsers"
            AT["Amazon Textract Parser"]:::external
            CL["Claude Parser"]:::external
        end
        subgraph "Local Parsing Methods"
            CA["Camelot Module"]:::internal
            DO["Docling Module"]:::internal
            GE["Gemini Module"]:::internal
            LP["Llama Parse Module"]:::internal
            LV["Llama Vision Module"]:::internal
            MK["Markitdown Module"]:::internal
            PM["Pdfminer Module"]:::internal
            PP["Pdfplumber Module"]:::internal
            MU["PyMupdf Module"]:::internal
            PD["PyPDF Module"]:::internal
            PDIR["PyPDF Directory Module"]:::internal
            PI["PyPDFium Module"]:::internal
            UIO["Unstructured-io Module"]:::internal
            MR["Marker Module"]:::internal
        end
    end

    %% OCR & Multimodal Layer
    subgraph "OCR & Multimodal"
        OCR["VLM OCR Module"]:::internal
    end

    %% Application Frontend Layer
    subgraph "Application Frontend"
        APP["PDF Parser App"]:::internal
        OAPP["VLM OCR App"]:::internal
    end

    %% Configuration & Environment Layer
    subgraph "Configuration & Dependencies"
        ENV[".env.example"]:::config
        REQ["requirements.txt"]:::config
    end

    %% Data Flow Connections
    A --> U
    B --> U
    C --> U
    A --> OCR

    V --> U

    %% From Utility to Core Parsing Engine
    U --> AT
    U --> CA

    %% Additional flow from Utility to API based parser
    U --> CL

    %% Core Parsing Engine outputs to Output Files (individual arrows)
    AT --> D
    CL --> D
    CA --> D
    DO --> D
    GE --> D
    LP --> D
    LV --> D
    MK --> D
    PM --> D
    PP --> D
    MU --> D
    PD --> D
    PDIR --> D
    PI --> D
    UIO --> D
    MR --> D

    %% OCR module output
    OCR --> D

    %% Output to Application Frontend
    D --> APP
    D --> OAPP

    %% Configuration flows to key modules
    ENV --> AT
    ENV --> CL
    ENV --> OCR
    REQ --> AT
    REQ --> CL
    REQ --> OCR
    ENV --> U
    REQ --> U

    %% Styling Classes
    classDef internal fill:#ADD8E6,stroke:#333,stroke-width:2px;
    classDef external fill:#98FB98,stroke:#333,stroke-width:2px;
    classDef config fill:#D3D3D3,stroke:#333,stroke-width:2px;

    %% Click Events
    click A "https://github.com/genieincodebottle/parsemypdf/tree/main/input/"
    click B "https://github.com/genieincodebottle/parsemypdf/tree/main/images/"
    click C "https://github.com/genieincodebottle/parsemypdf/tree/main/converted_images/"
    click D "https://github.com/genieincodebottle/parsemypdf/tree/main/output/"
    click U "https://github.com/genieincodebottle/parsemypdf/tree/main/utils/"
    click AT "https://github.com/genieincodebottle/parsemypdf/tree/main/parser/amazon-textract/"
    click CL "https://github.com/genieincodebottle/parsemypdf/tree/main/parser/claude/"
    click CA "https://github.com/genieincodebottle/parsemypdf/tree/main/parser/camelot/"
    click DO "https://github.com/genieincodebottle/parsemypdf/tree/main/parser/docling/"
    click GE "https://github.com/genieincodebottle/parsemypdf/tree/main/parser/gemini/"
    click LP "https://github.com/genieincodebottle/parsemypdf/tree/main/parser/llama-parse/"
    click LV "https://github.com/genieincodebottle/parsemypdf/tree/main/parser/llama-vision/"
    click MK "https://github.com/genieincodebottle/parsemypdf/tree/main/parser/markitdown/"
    click PM "https://github.com/genieincodebottle/parsemypdf/tree/main/parser/pdfminer/"
    click PP "https://github.com/genieincodebottle/parsemypdf/tree/main/parser/pdfplumber/"
    click MU "https://github.com/genieincodebottle/parsemypdf/tree/main/parser/pymupdf/"
    click PD "https://github.com/genieincodebottle/parsemypdf/tree/main/parser/pypdf/"
    click PDIR "https://github.com/genieincodebottle/parsemypdf/tree/main/parser/pypdfdirectory/"
    click PI "https://github.com/genieincodebottle/parsemypdf/tree/main/parser/pypdfium/"
    click UIO "https://github.com/genieincodebottle/parsemypdf/tree/main/parser/unstructured-io/"
    click MR "https://github.com/genieincodebottle/parsemypdf/tree/main/parser/marker/"
    click OCR "https://github.com/genieincodebottle/parsemypdf/tree/main/vlm_ocr/"
    click APP "https://github.com/genieincodebottle/parsemypdf/blob/main/pdf_parser_app.py"
    click OAPP "https://github.com/genieincodebottle/parsemypdf/blob/main/vlm_ocr_app.py"
    click ENV "https://github.com/genieincodebottle/parsemypdf/blob/main/.env.example"
    click REQ "https://github.com/genieincodebottle/parsemypdf/blob/main/requirements.txt"
```
