# iamarunbrahma/pdf-to-markdown architecture

> Architecture diagram of the GitHub repository iamarunbrahma/pdf-to-markdown, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/iamarunbrahma/pdf-to-markdown
- Repository: https://github.com/iamarunbrahma/pdf-to-markdown

## Overview

##### Project Overview and Main Components
- This project is a utility tool designed to convert PDF files into markdown-formatted output, ensuring preservation of formatting elements (e.g., tables, images, bold/italic text, code blocks) while also employing OCR for images.
- Following key components:
- Input Module: Handles receiving or specifying the PDF file (via command-line arguments).
- Configuration Manager: Reads settings from the config file (config/config.yaml) to configure extraction parameters.
- Extraction Engine: The central module in “extract.py” that orchestrates the extraction process.
- PDF Processing Submodules: Uses libraries like PyMuPDF (fitz) and pdfplumber to parse and extract text and layout.
- OCR and Image Processing Submodule: Utilizes pytesseract and OpenCV (cv2) for processing images and performing OCR; integrates with transformer-based models for image caption generation.
- Markdown Conversion Module: Takes the parsed data and converts it into well-structured Markdown output.
- Output Writer: Saves the resulting .md file into the intended output directory.
- Dependency Layers: Integration with external libraries such as numpy, Pillow, transformers, and torch.

##### Key Relationships and Interactions
- The Input Module feeds the PDF file to the Extraction Engine.
- The Extraction Engine interacts with:
- The Configuration Manager for settings.
- The PDF Parsing and OCR submodules in parallel.
- Data flows from the parsing modules (text extraction and OCR) to the Markdown Conversion module.
- The Markdown Conversion module then pushes the final formatted markdown to the Output Writer.

##### Architectural Patterns and Design Principles
- Separation of concerns: Different modules handle specific tasks (e.g., PDF parsing vs. OCR).
- Modular design: Each processing step (parsing, OCR, conversion) is encapsulated, making future extensions or debugging easier.
- Dependency injection: The configuration management component provides parameters across modules.
- Adaptability: The design supports handling both native text PDFs and scanned images.

##### Relevant Technologies and Libraries
- Python is the primary language.
- PyMuPDF (fitz) and pdfplumber for reading PDF content.
- pytesseract and OpenCV (cv2) for OCR and image processing.
- Additional libraries like numpy, Pillow, transformers, and torch assist in handling complex processing like OCR enhancements and image captioning.
- Configuration management via YAML (config.yaml).

##### Specific Elements
- Clear labels for each module/component.
- PDF Input Module
- Configuration Manager (config/config.yaml)
- Extraction Engine split into:
– PDF Parsing (PyMuPDF, pdfplumber)
– OCR & Image Processing (pytesseract, OpenCV, image caption generation)
- Markdown Conversion Module
- Output Writer (producing the .md file)

## Diagram (Mermaid)

```mermaid
flowchart TD
    A["PDF Input Module"]:::fileManagement
    B["Configuration Manager"]:::config
    C["Extraction Engine"]:::extraction
    D["PDF Parsing Submodule (PyMuPDF & pdfplumber)"]:::extractionSub
    E["OCR & Image Processing Submodule (pytesseract, OpenCV, Captioning)"]:::extractionSub
    F["Markdown Conversion Module"]:::processing
    G["Output Writer"]:::fileManagement
    H["Dependency Layers"]:::dependency

    A -->|"feeds"| C
    B -->|"configures"| C
    C -->|"orchestrates"| D
    C -->|"orchestrates"| E
    D -->|"outputsTo"| F
    E -->|"outputsTo"| F
    F -->|"writesTo"| G
    H ---|"providesDeps"| D
    H ---|"providesDeps"| E

    click A "https://github.com/iamarunbrahma/pdf-to-markdown/blob/main/extract.py"
    click B "https://github.com/iamarunbrahma/pdf-to-markdown/blob/main/config/config.yaml"
    click C "https://github.com/iamarunbrahma/pdf-to-markdown/blob/main/extract.py"
    click D "https://github.com/iamarunbrahma/pdf-to-markdown/blob/main/extract.py"
    click E "https://github.com/iamarunbrahma/pdf-to-markdown/blob/main/extract.py"
    click F "https://github.com/iamarunbrahma/pdf-to-markdown/blob/main/extract.py"
    click G "https://github.com/iamarunbrahma/pdf-to-markdown/blob/main/extract.py"
    click H "https://github.com/iamarunbrahma/pdf-to-markdown/blob/main/requirements.txt"

    classDef fileManagement fill:#f9e79f,stroke:#7d6608,stroke-width:2px;
    classDef config fill:#aed6f1,stroke:#1b4f72,stroke-width:2px;
    classDef extraction fill:#a9dfbf,stroke:#196f3d,stroke-width:2px;
    classDef extractionSub fill:#d5f5e3,stroke:#145a32,stroke-width:2px;
    classDef processing fill:#f5b7b1,stroke:#922b21,stroke-width:2px;
    classDef dependency fill:#d7dbdd,stroke:#566573,stroke-width:2px,stroke-dasharray: 5 5;
```
