# diegomarzaa/pdf-ocr-obsidian architecture

> Architecture diagram of the GitHub repository diegomarzaa/pdf-ocr-obsidian, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/diegomarzaa/pdf-ocr-obsidian
- Repository: https://github.com/diegomarzaa/pdf-ocr-obsidian

## Overview

##### Project Type and Overall Purpose
- This is a PDF-to-Markdown conversion pipeline that automates OCR processing on PDFs using the Mistral AI OCR API.
- It offers multiple interfaces: a hosted/local web application (using app.py along with static files and HTML templates) and a Jupyter Notebook mode for step-by-step processing.
- The project is aimed at converting PDF documents into structured markdown files with extracted images, organized following Obsidian-compatible styling.

##### File Structure and Architectural Clues
- Top-level Files:
- app.py serves as the entry point for the web application.
- pdf-markdown-ocr.ipynb supports the notebook mode.
- requirements.txt, .env.copy, and other configuration files indicate dependency management and environment configuration.
- Directories:
- "templates" and "static" indicate the presence of web interface components (possibly using Flask or a similar framework).
- "doc" contains usage documentation and media (like usage gifs) to illustrate how the system works.
- "venv" suggests the use of a virtual environment.
- Additional folders mentioned in the README (such as pdfs_to_process, ocr_output, and pdfs-done) indicate the workflow’s file handling and output storage components, even if they are not explicitly in the file tree.

##### Main Components
a. User Interfaces:
- Web Frontend: Represented by the static files (HTML/CSS/JS) in the "templates" and "static" directories.
- Jupyter Notebook Interface: Represented by pdf-markdown-ocr.ipynb for interactive use.
b. Web Application Backend:
- The app.py file serves as the backend that manages API requests from the web UI, triggers the OCR process, and handles file uploads/downloads.
c. OCR Pipeline Module:
- Embodied by the logic that reads from a source folder (e.g., pdfs_to_process), calls the Mistral AI OCR API, and processes the returned data (extracted text and images).
- Includes components for saving markdown (output.md), caching raw OCR responses (ocr_response.json), and organizing images in designated subdirectories.
d. External Service:
- Mistral AI OCR API: An external service invoked by the system for text recognition.
e. File Management:
- Input and Output Directories: pdfs_to_process for incoming PDFs, ocr_output for storing each PDF’s processed markdown and images, and pdfs-done for archiving processed PDFs.
f. Configuration & Environment Management:
- Use of environment variable files (.env) for storing API keys and other configuration parameters.

##### Data Flow and Interactions
- Web App Flow (Local and Hosted Web App):
- The user uploads a PDF via the web interface.
- app.py receives the PDF, then:
-> Calls the OCR processing routine that sends the document to the Mistral AI API.
-> Receives and caches the OCR response as JSON.
-> Extracts text and images, then generates a structured markdown file (output.md) and organizes images.
- The output is made available for preview/download.
- Notebook/CLI Flow:
- The notebook interacts with the same OCR processing logic.
- It scans the pdfs_to_process folder, processes each PDF via the OCR API, and writes the output to ocr_output, moving processed PDFs to pdfs-done.

##### Architectural Patterns and Key Tech
- The separation of concerns: Frontend (static assets and HTML templates) versus Backend (app.py and processing logic).
- Interaction with external APIs for OCR.
- Use of Python scripting for both web service and notebook based operation.

## Diagram (Mermaid)

```mermaid
flowchart TD
    User("User"):::frontend

    subgraph "User Interfaces"
        subgraph "Web Frontend"
            Static("Static Assets"):::frontend
            Templates("HTML Templates"):::frontend
        end
        Notebook("Jupyter Notebook (pdf-markdown-ocr.ipynb)"):::frontend
    end

    subgraph "Backend & Business Logic"
        Backend("Backend (app.py)"):::backend
        OCR("OCR Processor"):::backend
    end

    subgraph "External Services"
        OCRAPI("Mistral AI OCR API"):::external
    end

    subgraph "File Management"
        Input("Input: pdfs_to_process"):::storage
        Output("Output: ocr_output"):::storage
        Archive("Archive: pdfs-done"):::storage
    end

    Config(".env.copy"):::config
    Doc("Documentation"):::config

    %% Connections
    User -->|"uploads"| Static
    User -->|"uploads"| Templates
    User -->|"executes"| Notebook

    Templates -->|"submitsPDF"| Backend
    Notebook -->|"triggersOCR"| OCR
    Backend -->|"initiatesOCR"| OCR
    OCR -->|"requests"| OCRAPI
    OCRAPI -->|"response"| OCR
    OCR -->|"writesMarkdownAndImages"| Output
    OCR -->|"movesPDF"| Archive
    Input -->|"monitoredBy"| OCR
    Config -->|"configures"| Backend
    Config -->|"configures"| OCR
    Doc -->|"reference"| User

    %% Click Events
    click Static "https://github.com/diegomarzaa/pdf-ocr-obsidian/tree/main/static"
    click Templates "https://github.com/diegomarzaa/pdf-ocr-obsidian/blob/main/templates/index.html"
    click Notebook "https://github.com/diegomarzaa/pdf-ocr-obsidian/blob/main/pdf-markdown-ocr.ipynb"
    click Backend "https://github.com/diegomarzaa/pdf-ocr-obsidian/blob/main/app.py"
    click Config "https://github.com/diegomarzaa/pdf-ocr-obsidian/blob/main/.env.copy"
    click Doc "https://github.com/diegomarzaa/pdf-ocr-obsidian/tree/main/doc"

    %% Styles
    classDef frontend fill:#cce5ff,stroke:#004085,stroke-width:2px;
    classDef backend fill:#d4edda,stroke:#155724,stroke-width:2px;
    classDef external fill:#ffeeba,stroke:#856404,stroke-width:2px;
    classDef storage fill:#e2e3e5,stroke:#6c757d,stroke-width:2px;
    classDef config fill:#f8d7da,stroke:#721c24,stroke-width:2px;
```
