# catchthetornado/pdf-extract-api architecture

> Architecture diagram of the GitHub repository catchthetornado/pdf-extract-api, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/catchthetornado/pdf-extract-api
- Repository: https://github.com/catchthetornado/pdf-extract-api

## Overview

##### Main Components

a) Client Layer:
- CLI Client (in `/client` directory)
- HTTP API Endpoints
- Supported file formats (PDF, images)

b) Application Layer:
- FastAPI Application (main.py)
- Celery Task Queue System (tasks.py)
- OCR Processing Pipeline
- LLM Processing Pipeline

c) Storage Layer:
- Multiple Storage Strategies (shown in /storage_strategies)
- Redis Cache
- Storage Profiles System

d) Processing Components:
- OCR Strategies (shown in /ocr_strategies)
- Ollama LLM Integration
- Task Processing Workers

##### Component Relationships

**[Client Layer] → [API Layer] → [Task Queue] → [Workers] → [Storage]**
- ↕            ↕              ↕           ↕
- [Redis Cache] [OCR Strategies] [LLM] [Storage Strategies]

##### Specific Implementation Details

OCR Strategies:
- Marker
- LLama Vision
- Tesseract

Storage Strategies:
- Local Filesystem
- Google Drive
- AWS S3

##### Key Technical Elements

Components:
- Redis (for caching and Celery backend)
- Ollama (LLM processing)
- FastAPI (REST API)
- Celery (task processing)

##### Configuration Elements

Configuration sources:
- Environment variables
- Storage profiles
- Strategy selection

- The modular nature of the system (strategy pattern usage)
- Asynchronous processing capabilities
- Multiple storage options
- Caching mechanism

##### Deployment Context

Containers:
- FastAPI App container
- Celery Worker container
- Redis container
- Ollama container

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph Client
        CLI["CLI Client"]:::client
        API["HTTP API Endpoints"]:::client
        FileFormats["Supported Formats<br/>(PDF, Images)"]:::client
    end

    subgraph Application
        FastAPI["FastAPI Application"]:::app
        CeleryTasks["Celery Task Queue"]:::queue
        TaskStatus["Task Status Tracking"]:::app
    end

    subgraph Processing
        OCRPipeline["OCR Processing Pipeline"]:::processing
        LLMPipeline["LLM Processing Pipeline"]:::processing
        
        subgraph "OCR Strategies"
            Marker["Marker OCR"]:::strategy
            LlamaVision["Llama Vision OCR"]:::strategy
            Tesseract["Tesseract OCR"]:::strategy
        end
        
        subgraph "LLM Integration"
            Ollama["Ollama LLM"]:::llm
        end
    end

    subgraph Storage
        StorageManager["Storage Manager"]:::storage
        
        subgraph "Storage Strategies"
            LocalFS["Local Filesystem"]:::strategy
            GDrive["Google Drive"]:::strategy
            S3["AWS S3"]:::strategy
        end
        
        StorageProfiles["Storage Profiles"]:::config
    end

    subgraph Cache
        Redis["Redis Cache"]:::cache
    end

    %% Relationships
    CLI --> FastAPI
    API --> FastAPI
    FastAPI --> CeleryTasks
    CeleryTasks --> OCRPipeline
    OCRPipeline --> |Results| LLMPipeline
    OCRPipeline --> StorageManager
    LLMPipeline --> StorageManager
    
    OCRPipeline --> Marker & LlamaVision & Tesseract
    LLMPipeline --> Ollama
    
    StorageManager --> LocalFS & GDrive & S3
    StorageManager --> StorageProfiles
    
    FastAPI <--> Redis
    CeleryTasks <--> Redis

    %% Click Events
    click CLI "https://github.com/CatchTheTornado/pdf-extract-api/blob/main/client/cli.py"
    click FastAPI "https://github.com/CatchTheTornado/pdf-extract-api/blob/main/app/main.py"
    click CeleryTasks "https://github.com/CatchTheTornado/pdf-extract-api/blob/main/app/tasks.py"
    click OCRPipeline "https://github.com/CatchTheTornado/pdf-extract-api/blob/main/app/ocr_strategies/ocr_strategy.py"
    click Marker "https://github.com/CatchTheTornado/pdf-extract-api/blob/main/app/ocr_strategies/marker.py"
    click LlamaVision "https://github.com/CatchTheTornado/pdf-extract-api/blob/main/app/ocr_strategies/llama_vision.py"
    click Tesseract "https://github.com/CatchTheTornado/pdf-extract-api/blob/main/app/ocr_strategies/tesseract.py"
    click StorageManager "https://github.com/CatchTheTornado/pdf-extract-api/blob/main/app/storage_manager.py"
    click LocalFS "https://github.com/CatchTheTornado/pdf-extract-api/blob/main/app/storage_strategies/local_filesystem.py"
    click GDrive "https://github.com/CatchTheTornado/pdf-extract-api/blob/main/app/storage_strategies/google_drive.py"
    click S3 "https://github.com/CatchTheTornado/pdf-extract-api/blob/main/app/storage_strategies/aws_s3.py"
    click StorageProfiles "https://github.com/CatchTheTornado/pdf-extract-api/tree/main/storage_profiles"

    %% Styles
    classDef client fill:#2374ab
    classDef app fill:#48a9a6
    classDef processing fill:#4ad66d
    classDef storage fill:#e4cc37
    classDef cache fill:#d81159
    classDef strategy fill:#96ceb4
    classDef llm fill:#ff9f1c
    classDef queue fill:#d81159
    classDef config fill:#ffbf69
```
