How catchthetornado/text-extract-api works
Main Components
a) Client Layer:
• CLI Client (Python-based)
• REST API Endpoints (FastAPI)
• TypeScript Client Library
b) Application Core:
• FastAPI Application Server
• Celery Task Queue System
• Redis Cache
• Ollama LLM Service
c) Processing Strategies:
• OCR Processors:
• Marker
• LLama Vision
• Tesseract
• LLM Processing:
• Llama 3.1
• Llama 3.2-vision
d) Storage Layer:
• Storage Strategy Interface
• Implementations:
• Local Filesystem
• Google Drive
• AWS S3
Input Flow
• Client → FastAPI Endpoints
• FastAPI → Celery Queue
• Celery → OCR Processors
Processing Flow
• OCR Processors → Redis Cache
• Redis Cache → LLM Processing
• LLM Processing → Results Storage
Storage Flow
• Results → Storage Strategy Interface
• Storage Strategy → Specific Storage Implementation
Architectural Patterns to Highlight
• Strategy Pattern: For OCR and Storage implementations
• Queue-based Architecture: Using Celery for task processing
• Cache Layer: Redis for OCR results
• Microservices-like Structure: Separate services for OCR, LLM, and storage
Technical details
Environment Components:
• FastAPI (Main API Server)
• Celery (Task Queue)
• Redis (Cache & Message Broker)
• Ollama (LLM Service)
File Processing Support:
• Images
• Office Documents
Output Formats:
• Markdown
• JSON
• Structured Data