How podyapolskiy/follow-my-reading works
Main Components and Their Roles
• API Layer (FastAPI):
• Represented by the api/ directory. It includes versioned endpoints (v1) for handling audio (api/v1/audio.py), image (api/v1/image.py), authentication (api/v1/auth.py), and related tasks (comparison, text, etc.).
• This is where incoming requests (image and audio uploads) are received from clients.
• Core Processing:
• Located in the core/ directory, this layer is responsible for processing logic. It contains modules for audio splitting (core/processing/audio_split.py) and text processing (core/processing/text.py).
• It also contains a task system (core/task_system.py) which likely orchestrates distributed computing with Huey and integrates with Redis for queuing.
• Plugin System:
• The core/plugins/ directory (with base, loader, no_mem) and the plugins/ directory illustrate a flexible plugin architecture.
• Plugins (e.g., ara_tesseract_plugin.py, en_ru_easyocr_plugin.py, whisper_plugin.py, etc.) provide support for multi-language OCR and speech processing. They extend the base functionality and allow the system to support various recognition technologies.
• Deployment and Configuration:
• The deploy/ directory includes Dockerfiles and shell scripts for containerized or standalone deployments.
• Configuration files like config.py, example.config.json, and pyproject.toml further characterize how the system is set up.
• Testing and Documentation:
• The tests/ directory provides unit tests and sample audio/image files.
• Additional documentation is provided externally (GitLab Pages and a Wiki).
Relationships and Data Flow
• Client Interaction:
• API to Core Processing:
• The API routes delegate tasks to the core processing modules for handling image processing, audio segmentation, and comparisons (as seen in endpoints like comparison in api/v1/comparison.py).
• Asynchronous Task Processing:
• Leveraging Redis and Huey (not explicitly found in code but inferred from README), the task system queue requests such as audio splitting or plugin-based analysis to be processed asynchronously.
• Plugin Interactions:
• The plugin loader within core/plugins ensures that the proper language/technology-specific plugin (from the plugins/ folder) is invoked per processing requirement.
• The plugins are essentially separate, interchangeable components, so represent them as modular extensions connected to the processing layer.
• External Systems:
• Although there may not be a visible database layer aside from Redis, include an “External Services” label for Redis (for caching and messaging) and Huey (as the task queue).
Architectural Patterns and Technologies
• Use a layered architecture: Distinguish clearly between:
• The API/controller layer (FastAPI endpoints)
• The core business logic (task system, processing modules)
• The plugin extension mechanism (for language and OCR/speech variations)
• Asynchronous and distributed processing mechanism which is central to handling audio/image processing.