How rnwl/ml-pdf-splitter works
Below are detailed guidelines and instructions for capturing the architecture:
• ML PDF Splitter Service (Containerized App)
• This as the primary processing component. It hosts your core logic (see files in the src directory such as api.py, pdf2text.py, timed_logger.py, and utils.py) that handles the PDF splitting and parallel orchestration.
• That it is deployed on Azure Container Apps.
• PDF Extraction Azure Function
• This as a separate, external service that performs the extraction of text from each PDF chunk.
• That it is deployed on Azure Functions and is invoked by the splitter app.
• Data Store/File Repository (optional)
• Depending on whether the incoming PDFs are stored or managed as files within a repository (as hinted by the data directory), include a component that represents input (and possibly output) file storage.
• Deployment and CI/CD
• Components for CI/CD pipelines using GitHub workflows (see .github/workflows) and deployment scripts/configurations (Dockerfile, docker-compose.yml, Pipfile files).
Relationships and Interactions
• Client / User Request:
• If applicable, include an entry point or client that triggers the application – this could be shown as a user uploading PDF files, which then get routed to the ML PDF Splitter container.
• Communication Flow:
• Data Flow:
• If there’s intermediate storage involved (e.g., for storing chunks or logging results), represent this connection.
Architectural Patterns and Technologies
• Containerization & Cloud Deployment:
• That the app is containerized (as indicated by the Dockerfile and docker-compose configuration) and deployed on Azure Container Apps.
• Separation between the processing layer (Container App) and the function layer (Azure Functions).
• Parallel Processing:
• Design principle of making parallel asynchronous calls to the Azure Function, enabling rapid PDF decoding.
• Logging and Monitoring:
• Optionally, include a component for logging (referenced by timed_logger.py) to capture and monitor application behavior.
• Development and Operations:
• That the CI/CD process is handled via GitHub workflows, ensuring updates and deployments are continuously integrated and delivered.