How danilbaibak/real-time-ml-prediction works
Project type and purpose
• This is a backend‐only, RESTful Machine Learning service that exposes real‐time prediction endpoints.
• The README describes it as an example ML REST API covering model caching, asynchronous code, zero‐downtime pipeline updates, health checks, and separation of production vs. local environments.
File structure
Top‐level artifacts:
• docker/ & docker-compose-local.yml: contains Dockerfiles, bash init, Python requirements → containerized deployment
• Makefile: orchestration commands (init, up, restart, bash)
• src/: the application code
– server.py: entry point for FastAPI (or similar) REST API
– config.ini + logging.conf: configuration & logging
– models/: Pydantic schemas for request/response validation
– scripts/training.py: pipeline to train (and persist) an ML model
– sql/migrations/*.sql: Alembic-style migration scripts for pipeline_version and predictions tables
– utils/: helper modules (common functions, utilities)
– tests/: unit and integration test suites
README for additional insights
• It uses a Dockerized local environment.
• ML artifacts are saved under ml_pipelines (implied in container or volume)
• Exposes /health and /docs endpoints.
• Database stores training history and predictions.
a. Main components
1. API Layer (server.py)
• Exposes REST endpoints: /predict, /health, /docs
• Uses Pydantic models for input/output
2. Model Cache & Pipeline Loader
• In-memory (or external cache) to store the current ML pipeline
• Asynchronous reload endpoint or on-demand training triggers
3. ML Training Pipeline
• scripts/training.py reads data, fits model, serializes artifacts (ml_pipelines folder)
• Writes pipeline version to the database via migrations
4. Database (PostgreSQL)
• Tables: pipeline_version, predictions
• Manages metadata about model versions and persisted prediction logs
5. Docker / Container Orchestration
• Service container running the API + training scripts
• Database container
• docker-compose-local for local dev
6. External Clients
• Web or CLI clients calling /predict or /docs
b. Relationships & interactions
• Client → API Layer: HTTP calls (JSON in/out)
• API Layer → Model Cache: Fetches the loaded pipeline to compute predictions
• API Layer → Database:
– Reads latest pipeline_version for metadata/health
– Writes prediction results to predictions table
• ML Training Pipeline → Model Cache: Updates serialized artifacts on disk
• ML Training Pipeline → Database: Inserts new entry into pipeline_version
• Docker Compose: wires API container and DB container on a shared network
c. Architectural patterns & design principles
• Separation of concerns:
– server.py for routing, models/ for schema validation, utils/ for helper logic, scripts/ for data science pipeline
• Monolithic service with clear internal modules rather than microservices
• Asynchronous operation: training or reload can be invoked without blocking prediction requests
• Immutable artifacts: each pipeline version is uniquely tracked in the database
• Infrastructure as code: Dockerfiles, docker-compose, Makefile
d. Key technologies & libraries
• FastAPI (or Starlette) for REST endpoints and auto-generated Swagger (/docs)
• Pydantic for data validation (models/)
• Async Python (asyncio) for non-blocking operations
• SQL migrations (plain SQL under src/sql/migrations)
• Docker & docker-compose for containerization
• pytest or unittest for tests (src/tests/)
Essential elements to capture
• CI/CD: reference .github/workflows/linting.yml as a note for code quality.