How mustafakrc/ck-guru works
Project type and purpose
• CK-Guru is a full-stack, microservices-oriented defect-prediction platform.
• Purpose: ingest Git repositories, compute code metrics, prepare ML datasets, train/predict defect models, and expose a web UI.
File structure
• Top level:
• frontend/ – Next.js React app served via Nginx
• backend/ – FastAPI REST API with SQLAlchemy (async) and Celery task dispatch
• worker/ – three Celery worker services (ingestion, dataset, ml)
• shared/ – common code (models, schemas, Celery config, utilities)
• docker-compose.yml – orchestrates all services
• .env, config files, Dockerfiles for each component
• Patterns:
• Microservices via Docker Compose
• API (FastAPI) → Message Broker (RabbitMQ) → Workers (Celery)
• Shared library factoring DB/session/schemas
Architecture from README
• REST API (FastAPI)
• Frontend UI (Next.js + Nginx)
• Celery workers (ingestion, dataset, ML)
• PostgreSQL, RabbitMQ, Redis, MinIO, Flower
a. Frontend:
• Next.js app (React)
• Served by Nginx reverse proxy
• Communicates over HTTP(S) with Backend API
b. Backend:
• FastAPI server
• Defines REST endpoints (Repositories, Datasets, Jobs, XAI)
• Uses SQLAlchemy async against PostgreSQL
• Publishes tasks to RabbitMQ queues via Celery (ingestion, dataset, ml)
• Reads task status/results from Redis
c. Workers:
• Ingestion Worker:
• Clones Git repos
• Invokes CK Tool (Java jar) & CommitGuru
• Fetches GitHub issues via API
• Persists raw data to PostgreSQL
• Dataset Worker:
• Reads ingested data from PostgreSQL
• Applies cleaning rules & transformations
• Writes dataset files to MinIO (Parquet)
• Updates metadata in PostgreSQL
• ML Worker:
• Trains models (Scikit-learn), runs Optuna
• Generates model artifacts (.pkl) & XAI outputs
• Saves artifacts to MinIO
• Persists metadata & results in PostgreSQL
d. Shared Library:
• Shared DB models, Pydantic schemas, Celery app configuration
• Imported by backend & all workers
e. Infrastructure:
• PostgreSQL (metadata store)
• RabbitMQ (Celery broker)
• Redis (Celery result backend)
• MinIO (S3-compatible object storage)
• Flower (Celery monitor)
Relationships & data flow
1. User → Frontend (HTTP) → Backend
2. Backend enqueues Celery tasks in RabbitMQ
3. Workers consume tasks from RabbitMQ
4. Workers read/write metadata to PostgreSQL
5. Dataset & ML workers store large artifacts in MinIO
6. Backend polls Redis for task status, returns to Frontend
7. Flower connects to Celery/Redis for UI monitoring
Architectural patterns & design principles
• Microservices/Separation of concerns: UI, API, background services
• Asynchronous task queue (Celery)
• Shared library to avoid duplication
• Containerization (Docker Compose)
• Config via environment variables
• Clean code layering: API → Services → Repositories → Models
Elements
• Blue – UI
• Green – API
• Orange – Workers
• Grey – Shared code