# mustafakrc/ck-guru architecture

> Architecture diagram of the GitHub repository mustafakrc/ck-guru, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mustafakrc/ck-guru
- Repository: https://github.com/mustafakrc/ck-guru

## Overview

##### Project type and purpose
- CK-Guru is a full-stack, microservices-oriented defect-prediction platform.
- Purpose: ingest Git repositories, compute code metrics, prepare ML datasets, train/predict defect models, and expose a web UI.

##### File structure
- Top level:
- frontend/ – Next.js React app served via Nginx
- backend/ – FastAPI REST API with SQLAlchemy (async) and Celery task dispatch
- worker/ – three Celery worker services (ingestion, dataset, ml)
- shared/ – common code (models, schemas, Celery config, utilities)
- docker-compose.yml – orchestrates all services
- .env, config files, Dockerfiles for each component
- Patterns:
- Microservices via Docker Compose
- API (FastAPI) → Message Broker (RabbitMQ) → Workers (Celery)
- Shared library factoring DB/session/schemas

##### Architecture from README
- REST API (FastAPI)
- Frontend UI (Next.js + Nginx)
- Celery workers (ingestion, dataset, ML)
- PostgreSQL, RabbitMQ, Redis, MinIO, Flower

a. Frontend:
- Next.js app (React)
- Served by Nginx reverse proxy
- Communicates over HTTP(S) with Backend API
b. Backend:
- FastAPI server
- Defines REST endpoints (Repositories, Datasets, Jobs, XAI)
- Uses SQLAlchemy async against PostgreSQL
- Publishes tasks to RabbitMQ queues via Celery (ingestion, dataset, ml)
- Reads task status/results from Redis
c. Workers:
- Ingestion Worker:
- Clones Git repos
- Invokes CK Tool (Java jar) & CommitGuru
- Fetches GitHub issues via API
- Persists raw data to PostgreSQL
- Dataset Worker:
- Reads ingested data from PostgreSQL
- Applies cleaning rules & transformations
- Writes dataset files to MinIO (Parquet)
- Updates metadata in PostgreSQL
- ML Worker:
- Trains models (Scikit-learn), runs Optuna
- Generates model artifacts (.pkl) & XAI outputs
- Saves artifacts to MinIO
- Persists metadata & results in PostgreSQL
d. Shared Library:
- Shared DB models, Pydantic schemas, Celery app configuration
- Imported by backend & all workers
e. Infrastructure:
- PostgreSQL (metadata store)
- RabbitMQ (Celery broker)
- Redis (Celery result backend)
- MinIO (S3-compatible object storage)
- Flower (Celery monitor)

##### Relationships & data flow
1. User → Frontend (HTTP) → Backend
2. Backend enqueues Celery tasks in RabbitMQ
3. Workers consume tasks from RabbitMQ
4. Workers read/write metadata to PostgreSQL
5. Dataset & ML workers store large artifacts in MinIO
6. Backend polls Redis for task status, returns to Frontend
7. Flower connects to Celery/Redis for UI monitoring

##### Architectural patterns & design principles
- Microservices/Separation of concerns: UI, API, background services
- Asynchronous task queue (Celery)
- Shared library to avoid duplication
- Containerization (Docker Compose)
- Config via environment variables
- Clean code layering: API → Services → Repositories → Models

##### Elements
- Blue – UI
- Green – API
- Orange – Workers
- Grey – Shared code

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Frontend Layer
    subgraph "Frontend"
        style FrontendLayer stroke:#000,stroke-width:2px
        Nginx["Nginx Reverse Proxy"]:::frontend
        NextJS["Next.js App"]:::frontend
        Components["UI Components"]:::frontend
    end
    click Nginx "https://github.com/mustafakrc/ck-guru/blob/main/frontend/nginx.conf"
    click NextJS "https://github.com/mustafakrc/ck-guru/tree/main/frontend/app/"
    click Components "https://github.com/mustafakrc/ck-guru/tree/main/frontend/components/"
    click FrontendLayer "https://github.com/mustafakrc/ck-guru/tree/main/frontend/"

    %% Backend Layer
    subgraph "Backend API"
        style BackendLayer stroke:#000,stroke-width:2px
        BackendAPI["FastAPI Server"]:::backend
        Endpoints["API Endpoints"]:::backend
        CeleryConfig["Celery App Config"]:::backend
        CRUD["CRUD Operations"]:::backend
        Services["Business Logic Services"]:::backend
        Alembic["DB Migrations"]:::backend
    end
    click BackendAPI "https://github.com/mustafakrc/ck-guru/blob/main/backend/app/main.py"
    click Endpoints "https://github.com/mustafakrc/ck-guru/tree/main/backend/app/api/v1/endpoints/"
    click CeleryConfig "https://github.com/mustafakrc/ck-guru/blob/main/backend/app/core/celery_app.py"
    click CRUD "https://github.com/mustafakrc/ck-guru/tree/main/backend/app/crud/"
    click Services "https://github.com/mustafakrc/ck-guru/tree/main/backend/app/services/"
    click Alembic "https://github.com/mustafakrc/ck-guru/tree/main/backend/alembic/"
    click BackendLayer "https://github.com/mustafakrc/ck-guru/tree/main/backend/app/"

    %% Shared Library
    subgraph "Shared Library"
        style SharedLayer stroke:#888,stroke-width:2px,stroke-dasharray: 5 5
        Shared["Common Modules"]:::shared
    end
    click Shared "https://github.com/mustafakrc/ck-guru/tree/main/shared/"

    %% Workers Layer
    subgraph "Celery Workers"
        style WorkerLayer stroke:#000,stroke-width:2px
        Ingestion["Ingestion Worker"]:::worker
        Dataset["Dataset Worker"]:::worker
        ML["ML Worker"]:::worker
    end
    click Ingestion "https://github.com/mustafakrc/ck-guru/blob/main/worker/ingestion/app/tasks.py"
    click Ingestion "https://github.com/mustafakrc/ck-guru/blob/main/worker/ingestion/app/main.py"
    click Ingestion "https://github.com/mustafakrc/ck-guru/tree/main/worker/ingestion/services/"
    click Ingestion "https://github.com/mustafakrc/ck-guru/blob/main/worker/ingestion/entrypoint.sh"
    click Dataset "https://github.com/mustafakrc/ck-guru/blob/main/worker/dataset/app/tasks.py"
    click Dataset "https://github.com/mustafakrc/ck-guru/blob/main/worker/dataset/app/main.py"
    click Dataset "https://github.com/mustafakrc/ck-guru/tree/main/worker/dataset/services/"
    click Dataset "https://github.com/mustafakrc/ck-guru/blob/main/worker/dataset/entrypoint.sh"
    click ML "https://github.com/mustafakrc/ck-guru/blob/main/worker/ml/app/tasks.py"
    click ML "https://github.com/mustafakrc/ck-guru/blob/main/worker/ml/app/main.py"
    click ML "https://github.com/mustafakrc/ck-guru/tree/main/worker/ml/services/"
    click ML "https://github.com/mustafakrc/ck-guru/blob/main/worker/ml/entrypoint.sh"
    click WorkerLayer "https://github.com/mustafakrc/ck-guru/tree/main/worker/"

    %% Infrastructure Layer
    subgraph "Infrastructure"
        style InfraLayer stroke:#000,stroke-width:2px
        Postgres["PostgreSQL"]:::db
        RabbitMQ["RabbitMQ"]:::queue
        Redis["Redis"]:::db
        MinIO["MinIO Object Storage"]:::storage
        Flower["Flower Monitor"]:::monitor
    end
    click Orchestration "https://github.com/mustafakrc/ck-guru/blob/main/docker-compose.yml"

    %% Connections
    Nginx -->|"HTTP(S)"| BackendAPI
    NextJS -->|"HTTP(S)"| BackendAPI
    BackendAPI -->|"AMQP (enqueue)"| RabbitMQ
    RabbitMQ -->|"AMQP (consume)"| Ingestion
    RabbitMQ -->|"AMQP (consume)"| Dataset
    RabbitMQ -->|"AMQP (consume)"| ML
    BackendAPI -->|"SQL (read/write)"| Postgres
    Ingestion -->|"SQL (persist raw data)"| Postgres
    Dataset -->|"SQL (read/write)"| Postgres
    ML -->|"SQL (persist metadata)"| Postgres
    Dataset -->|"S3 API (store datasets)"| MinIO
    ML -->|"S3 API (store artifacts)"| MinIO
    BackendAPI -->|"Redis (status poll)"| Redis
    Flower -->|"Monitor tasks"| RabbitMQ
    Flower -->|"Monitor results"| Redis

    %% Shared library usage
    Shared --> BackendAPI
    Shared --> Ingestion
    Shared --> Dataset
    Shared --> ML

    %% Styles
    classDef frontend fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef backend fill:#90EE90,stroke:#333,stroke-width:1px
    classDef worker fill:#FFA500,stroke:#333,stroke-width:1px
    classDef shared fill:#D3D3D3,stroke:#666,stroke-width:1px,stroke-dasharray: 2 2
    classDef db fill:#F5DEB3,stroke:#333,stroke-width:1px
    classDef queue fill:#FFFACD,stroke:#333,stroke-width:1px
    classDef storage fill:#98FB98,stroke:#333,stroke-width:1px
    classDef monitor fill:#FFE4E1,stroke:#333,stroke-width:1px
```
