# uhhfeef/cambai-challenge architecture

> Architecture diagram of the GitHub repository uhhfeef/cambai-challenge, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-24). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/uhhfeef/cambai-challenge
- Repository: https://github.com/uhhfeef/cambai-challenge

## Overview

##### Project Type & Purpose
- This is a full-stack backend service built in Python using FastAPI.
- The primary purpose is secure API key management with multi-tenant support. It ensures high availability (via Redis master-replica and Sentinel), logs all operations to Loki (for audit/log management), and exposes REST API endpoints for authentication, data operations, and API key management.

##### Main Components
a. FastAPI Application:
- REST API endpoints (Authentication, Data Operations, API Key management).
- Mention core modules (e.g., app/api/routes for endpoints, app/core for config and security, app/models for data structures, and app/tasks for background processing via Huey).

b. Database Layer:
- Redis as the primary datastore with a strategy for master-replica connections.
- Redis Sentinel (for monitoring and automatic failover).

c. Background Task Processing:
- Huey, which manages background logging tasks.

d. Logging and Monitoring:
- Loki for audit log storage.
- Grafana for log visualization.
- Prometheus for monitoring and alerting.

e. Deployment Infrastructure:
- Kubernetes clusters with deployment objects for FastAPI app, Redis, Huey tasks, Loki, Grafana, and Prometheus.
- Configuration files in the k8s directory (organized by component) indicate a microservices-style deployment.

##### Relationships & Data Flow
- The FastAPI application processes incoming API requests and interacts with the Redis database.
- Redis connection management includes trying multiple Redis pods (with failure handling via Sentinel).
- Background tasks (scheduled via Huey) pick up log offloading and auditing operations.
- Audit logs generated during API operations are stored temporarily in Redis and later forwarded to Loki.

##### Architectural Patterns & Principles
- Modular application structure: Separation of concerns among API endpoints, core logic (security, configurations), database interactions, and asynchronous task processing.
- Resilience/failover pattern: Smart retry mechanisms and handling of Redis ReadOnlyError during master-replica failovers.
- Multi-tenancy: Ensuring data isolation in API operations and logs (using tenant IDs in both the application and Loki logs).
- Containerization and orchestration via Docker and Kubernetes for scalable deployments.

##### Detailed Component Separation
- Divide the application into sub-components:
- "API Layer": Represent the FastAPI endpoints, clearly separating authentication endpoints, data operations, and API key management.
- "Core Services": Include the modules for configuration, security, and data models.
- "Database Services": A main block for Redis with annotations about the master-replica architecture and Sentinel for failover.
- "Monitoring & Logging": Separate blocks for Loki (with a note on multi-tenancy and audit logging), Grafana, and Prometheus.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External
    ExternalClient["External Client"]:::external

    %% FastAPI Application Subgraph
    subgraph "FastAPI Application"
        APIEntry["FastAPI Entry (app.py)"]:::service
        APIRoutes["API Routes (app/api/routes)"]:::service
        CoreServices["Core Services (app/core)"]:::service
        DataModels["Data Models (app/models)"]:::service
    end

    %% Database Services Subgraph
    subgraph "Database Services"
        RedisDB["Redis Database (Master/Replica)"]:::database
        Sentinel["Redis Sentinel"]:::database
    end

    %% Task Processing Subgraph
    subgraph "Task Processing"
        HueyTask["Huey Task Queue"]:::task
    end

    %% Monitoring and Logging Subgraph
    subgraph "Monitoring & Logging"
        LokiLog["Loki (Audit Logging)"]:::monitoring
        GrafanaDash["Grafana (Visualization)"]:::monitoring
        PrometheusMon["Prometheus (Monitoring)"]:::monitoring
    end

    %% Kubernetes Deployment Infrastructure Subgraph
    subgraph "Deployment Infrastructure (Kubernetes)"
        K8sFastAPI["FastAPI Deployment (k8s/fastapi)"]:::k8s
        K8sRedis["Redis Deployment (k8s/redis)"]:::k8s
        K8sHuey["Huey Deployment (k8s/fastapi/huey-deployment.yaml)"]:::k8s
        K8sLoki["Loki Deployment (k8s/loki)"]:::k8s
        K8sGrafana["Grafana Deployment (k8s/grafana)"]:::k8s
        K8sPrometheus["Prometheus Deployment (k8s/prometheus)"]:::k8s
    end

    %% Connections
    ExternalClient -->|"sends_requests"| APIEntry

    %% FastAPI interactions
    APIEntry -->|"calls_endpoints"| APIRoutes
    APIEntry -->|"uses"| CoreServices
    APIEntry -->|"utilizes"| DataModels
    APIEntry -->|"read/write"| RedisDB
    APIEntry -->|"triggers"| HueyTask

    %% Redis and Sentinel relationship
    RedisDB -- "monitored_by" --> Sentinel
    Sentinel -- "failover_to" --> RedisDB

    %% Background task flows
    HueyTask -->|"offloads_logs"| LokiLog

    %% Monitoring interactions
    LokiLog -->|"feeds"| GrafanaDash
    PrometheusMon ---|"scrapes"| APIEntry
    PrometheusMon ---|"scrapes"| RedisDB

    %% Kubernetes deployments linking to components
    K8sFastAPI ---|"hosts"| APIEntry
    K8sRedis ---|"hosts"| RedisDB
    K8sHuey ---|"hosts"| HueyTask
    K8sLoki ---|"hosts"| LokiLog
    K8sGrafana ---|"hosts"| GrafanaDash
    K8sPrometheus ---|"hosts"| PrometheusMon

    %% Click Events for FastAPI Application
    click APIEntry "https://github.com/uhhfeef/cambai-challenge/blob/main/app.py"
    click APIRoutes "https://github.com/uhhfeef/cambai-challenge/tree/main/app/api/routes"
    click CoreServices "https://github.com/uhhfeef/cambai-challenge/tree/main/app/core"
    click DataModels "https://github.com/uhhfeef/cambai-challenge/tree/main/app/models"

    %% Click Events for Redis Database
    click RedisDB "https://github.com/uhhfeef/cambai-challenge/tree/main/app/db"
    click K8sRedis "https://github.com/uhhfeef/cambai-challenge/tree/main/k8s/redis"

    %% Click Events for Background Task Processing (Huey)
    click HueyTask "https://github.com/uhhfeef/cambai-challenge/tree/main/app/tasks"
    click K8sHuey "https://github.com/uhhfeef/cambai-challenge/blob/main/k8s/fastapi/huey-deployment.yaml"

    %% Click Events for Logging and Monitoring - Loki
    click LokiLog "https://github.com/uhhfeef/cambai-challenge/tree/main/k8s/loki"
    click LokiLog "https://github.com/uhhfeef/cambai-challenge/blob/main/tests/direct_loki_test.py"

    %% Click Events for Logging and Monitoring - Grafana
    click GrafanaDash "https://github.com/uhhfeef/cambai-challenge/tree/main/k8s/grafana"

    %% Click Events for Logging and Monitoring - Prometheus
    click PrometheusMon "https://github.com/uhhfeef/cambai-challenge/tree/main/k8s/prometheus"
    click PrometheusMon "https://github.com/uhhfeef/cambai-challenge/blob/main/tests/monitor.sh"
    click PrometheusMon "https://github.com/uhhfeef/cambai-challenge/blob/main/tests/monitor_hpa.sh"

    %% Click Events for Kubernetes Deployments
    click K8sFastAPI "https://github.com/uhhfeef/cambai-challenge/tree/main/k8s/fastapi"
    click K8sLoki "https://github.com/uhhfeef/cambai-challenge/tree/main/k8s/loki"
    click K8sGrafana "https://github.com/uhhfeef/cambai-challenge/tree/main/k8s/grafana"
    click K8sPrometheus "https://github.com/uhhfeef/cambai-challenge/tree/main/k8s/prometheus"

    %% Styles
    classDef external fill:#d3d3d3,stroke:#000,stroke-width:1px;
    classDef service fill:#f9c74f,stroke:#000,stroke-width:1px;
    classDef database fill:#90be6d,stroke:#000,stroke-width:1px;
    classDef task fill:#f9844a,stroke:#000,stroke-width:1px;
    classDef monitoring fill:#577590,stroke:#000,stroke-width:2px,color:#fff;
    classDef k8s fill:#43aa8b,stroke:#000,stroke-width:1px;
```
