# edge-video-services/ekya architecture

> Architecture diagram of the GitHub repository edge-video-services/ekya, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/edge-video-services/ekya
- Repository: https://github.com/edge-video-services/ekya

## Overview

##### Project Type and Purpose
- Ekya is a research‐grade, full-pipeline continuous‐learning system for video analytics on edge servers. It ingests live video streams (or prerecorded MP4s), performs object detection/inference, periodically retrains models in the background, and intelligently allocates GPU resources among inference and retraining jobs via a scheduler (e.g., “Thief Scheduler”).
- It uses Ray for distributed execution (actors/tasks), NVIDIA MPS for fine‐grained GPU sharing, and supports multiple datasets (Cityscapes, Waymo, UrbanTraffic, UrbanBuilding).

##### Top-Level Module Breakdown
- ekya/datasets & deployment: video samplelist generation and dataset preparation scripts
- ekya/classes & ekya/models: abstractions for cameras, ML models (ResNet, iCaRL, object detectors), and inference managers
- ekya/microprofilers: short “probe” runs to estimate retraining job resource→accuracy curves (“promise”)
- ekya/schedulers: pluggable scheduling algorithms (thief, fair, no_retrain, utilitysim) implementing a common BaseScheduler interface
- ekya/drivers & ekya/experiment_drivers: orchestrate end‐to‐end runs for paper figures, hyperparam sweeps, and dataset‐specific “driver_*.sh” scripts
- ekya/simulation & simulator/: lightweight simulators for validation and utilitysim schedule generation
- ekya/model_cache: baseline continuous‐model‐selection pipeline for comparison
- ekya/benchmarking & ekya/analysis & ekya/plotting_scripts & ekya/viz: offline analysis, benchmarking, and figure‐generation notebooks/scripts
- utils, tests, setup.py, environment.yml: support code, CI, and packaging

##### Key Architectural Components
A. Data Ingestion & Preprocessing
- Video streams (live MP4s or dataset folders) → ekya/datasets scripts → “samplelists” CSVs
- Dataset loader feeds frames into Ray actors for inference or retraining
B. Microprofiling
- ekya/microprofilers runs short training jobs with varying hyperparams to generate performance profiles
- Profiles stored in memory or on disk (profile_generator → runtime_data)
C. Scheduler & Resource Manager
- ekya/schedulers/BaseScheduler defines reallocation_callback() and get_inference_schedule()
- Implements Thief Scheduler: steal GPU cycles from low‐promise jobs → reallocate to high‐promise ones
- Scheduler consults microprofile data, current camera/job status, GPU capacity
D. Execution Engines
- Inference Engine: Ray remote actors running small‐chunk inference tasks continuously
- Retraining Engine: Ray actors running fine‐tuning jobs on per-camera models (with history weighting, ICaRL support)
- GPU managed via NVIDIA MPS daemon (fine‐grain resource slicing)
E. Model Cache (Baseline)
- ekya/model_cache: trains a set of “scenario” models offline and selects per‐window model at runtime
F. Simulation & Benchmarking
- simulator/ and ekya/simulation: discrete‐event simulators for scheduler validation
- ekya/benchmarking/mps: end-to-end profiling of candidate models on CIFAR10, Cityscapes for MPS overhead analysis
G. Logging & Visualization
- logs and results written under timestamped directories (results/ekya_expts/…)
- ekya/viz and ekya/plotting_scripts: Jupyter notebooks and scripts for generating figures

##### Interactions & Data Flows
- Inference Engine ──► Outputs detections & accuracy metrics
- Retraining Engine ──► Updated model checkpoints
- Inference Engine & Retraining Engine receive updated resource weights

##### Architectural Patterns & Principles
- Modular, plug-in schedulers via common interface (Open/Closed Principle)
- Actor model concurrency using Ray remote functions/actors for inference and training jobs
- Resource slicing via NVIDIA MPS and scheduler callbacks
- Separation of concerns: data ingestion, profiling, scheduling, execution, analysis

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Layer 1: Data Ingestion
    subgraph "Data Ingestion"
        direction TB
        VideoSource["Video Streams"]:::external
        SampleListGen["Samplelist Generator"]:::compute
        DataLoader["Data Loader"]:::compute
    end

    %% Layer 2: Profiling
    subgraph "Profiling"
        direction TB
        Microprofiler["Microprofiler Actors"]:::compute
        ProfileStore["Profile Store (DB)"]:::data
    end

    %% Layer 3: Scheduling
    subgraph "Scheduling"
        direction TB
        Scheduler["Thief / Fair / NoRetrain / UtilitySim Scheduler"]:::scheduler
    end

    %% Layer 4: Execution
    subgraph "Execution"
        direction TB
        RayCluster["Ray Cluster"]:::external
        MPS["NVIDIA MPS daemon"]:::external
        Inference["Ray Inference Actors"]:::compute
        Retraining["Ray Retraining Actors"]:::compute
    end

    %% Layer 5: Storage & Logging
    subgraph "Storage & Logging"
        direction TB
        Checkpoints["Model Checkpoint Store"]:::data
        Logs["Metrics Log"]:::data
    end

    %% Layer 6: Analysis
    subgraph "Analysis"
        direction TB
        Benchmark["Benchmarking & MPS Profiling"]:::compute
        Simulation["Simulator Modules"]:::compute
        Viz["Visualization Notebooks"]:::compute
    end

    %% Connections: Data Flow
    VideoSource --> SampleListGen --> DataLoader
    DataLoader --> Inference
    DataLoader --> Retraining

    %% Profiling Flow
    Retraining --> Microprofiler --> ProfileStore

    %% Scheduling Flow
    ProfileStore --> Scheduler
    DataLoader --> Scheduler
    Scheduler -.->|resource allocation| Inference
    Scheduler -.->|resource allocation| Retraining

    %% Execution Engines with external dependencies
    RayCluster --> Inference
    RayCluster --> Retraining
    MPS --> Inference
    MPS --> Retraining

    %% Storage & Logging Flow
    Inference --> Logs
    Retraining --> Checkpoints

    %% Analysis Flow
    Logs --> Viz
    Logs --> Benchmark
    ProfileStore --> Simulation

    %% Click Events
    click SampleListGen "https://github.com/edge-video-services/ekya/tree/main/ekya/datasets/scripts/"
    click DataLoader "https://github.com/edge-video-services/ekya/tree/main/ekya/datasets/"
    click Microprofiler "https://github.com/edge-video-services/ekya/blob/main/ekya/microprofilers/base_microprofiler.py"
    click ProfileStore "https://github.com/edge-video-services/ekya/blob/main/ekya/microprofilers/runtime_data.py"
    click Scheduler "https://github.com/edge-video-services/ekya/blob/main/ekya/schedulers/scheduler.py"
    click Inference "https://github.com/edge-video-services/ekya/blob/main/ekya/models/inference_manager.py"
    click Retraining "https://github.com/edge-video-services/ekya/blob/main/ekya/models/icarl_model.py"
    click RayCluster "https://github.com/edge-video-services/ekya/blob/main/ekya/utils/loggeractor.py"
    click MPS "https://github.com/edge-video-services/ekya/blob/main/ekya/utils/mps.py"
    click Checkpoints "https://github.com/edge-video-services/ekya/blob/main/ekya/model_cache/run.py"
    click Logs "https://github.com/edge-video-services/ekya/blob/main/ekya/utils/monitoractor.py"
    click Benchmark "https://github.com/edge-video-services/ekya/blob/main/ekya/benchmarking/checkpointing.py"
    click Simulation "https://github.com/edge-video-services/ekya/blob/main/ekya/simulation/run.py"
    click Viz "https://github.com/edge-video-services/ekya/tree/main/ekya/viz/"

    %% Styles
    classDef compute fill:#AED6F1,stroke:#21618C,color:#1B2631
    classDef data fill:#ABEBC6,stroke:#196F3D,color:#1B2631
    classDef scheduler fill:#F9E79F,stroke:#B7950B,color:#1B2631
    classDef external fill:#D5D8DC,stroke:#566573,color:#1B2631
```
