# eurekaer/fedvad architecture

> Architecture diagram of the GitHub repository eurekaer/fedvad, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/eurekaer/fedvad
- Repository: https://github.com/eurekaer/fedvad

## Overview

##### Project Type and Purpose
- This is an open-source, research-oriented Federated Learning system for Video Anomaly Detection (FedVAD) that integrates GPT-driven semantic distillation.
- It orchestrates multiple local “clients” (each holding video feature subsets) to train a shared model without centralizing raw data, and refines local representations via a semantic distillation step powered by GPT/BLIP2/BERT utilities.

##### Major Components
a. Data & Feature Extraction
- Raw video datasets (UCF-Crime, XD-Violence, ShanghaiTech, UBnormal)
- Off-the-shelf I3D feature extractor (external repo)
b. Client Splitting Module (`Client_Split/…`)
- Strategies: Unsupervised, Weakly-Supervised (ST and XD variants)
- Outputs per-client feature lists and ground-truth arrays
c. Federated Learning Engine (`Code/…`)
- main.py – Coordinates training rounds, dispatches model to clients
- train.py – Local client training logic (forward/backward)
- test.py – Local evaluation on test splits
- distill.py – Semantic distillation using GPT embeddings
- fusion.py – Merges local model updates & distilled semantics
- cluster_utils.py – Helper functions for client clustering
- loss.py, layers.py, modules.py, model.py – Core model definitions and loss functions
- log.py – Experiment logging and metrics aggregation
d. External Semantic Services (`other/…`)
- bert_utils.ipynb, blip2_utils.ipynb, gpt4_utils.ipynb – Notebooks and helper scripts to query/fine-tune large language/vision models
e. Environment & Configuration
- environment.yml – Conda environment specification
- README.md – High-level description, requirements, dataset links

##### Relationships & Data Flow
1. Data Preparation: Raw Videos → Off-the-Shelf Feature Extractors → Saved Features
2. Split → Client_Split → Per-Client Feature Lists & GT Labels
3. Federated Loop (in main.py):
a. Server initializes global model
b. Server broadcasts model → Clients
c. Each Client runs train.py:
– Forward on local features
– Optionally call distill.py → query GPT/BLIP2 for semantic embeddings
– Compute local gradients (loss.py) & update local model
d. Clients return updates → fusion.py merges them (e.g., weighted averaging + distilled semantics)
e. log.py captures metrics each round
f. Repeat until convergence
4. Final Evaluation: Global model + per-client test.py → Performance metrics

##### Architectural Patterns & Design Principles
- Federated Client-Server Pattern (central aggregator, multiple learners)
- Modular separation: data splitting, model definition, training, distillation, logging
- Plugin-style semantic distillation: clear extensibility points for adding new LLM/VLM services
- Config-first: environment.yml centralizes dependency management

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph "Data & Preprocessing"
        raw_videos[/"Raw Video Datasets"/]:::data
        i3d_feat["I3D Feature Extractor"]:::service
        features[(Features Storage)]:::data
        raw_videos -->|"videos"| i3d_feat
        i3d_feat -->|"features"| features
    end

    subgraph "Client Splitting Module"
        cs[/"Client_Split/"/]:::service
        unsup_des["Unsupervised/des.py"]:::service
        click unsup_des "https://github.com/eurekaer/fedvad/blob/main/Client_Split/Unsupervised/des.py"
        wsst_des["ST/des.py"]:::service
        click wsst_des "https://github.com/eurekaer/fedvad/blob/main/Client_Split/Weakly_Supervised/ST/des.py"
        wsxd_des["XD/des.py"]:::service
        click wsxd_des "https://github.com/eurekaer/fedvad/blob/main/Client_Split/Weakly_Supervised/XD/des.py"
        features --> cs
        cs --> unsup_des
        cs --> wsst_des
        cs --> wsxd_des
        cs -->|"feature lists & GT labels"| client_grp
    end

    subgraph "Federated Learning Engine"
        srv["Server (main.py)"]:::server
        click srv "https://github.com/eurekaer/fedvad/blob/main/Code/main.py"
        fus["fusion.py"]:::service
        click fus "https://github.com/eurekaer/fedvad/blob/main/Code/fusion.py"
        cluster["cluster_utils.py"]:::service
        click cluster "https://github.com/eurekaer/fedvad/blob/main/Code/cluster_utils.py"
        srv -->|init & dispatch model weights| client_grp

        subgraph "Client Node"
            client_grp((Clients))
            train["train.py"]:::client
            click train "https://github.com/eurekaer/fedvad/blob/main/Code/train.py"
            distill["distill.py"]:::client
            click distill "https://github.com/eurekaer/fedvad/blob/main/Code/distill.py"
            loss["loss.py"]:::service
            click loss "https://github.com/eurekaer/fedvad/blob/main/Code/loss.py"
            layers["layers.py"]:::service
            click layers "https://github.com/eurekaer/fedvad/blob/main/Code/layers.py"
            modules["modules.py"]:::service
            click modules "https://github.com/eurekaer/fedvad/blob/main/Code/modules.py"
            modeldef["model.py"]:::service
            click modeldef "https://github.com/eurekaer/fedvad/blob/main/Code/model.py"
            test["test.py"]:::client
            click test "https://github.com/eurekaer/fedvad/blob/main/Code/test.py"

            client_grp --> train
            train -->|forward/backward| loss
            train -->|uses layers| layers
            train -->|uses modules| modules
            train -->|uses model definition| modeldef
            train --> distill
            test -->|"evaluation metrics"| log
        end

        srv -->|collect updates| fus
        fus --> srv
    end

    subgraph "Logging & Monitoring"
        log["log.py"]:::service
        click log "https://github.com/eurekaer/fedvad/blob/main/Code/log.py"
        metrics[(Metrics Store)]:::data
        srv --> log
        client_grp --> log
        log --> metrics
    end

    subgraph "External Semantic Services"
        distill_ext((Semantic API)):::external
        bert["BERT Utils"]:::external
        click bert "https://github.com/eurekaer/fedvad/blob/main/other/bert_utils.ipynb"
        blip2["BLIP2 Utils"]:::external
        click blip2 "https://github.com/eurekaer/fedvad/blob/main/other/blip2_utils.ipynb"
        gpt4["GPT-4 Utils"]:::external
        click gpt4 "https://github.com/eurekaer/fedvad/blob/main/other/gpt4_utils.ipynb"
        des_ext["des.py"]:::external
        click des_ext "https://github.com/eurekaer/fedvad/blob/main/other/des.py"
        yolo_py["yolov8.py"]:::external
        click yolo_py "https://github.com/eurekaer/fedvad/blob/main/other/yolov8.py"
        yolo_pt["yolov8n.pt"]:::external
        click yolo_pt "https://github.com/eurekaer/fedvad/blob/main/other/yolov8n.pt"

        distill_ext --> bert
        distill_ext --> blip2
        distill_ext --> gpt4
        distill_ext --> des_ext
        distill_ext --> yolo_py
        distill_ext --> yolo_pt
        distill -->|"semantic embeddings"| distill_ext
    end

    subgraph "Environment & Configuration"
        env["environment.yml"]:::service
        click env "https://github.com/eurekaer/fedvad/blob/main/Code/environment.yml"
        readme["README.md"]:::service
        click readme "https://github.com/eurekaer/fedvad/blob/main/README.md"
    end

    classDef data fill:#cce5ff,stroke:#333,stroke-width:1px
    classDef service fill:#e6ffe6,stroke:#333,stroke-width:1px
    classDef server fill:#ffd9b3,stroke:#333,stroke-width:1px
    classDef client fill:#ffe6f2,stroke:#333,stroke-width:1px
    classDef external fill:#f2f2f2,stroke:#333,stroke-width:1px
```
