# jayzhang42/federatedgpt-shepherd architecture

> Architecture diagram of the GitHub repository jayzhang42/federatedgpt-shepherd, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jayzhang42/federatedgpt-shepherd
- Repository: https://github.com/jayzhang42/federatedgpt-shepherd

## Overview

##### Project type and purpose
- Shepherd is a full-stack research framework for federated instruction tuning of large language models (LLMs).
- It orchestrates multiple simulated clients that each perform local fine-tuning (using PEFT/LoRA) on private instruction datasets, and a central aggregator that merges their updates into a global model.
- It also supports data preparation, global inference (via Gradio), and evaluation.

##### File structure
- Top-level scripts and files:
- client_data_allocation.py – splits/generates per-client datasets
- main.py – entry point for federated fine-tuning (server orchestration)
- GlobalModel_generate.py – global model inference via Gradio
- Directories:
- fed_utils/ – core FL primitives
– client.py – client API (GeneralClient class)
– client_participation_scheduling.py – selects a subset of clients each round
– model_aggregation.py – federated averaging/aggregation logic
– evaluation.py – evaluation metrics, logging
– other.py – utility routines
- templates/ – JSON templates for instruction-fine-tuning
- utils/ – shared helpers for callbacks and prompt construction
- Configuration files:
- requirements.txt – dependencies (Hugging Face transformers, PEFT, bitsandbytes, Gradio)
- new-databricks-dolly-15k.json – raw data source for splitting
- assets/ – images for README and paper illustrations

##### README for additional insights
- Describes “FedIT” algorithm: federated instruction tuning using LoRA on clients + HF PEFT + bitsandbytes.
- Workflow: data preparation → federated rounds → global inference.
- Highlights key hyperparameters (num_clients, rounds, LoRA ranks, scheduler fraction).
- Mentions integration with Hugging Face Model Hub and Gradio.

##### Essential details
- Utils callbacks/prompter feeding into local training.
- Templates directory as configuration inputs to clients.
- Gradio inference flow as a distinct final stage.
- LoRA adapter flow within the local trainer.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Preparation Layer
    subgraph "Data Preparation Layer"
        direction TB
        RawData["Raw Data\n(new-databricks-dolly-15k.json)"]:::datastore
        DataPrep["Data Preparation\n(client_data_allocation.py)"]:::process
        PerClientData["Per-Client Datasets\n(./data/)"]:::datastore
    end

    RawData -->|reads raw JSON| DataPrep
    DataPrep -->|writes per-client JSON| PerClientData

    %% Server Side
    subgraph "Server Side" 
        direction TB
        Orchestrator["Orchestrator\n(main.py)"]:::server
        Scheduler["Client Scheduler\n(fed_utils/client_participation_scheduling.py)"]:::server
        Aggregator["Model Aggregator\n(fed_utils/model_aggregation.py)"]:::server
        Evaluation["Evaluation & Logging\n(fed_utils/evaluation.py)"]:::server
        ModelStore["Global Model Store\n(weights & LoRA adapters)"]:::datastore
        Dependencies["Dependencies\n(requirements.txt)"]:::config
        Assets["Assets\n(assets/)"]:::config
    end

    PerClientData -->|provides data| Orchestrator
    Orchestrator -->|selects clients| Scheduler
    Scheduler -->|client list| Orchestrator
    Orchestrator -->|sends global weights| ModelStore
    ModelStore -->|provides weights| Orchestrator

    %% Client Side
    subgraph "Client Side"
        direction TB
        ClientAPI["Local Trainer\n(GeneralClient)"]:::client
        Templates["Instruction Templates\n(templates/)"]:::config
        Callbacks["Callback Helpers\n(utils/callbacks.py)"]:::client
        Prompter["Prompt Utilities\n(utils/prompter.py)"]:::client
        OtherUtils["Misc FL Utilities\n(fed_utils/other.py)"]:::client
    end

    Orchestrator -->|invokes| ClientAPI
    ClientAPI -->|loads data| PerClientData
    ClientAPI -->|loads templates| Templates
    ClientAPI -->|uses callbacks| Callbacks
    ClientAPI -->|uses prompting| Prompter
    ClientAPI -->|uses utilities| OtherUtils
    ClientAPI -->|trains LoRA| ClientAPI
    ClientAPI -->|sends weight updates| Aggregator

    Aggregator -->|updates global weights| ModelStore
    Aggregator -->|triggers eval| Evaluation

    %% Inference
    subgraph "Inference"
        direction TB
        HFHub["Hugging Face Model Hub"]:::external
        GradioUI["Global Inference UI\n(GlobalModel_generate.py)"]:::server
        EndUser["End User"]:::external
    end

    HFHub -->|provides base model| ModelStore
    ModelStore -->|loads weights| GradioUI
    GradioUI -->|serves interface| EndUser

    %% Click Events
    click Orchestrator "https://github.com/jayzhang42/federatedgpt-shepherd/blob/main/main.py"
    click DataPrep "https://github.com/jayzhang42/federatedgpt-shepherd/blob/main/client_data_allocation.py"
    click RawData "https://github.com/jayzhang42/federatedgpt-shepherd/blob/main/new-databricks-dolly-15k.json"
    click ClientAPI "https://github.com/jayzhang42/federatedgpt-shepherd/blob/main/fed_utils/client.py"
    click Scheduler "https://github.com/jayzhang42/federatedgpt-shepherd/blob/main/fed_utils/client_participation_scheduling.py"
    click Aggregator "https://github.com/jayzhang42/federatedgpt-shepherd/blob/main/fed_utils/model_aggregation.py"
    click Evaluation "https://github.com/jayzhang42/federatedgpt-shepherd/blob/main/fed_utils/evaluation.py"
    click OtherUtils "https://github.com/jayzhang42/federatedgpt-shepherd/blob/main/fed_utils/other.py"
    click Templates "https://github.com/jayzhang42/federatedgpt-shepherd/tree/main/templates/"
    click Callbacks "https://github.com/jayzhang42/federatedgpt-shepherd/blob/main/utils/callbacks.py"
    click Prompter "https://github.com/jayzhang42/federatedgpt-shepherd/blob/main/utils/prompter.py"
    click GradioUI "https://github.com/jayzhang42/federatedgpt-shepherd/blob/main/GlobalModel_generated.py"
    click Dependencies "https://github.com/jayzhang42/federatedgpt-shepherd/blob/main/requirements.txt"
    click Assets "https://github.com/jayzhang42/federatedgpt-shepherd/tree/main/assets/"

    %% Styles
    classDef server fill:#D6EAF8,stroke:#85C1E9
    classDef client fill:#D5F5E3,stroke:#58D68D
    classDef external fill:#F2F3F4,stroke:#AAB7B8
    classDef datastore fill:#FCF3CF,stroke:#F7DC6F
    classDef process fill:#FADBD8,stroke:#EC7063
    classDef config fill:#E8DAEF,stroke:#AF7AC5
```
