How jayzhang42/federatedgpt-shepherd works
Project type and purpose
• Shepherd is a full-stack research framework for federated instruction tuning of large language models (LLMs).
• It orchestrates multiple simulated clients that each perform local fine-tuning (using PEFT/LoRA) on private instruction datasets, and a central aggregator that merges their updates into a global model.
• It also supports data preparation, global inference (via Gradio), and evaluation.
File structure
• Top-level scripts and files:
• client_data_allocation.py – splits/generates per-client datasets
• main.py – entry point for federated fine-tuning (server orchestration)
• GlobalModel_generate.py – global model inference via Gradio
• Directories:
• fed_utils/ – core FL primitives
– client.py – client API (GeneralClient class)
– client_participation_scheduling.py – selects a subset of clients each round
– model_aggregation.py – federated averaging/aggregation logic
– evaluation.py – evaluation metrics, logging
– other.py – utility routines
• templates/ – JSON templates for instruction-fine-tuning
• utils/ – shared helpers for callbacks and prompt construction
• Configuration files:
• requirements.txt – dependencies (Hugging Face transformers, PEFT, bitsandbytes, Gradio)
• new-databricks-dolly-15k.json – raw data source for splitting
• assets/ – images for README and paper illustrations
README for additional insights
• Describes “FedIT” algorithm: federated instruction tuning using LoRA on clients + HF PEFT + bitsandbytes.
• Workflow: data preparation → federated rounds → global inference.
• Highlights key hyperparameters (num_clients, rounds, LoRA ranks, scheduler fraction).
• Mentions integration with Hugging Face Model Hub and Gradio.
Essential details
• Utils callbacks/prompter feeding into local training.
• Templates directory as configuration inputs to clients.
• Gradio inference flow as a distinct final stage.
• LoRA adapter flow within the local trainer.