# datawhalechina/llm-deploy architecture
> Architecture diagram of the GitHub repository datawhalechina/llm-deploy, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-27). It is an AI-made overview: check details against the source.
- Interactive diagram: https://gitdiagram.com/datawhalechina/llm-deploy
- Repository: https://github.com/datawhalechina/llm-deploy
## Overview
1. This repository is a documentation-first project, not an application runtime. Its primary purpose is to teach LLM inference and deployment through a chapterized course, with theory, notebooks, code samples, slides, and a static docs site.
2. The top-level architecture has three content layers:
- `docs/` for the published book/site content
- `PPT/` for slide decks used in teaching
- chapter-specific executable examples embedded under `docs/chapter*/code` or `docs/chapter*/jupyter`
3. The publishing boundary is the `docs/` tree. The presence of `docs/index.html`, `docs/.nojekyll`, and `_sidebar.md` indicates a static-site workflow, likely for GitHub Pages or a similar static host. Most chapter markdown files are intended to be navigated as a book rather than executed directly.
4. The repository is organized as a curriculum with 10 chapters. Each chapter is a thematic subsystem covering a different aspect of model deployment, such as quantization, distillation, pruning, low-rank methods, representations, runtime, frameworks, parallelism, concurrency, and memory.
5. Chapter content is split into two kinds of assets:
- narrative material in `chapterX_*.md`
- practical material in notebooks, code, or experiments directories
This keeps conceptual explanation separate from executable demos.
6. The most important hands-on lab is Chapter 2, “distillation.” It includes a self-contained example pipeline under `docs/chapter2/code/BabyLlama/` with notebooks for data cleaning/tokenization, teacher training, and distillation. Supporting Python modules such as `babylm_dataset.py` and `mrclean.py` provide dataset preparation and text cleaning.
7. Chapter 3 focuses on pruning and contains the most code-heavy subsystem. Under `docs/chapter3/jupyter/LLMPruner/` there is a mini-library with:
- dataset helpers
- LLaMA model wrappers
- pruning algorithms and scheduling
- utility and prompt tooling
- a vendored `torch_pruning` package
This looks like a research/prototype environment for structured model compression rather than a production package.
8. The pruning code shows two model boundary types:
- native LLaMA-style model code under `models/llama`
- Hugging Face-compatible model support under `models/hf_llama`
This suggests the chapter compares custom model internals with HF-based integration paths.
9. Chapter 5 includes a smaller but clear deployment-oriented experiment set. The `experiments/` folder contains `detect.py`, `export.py`, and pretrained YOLO artifacts (`yolov5s.pt`, `yolov5s.onnx`), indicating a practical export-and-inference workflow that demonstrates model conversion and runtime inference rather than LLM-specific code.
10. The repository also contains multiple notebook-driven experiment tracks under Chapter 2, such as `CoT`, `ICL`, and `InstructFollowing`. These represent different data/behavior transfer paradigms for LLM distillation and are isolated as separate instructional modules.
11. External dependencies are mostly local and notebook-based rather than service-based. The repo includes environment specs like `env.yml`, `requirement.txt`, and chapter-specific `requirements.txt`, which implies reproducible teaching environments rather than a long-running backend service architecture.
12. Assets are organized per chapter, with `images/` folders supporting the narrative and `models/` or downloadable scripts providing model weights or retrieval helpers. This means diagrams for the repo should treat model files as offline inputs to tutorial pipelines, not as deployed system components.
13. From an architectural viewpoint, the key data flow is: source theory → chapter markdown → notebooks/code demos → model artifacts/results → rendered static documentation. The code samples are subordinate to the learning content, and the site is the final delivery surface.
14. In short, the repository’s main boundary is between “educational content” and “executable demonstrations.” The documentation site is the product, while the notebooks and code directories are supporting laboratories for distillation, pruning, and export/inference workflows.
Main entry points: [`docs/chapter0/chapter0_1.md`](https://github.com/datawhalechina/llm-deploy/blob/main/docs/chapter0/chapter0_1.md) (Chapters), [`docs/chapter3/jupyter/LLMPruner`](https://github.com/datawhalechina/llm-deploy/tree/main/docs/chapter3/jupyter/LLMPruner) (LLMPruner), [`docs/index.html`](https://github.com/datawhalechina/llm-deploy/blob/main/docs/index.html) (Docs site), [`docs/chapter2/code/BabyLlama`](https://github.com/datawhalechina/llm-deploy/tree/main/docs/chapter2/code/BabyLlama) (BabyLlama)
## Diagram (Mermaid)
```mermaid
flowchart TD
subgraph group_publishing["Publishing"]
node_site_shell["Docs site
static-site
[index.html]"]
node_sidebar["Nav map
site-nav
[_sidebar.md]"]
node_chapter_md["Chapters
markdown curriculum
[chapter0_1.md]"]
end
subgraph group_labs["Hands-on Labs"]
node_chapter2_lab["BabyLlama
distillation lab"]
node_chapter2_tracks["Transfer tracks
notebook labs"]
node_chapter2_env["Env spec
reproducibility
[env.yml]"]
node_chapter3_lab["LLMPruner
pruning lab"]
node_tp_core["Torch pruning
pruning core"]
node_model_paths["Model wrappers
llama integration"]
node_pruner_api["Pruner API
pruning orchestration"]
node_chapter5_exp["Export demo
deployment experiment"]
end
subgraph group_assets["Assets"]
node_chapter_images["Chapter images
visual assets"]
node_slides["Slide decks
pptx teaching"]
node_chapter2_models[("Model inputs
offline artifacts")]
node_chapter5_artifacts[("YOLO artifacts
weights and runtime
[yolov5s.onnx]")]
end
node_root["llm-deploy"]
node_root -->|"publishes"| node_site_shell
node_site_shell -->|"navigates"| node_sidebar
node_site_shell -->|"renders"| node_chapter_md
node_chapter_md -.->|"uses"| node_chapter_images
node_root -->|"supports"| node_slides
node_chapter_md -->|"pairs with"| node_chapter2_lab
node_chapter_md -->|"pairs with"| node_chapter2_tracks
node_chapter2_lab -.->|"reproducible with"| node_chapter2_env
node_chapter2_lab -->|"consumes"| node_chapter2_models
node_chapter_md -->|"pairs with"| node_chapter3_lab
node_chapter3_lab -->|"depends on"| node_tp_core
node_chapter3_lab -->|"wraps"| node_model_paths
node_chapter3_lab -->|"orchestrates"| node_pruner_api
node_chapter5_exp -->|"uses"| node_chapter5_artifacts
node_chapter_md -->|"illustrates"| node_chapter5_exp
node_root -->|"organizes"| node_chapter_md
click node_site_shell "https://github.com/datawhalechina/llm-deploy/blob/main/docs/index.html"
click node_sidebar "https://github.com/datawhalechina/llm-deploy/blob/main/docs/_sidebar.md"
click node_chapter_md "https://github.com/datawhalechina/llm-deploy/blob/main/docs/chapter0/chapter0_1.md"
click node_chapter_images "https://github.com/datawhalechina/llm-deploy/tree/main/docs/chapter2/images"
click node_slides "https://github.com/datawhalechina/llm-deploy/tree/main/PPT"
click node_chapter2_lab "https://github.com/datawhalechina/llm-deploy/tree/main/docs/chapter2/code/BabyLlama"
click node_chapter2_tracks "https://github.com/datawhalechina/llm-deploy/tree/main/docs/chapter2/code/CoT"
click node_chapter2_env "https://github.com/datawhalechina/llm-deploy/blob/main/docs/chapter2/code/env.yml"
click node_chapter2_models "https://github.com/datawhalechina/llm-deploy/tree/main/docs/chapter2/models"
click node_chapter3_lab "https://github.com/datawhalechina/llm-deploy/tree/main/docs/chapter3/jupyter/LLMPruner"
click node_tp_core "https://github.com/datawhalechina/llm-deploy/tree/main/docs/chapter3/jupyter/LLMPruner/torch_pruning"
click node_model_paths "https://github.com/datawhalechina/llm-deploy/tree/main/docs/chapter3/jupyter/LLMPruner/models"
click node_pruner_api "https://github.com/datawhalechina/llm-deploy/tree/main/docs/chapter3/jupyter/LLMPruner/pruner"
click node_chapter5_exp "https://github.com/datawhalechina/llm-deploy/tree/main/docs/chapter5/experiments"
click node_chapter5_artifacts "https://github.com/datawhalechina/llm-deploy/blob/main/docs/chapter5/experiments/yolov5s.onnx"
classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_site_shell,node_sidebar,node_chapter_md toneBlue
class node_chapter2_lab,node_chapter2_tracks,node_chapter2_env,node_chapter3_lab,node_tp_core,node_model_paths,node_pruner_api,node_chapter5_exp toneAmber
class node_chapter_images,node_slides,node_chapter2_models,node_chapter5_artifacts toneMint
class node_root toneNeutral
```
## Components
### Publishing
Static docs site and chapter markdown that form the course delivery surface.
- [Docs site](https://github.com/datawhalechina/llm-deploy/blob/main/docs/index.html): static-site · Published book/site boundary for the curriculum. · `docs/index.html`
- [Nav map](https://github.com/datawhalechina/llm-deploy/blob/main/docs/_sidebar.md): site-nav · Table-of-contents and site bootstrap for chapter navigation. · `docs/_sidebar.md`
- [Chapters](https://github.com/datawhalechina/llm-deploy/blob/main/docs/chapter0/chapter0_1.md): markdown curriculum · Narrative chapter pages for the 10-part course. · `docs/chapter0/chapter0_1.md`
### Hands-on Labs
Executable notebooks, code samples, and experiments embedded in the curriculum.
- [BabyLlama](https://github.com/datawhalechina/llm-deploy/tree/main/docs/chapter2/code/BabyLlama): distillation lab · Self-contained distillation pipeline with cleaning, teacher training, and student distillation. · `docs/chapter2/code/BabyLlama`
- [Transfer tracks](https://github.com/datawhalechina/llm-deploy/tree/main/docs/chapter2/code/CoT): notebook labs · Separate distillation experiments for CoT, ICL, and instruction following. · `docs/chapter2/code/CoT`
- [Env spec](https://github.com/datawhalechina/llm-deploy/blob/main/docs/chapter2/code/env.yml): reproducibility · Chapter-specific environment definition for notebooks and samples. · `docs/chapter2/code/env.yml`
- [LLMPruner](https://github.com/datawhalechina/llm-deploy/tree/main/docs/chapter3/jupyter/LLMPruner): pruning lab · Code-heavy prototype for structured pruning of LLaMA-style models. · `docs/chapter3/jupyter/LLMPruner`
- [Torch pruning](https://github.com/datawhalechina/llm-deploy/tree/main/docs/chapter3/jupyter/LLMPruner/torch_pruning): pruning core · Vendored pruning algorithms, dependency tracking, and operator utilities. · `docs/chapter3/jupyter/LLMPruner/torch_pruning`
- [Model wrappers](https://github.com/datawhalechina/llm-deploy/tree/main/docs/chapter3/jupyter/LLMPruner/models): llama integration · Native LLaMA and Hugging Face-compatible model implementations. · `docs/chapter3/jupyter/LLMPruner/models`
- [Pruner API](https://github.com/datawhalechina/llm-deploy/tree/main/docs/chapter3/jupyter/LLMPruner/pruner): pruning orchestration · Chapter-specific pruning entry points for native and HF model paths. · `docs/chapter3/jupyter/LLMPruner/pruner`
- [Export demo](https://github.com/datawhalechina/llm-deploy/tree/main/docs/chapter5/experiments): deployment experiment · YOLO export and detection workflow showing model conversion and inference. · `docs/chapter5/experiments`
### Assets
Slide decks, images, and model artifacts used by the teaching material.
- [Chapter images](https://github.com/datawhalechina/llm-deploy/tree/main/docs/chapter2/images): visual assets · Illustrations and figures embedded in chapter pages. · `docs/chapter2/images`
- [Slide decks](https://github.com/datawhalechina/llm-deploy/tree/main/PPT): pptx teaching · Presentation materials that complement the written chapters. · `PPT`
- [Model inputs](https://github.com/datawhalechina/llm-deploy/tree/main/docs/chapter2/models): offline artifacts · Download helper and model assets used by the distillation exercises. · `docs/chapter2/models`
- [YOLO artifacts](https://github.com/datawhalechina/llm-deploy/blob/main/docs/chapter5/experiments/yolov5s.onnx): weights and runtime · Pretrained weights and ONNX export used by the deployment example. · `docs/chapter5/experiments/yolov5s.onnx`
### Other components
- **llm-deploy**: repo · Documentation-first repository for teaching LLM inference and deployment.
## Connections
- llm-deploy → Docs site: publishes · The repository’s main output is the static documentation site.
- Docs site → Nav map: navigates · Site navigation is driven by the chapter sidebar.
- Docs site → Chapters: renders · Chapter markdown pages are the primary published content.
- Chapters → Chapter images: uses · Narrative pages reference chapter-specific visual assets.
- llm-deploy → Slide decks: supports · Slide decks complement the written course content.
- Chapters → BabyLlama: pairs with · Chapter 2 theory is paired with executable distillation notebooks.
- Chapters → Transfer tracks: pairs with · Chapter 2 also branches into separate transfer-learning notebooks.
- BabyLlama → Env spec: reproducible with · The distillation lab is backed by a chapter-specific environment spec.
- BabyLlama → Model inputs: consumes · Tutorial notebooks rely on downloadable model inputs and helpers.
- Chapters → LLMPruner: pairs with · Chapter 3 theory is paired with a pruning codebase.
- LLMPruner → Torch pruning: depends on · The pruning notebooks build on vendored pruning algorithms.
- LLMPruner → Model wrappers: wraps · The lab provides both native and Hugging Face model boundaries.
- LLMPruner → Pruner API: orchestrates · Pruning entry points coordinate model adaptation and scheduling.
- Export demo → YOLO artifacts: uses · The export/inference demo runs on stored weights and ONNX output.
- Chapters → Export demo: illustrates · Chapter 5 includes a practical model export workflow.
- llm-deploy → Chapters: organizes · The repository is structured as a 10-chapter curriculum.