# thudm/glm-4 architecture

> Architecture diagram of the GitHub repository thudm/glm-4, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/thudm/glm-4
- Repository: https://github.com/thudm/glm-4

## Overview

##### Overview and Project Purpose
- The repository is centered on the GLM-4/GLM-Z1 series of large language models. It includes not only the models themselves but also distinct modules for finetuning, inference, and demonstration (client-side) use.
- The README explains the evolution of several variants (e.g., GLM-4-32B-0414, GLM-Z1-Rumination-32B-0414) along with detailed instructions on model usage and prompt implementations, which suggests that the project serves as an end-to-end solution for experimenting with, fine-tuning, and deploying these large-scale language models.

##### Analyzing the File Structure
- Top-level directories include:
- .github: Contains issue and pull request templates.
- demo: Contains multiple demonstration examples including “composite_demo” and “intel_device_demo.” Notice that the composite demo includes both a browser/front-end (with TypeScript/JavaScript files and package configurations) and a set of back-end client Python scripts.
- finetune: Contains scripts, configuration files, and requirements for model finetuning. This module likely shows training and fine-tuning procedures.
- inference: Contains various scripts for running inference (including CLI demos, web demos, and batch processing demos) that likely serve as the deployment layer for the models.
- resources: Provides additional documentation (like WeChat instructions) that offer user or deployment guidance.
- Other files such as the pre-commit-config and pyproject.toml indicate standard development tooling.

##### Main Components
a. Model Core:
- The large language models (GLM-4/GLM-Z1 series) form the heart of the system. Representation should include multiple model variants (e.g., “Chat”, “Reasoning”, “Base”) to reflect scalability and different use-case adaptations.

b. Training (Finetune) Module:
- “finetune” directory as the component responsible for configuring, running, and managing model training and fine-tuning.
- Configuration files (e.g., JSON, YAML for LoRA, SFT, ds_zero settings) that steer the training process.

c. Inference Module:
- The “inference” directory should be represented as the service component that handles incoming requests (both CLI and web-based) and runs live model inference.
- Individual scripts that demonstrate single-request APIs versus batch or stress test scenarios.

d. Demo/Client Modules:
- Under the “demo” directory, there are two distinct demonstrations:
- Composite Demo:
» Browser Component: A front-end module (using Node.js/TypeScript) that likely provides a UI to interact with the LLM.
» Back-end Component: Python clients (and a set of tool utilities) that interact with various model service endpoints.
- Intel Device Demo:
» Separate demos that target device-specific inference (one using “itrex” and one using “openvino” for conversion and demos). This suggests specialized deployment paths for hardware acceleration or device-specific inference.

e. External Tools and Integration:
- The README and various scripts refer to integrations with external frameworks and libraries (vLLM, transformers, llama.cpp and external RL and context managers like YaRN).
- Although these aren’t part of the file tree directly, they should be shown as external dependencies via separate labeled nodes that your inference and training components rely on.

##### Interactions and Data Flow
- The Finetuning module produces or updates trained models that are subsequently loaded by the Inference module.
- The Inference module accepts incoming requests from various clients (demos, web UIs, CLI demos) and calls upon the model core for predictions.
- The browser-based composite demo and device-specific demos send requests to the backend (via APIs or direct function calls), representing a clear separation between front-end and back-end layers.
- Model Serving frameworks (e.g., vLLM, transformers) should be depicted connecting with the model core, showing how layer extensions and tools (e.g., YaRN configuration for long-context support) are integrated.

##### Architectural Patterns and Design Principles
- Component Separation: Clearly separate training, inference, and client/demo components.
- Modular Design: Indicate that each major directory (finetune, inference, demo) is logically independent but connected through well-defined interfaces.
- Extensibility: Highlight how external tools or APIs (agent tools and prompt templates outlined in the README) extend the capabilities of the core model.

##### Technology and Framework References
- Front-end: Node.js/TypeScript (in the composite demo browser folder).
- Back-end: Python scripts (clients, main application scripts within demo and inference folders).
- Configuration Files: YAML and JSON (for finetuning configurations, pre-commit hooks, tsconfig.json).
- External Libraries: vLLM, transformers, and integration modules like llama.cpp (referenced in the README).

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Training & Model Core Layer"
        A["Finetune Module"]:::module
        B["Model Core (GLM-4/GLM-Z1 Variants)"]:::module
        A -->|"ModelUpdate"| B
        H["External Dependencies (vLLM,transformers,YaRN,llama.cpp)"]:::external
        H -->|"Integrates"| B
    end

    subgraph "Inference & Deployment Layer"
        C["Inference Service"]:::service
        B -->|"PredictionCalls"| C
    end

    subgraph "Demo Clients"
        D["Composite Demo - Frontend Browser"]:::frontend
        E["Composite Demo - Backend Client"]:::backend
        F["Intel Device Demo - itrex"]:::device
        G["Intel Device Demo - openvino"]:::device
        C -->|"APICall"| D
        C -->|"APICall"| E
        C -->|"APICall"| F
        C -->|"APICall"| G
    end

    click A "https://github.com/thudm/glm-4/tree/main/finetune/"
    click C "https://github.com/thudm/glm-4/tree/main/inference/"
    click D "https://github.com/thudm/glm-4/tree/main/demo/composite_demo/browser/"
    click E "https://github.com/thudm/glm-4/tree/main/demo/composite_demo/src/"
    click F "https://github.com/thudm/glm-4/tree/main/demo/intel_device_demo/itrex/"
    click G "https://github.com/thudm/glm-4/tree/main/demo/intel_device_demo/openvino/"

    classDef module fill:#AED6F1,stroke:#1F618D,stroke-width:2px;
    classDef external fill:#F9E79F,stroke:#B9770E,stroke-width:2px;
    classDef service fill:#A9DFBF,stroke:#145A32,stroke-width:2px;
    classDef frontend fill:#F5B7B1,stroke:#922B21,stroke-width:2px;
    classDef backend fill:#E8DAEF,stroke:#6C3483,stroke-width:2px;
    classDef device fill:#D5F5E3,stroke:#196F3D,stroke-width:2px;
```
