# xiuzbl/tree-instruct architecture

> Architecture diagram of the GitHub repository xiuzbl/tree-instruct, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/xiuzbl/tree-instruct
- Repository: https://github.com/xiuzbl/tree-instruct

## Overview

##### Overall Project Components
a. Data Storage and Collection:
- A “DATA” component that stores the original instruction datasets and their metadata. Note the subdirectories (alpaca1k_original, alpaca3k_original, alpaca4k_original, and various tree datasets like tree1k_10nodes) that contain raw data files (JSON, parquet, JSONL).
b. Data Generation Pipeline:
- Scripts “generate_tree_instructions.py” and “generate_tree_responses.py”. These form the data transformation pipeline that takes the raw instructions and generates complex instructions using the semantic tree modifications.
c. Model Training Infrastructure:
- A Training component that uses the “ochat” modules. Under ochat/training_deepspeed, you have configuration files (e.g., deepspeed_config.json, mydeepspeed_config.json), dataset preparation via “parquet_dataset.py”, and the main training script (“train.py”). This part is based on advanced training setups likely built upon deepspeed to support LLaMA2-13B training.
d. Serving / Inference Layer:
- Another component is for serving/inference, located in “ochat/serving”. Here you have the “openai_api_server.py”, “openai_api_protocol.py”, and an “async_tokenizer.py” module that together support asynchronous queries to the trained model, implementing an OpenAI-like API interface.
e. Evaluation and Experimental Modules:
- The “ochat/evaluation” directory contains multiple scripts (such as run_eval.py, match_answer.py) and a rich collection of evaluation data (fs_cothub, zs, etc.) which shows that evaluation of the model’s performance is a key part of the workflow.
- The “ochat/experimental” directory includes notebooks and older dataset generation experiments. This can be drawn as an optional experimental sandbox that feeds into model refinements or additional research experiments.
f. Configuration and Model Modules:
- The “ochat/config” and “ochat/models” folders support configuration (including model parameters) and model implementation details (for example, the custom LLaMA variant in “unpadded_llama.py”).

##### Determining Relationships and Data Flow
- Lastly, include an inference/serving block (ochat/serving) that wraps the model into an OpenAI-style API interface, making it accessible for real-time queries.

##### Architectural Patterns and Technologies
- That the system uses a pipeline-like architecture: Data ingestion → Data processing/transformation → Model training → Evaluation → Inference.
- Use of deepspeed for scalable training, which is a key design decision.
- That the integration with OpenChat (and indirectly the LLaMA2-13B model) is a crucial component in both training and inference.
- That the evaluation component is modular and extensible, featuring various datasets for testing different facets (logical reasoning, arithmetic, etc.) of model performance.

##### Additional Details
- Incorporate configuration files if needed (like those found under ochat/config) that detail parameters for training and inference.
- Since this is research-oriented, you might also note the iterative nature of the pipeline where experimental outcomes feed back into data generation improvements.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Storage Subgraph
    subgraph "Data Storage (DATA)"
        DS1["alpaca1k_original"]:::data
        DS2["alpaca3k_original"]:::data
        DS3["alpaca4k_original"]:::data
        DS4["tree1k_10nodes"]:::data
        DS5["tree1k_3nodes"]:::data
        DS6["tree1k_6nodes"]:::data
        DS7["wizardlm"]:::data
    end

    %% Data Generation Pipeline Subgraph
    subgraph "Data Generation Pipeline"
        G1["generate_tree_instructions.py"]:::gen
        G2["generate_tree_responses.py"]:::gen
        G1 -->|"transforms"| G2
    end

    %% Model Training (DeepSpeed Pipeline) Subgraph
    subgraph "Model Training (DeepSpeed Pipeline)"
        T3["deepspeed_config.json"]:::train
        T4["mydeepspeed_config.json"]:::train
        T2["parquet_dataset.py"]:::train
        T1["train.py"]:::train
    end

    %% Evaluation Module Subgraph
    subgraph "Model Evaluation"
        E1["run_eval.py"]:::evaluation
        E2["match_answer.py"]:::evaluation
    end

    %% Serving / Inference Layer Subgraph
    subgraph "Inference API Server"
        S1["openai_api_server.py"]:::serving
        S2["openai_api_protocol.py"]:::serving
        S3["async_tokenizer.py"]:::serving
    end

    %% Experimental Module
    EX["Experimental Sandbox"]:::experimental

    %% Configuration and Model Modules Subgraph
    subgraph "Configuration & Model"
        C1["Configuration Module"]:::config
        C2["Model Module"]:::config
    end

    %% Data Flow Connections
    %% From Data Storage to Data Generation Pipeline (using DS1 as representative)
    DS1 ---|"raw-data"| G1

    %% From Data Generation Pipeline to Model Training (using G2 to T1)
    G2 ---|"transformed-data"| T1

    %% From Model Training to Inference API Server (using T1 to S1)
    T1 ---|"trained-model"| S1

    %% From Model Training to Model Evaluation (using T1 to E1)
    T1 ---|"evaluation-input"| E1

    %% Feedback from Evaluation to Data Generation Pipeline (using E1 to G1)
    E1 ---|"feedback"| G1

    %% Experimental module feeds into Model Training
    EX ---|"research-experiments"| T1

    %% Configuration and Model modules support both Training and Serving layers
    C1 ---|"configuration"| T1
    C2 ---|"model-details"| S1

    %% Click Events
    click DS1 "https://github.com/xiuzbl/tree-instruct/tree/main/DATA/alpaca1k_original"
    click DS2 "https://github.com/xiuzbl/tree-instruct/tree/main/DATA/alpaca3k_original"
    click DS3 "https://github.com/xiuzbl/tree-instruct/tree/main/DATA/alpaca4k_original"
    click DS4 "https://github.com/xiuzbl/tree-instruct/tree/main/DATA/tree1k_10nodes"
    click DS5 "https://github.com/xiuzbl/tree-instruct/tree/main/DATA/tree1k_3nodes"
    click DS6 "https://github.com/xiuzbl/tree-instruct/tree/main/DATA/tree1k_6nodes"
    click DS7 "https://github.com/xiuzbl/tree-instruct/tree/main/DATA/wizardlm"

    click G1 "https://github.com/xiuzbl/tree-instruct/blob/main/generate_tree_instructions.py"
    click G2 "https://github.com/xiuzbl/tree-instruct/blob/main/generate_tree_responses.py"

    click T1 "https://github.com/xiuzbl/tree-instruct/blob/main/ochat/training_deepspeed/train.py"
    click T2 "https://github.com/xiuzbl/tree-instruct/blob/main/ochat/training_deepspeed/parquet_dataset.py"
    click T3 "https://github.com/xiuzbl/tree-instruct/blob/main/ochat/training_deepspeed/deepspeed_config.json"
    click T4 "https://github.com/xiuzbl/tree-instruct/blob/main/ochat/training_deepspeed/mydeepspeed_config.json"

    click S1 "https://github.com/xiuzbl/tree-instruct/blob/main/ochat/serving/openai_api_server.py"
    click S2 "https://github.com/xiuzbl/tree-instruct/blob/main/ochat/serving/openai_api_protocol.py"
    click S3 "https://github.com/xiuzbl/tree-instruct/blob/main/ochat/serving/async_tokenizer.py"

    click E1 "https://github.com/xiuzbl/tree-instruct/blob/main/ochat/evaluation/run_eval.py"
    click E2 "https://github.com/xiuzbl/tree-instruct/blob/main/ochat/evaluation/match_answer.py"

    click EX "https://github.com/xiuzbl/tree-instruct/tree/main/ochat/experimental"

    click C1 "https://github.com/xiuzbl/tree-instruct/tree/main/ochat/config"
    click C2 "https://github.com/xiuzbl/tree-instruct/blob/main/ochat/models/unpadded_llama.py"

    %% Style Classes
    classDef data fill:#cce5ff,stroke:#004085,stroke-width:2px;
    classDef gen fill:#d1ecf1,stroke:#0c5460,stroke-width:2px;
    classDef train fill:#d4edda,stroke:#155724,stroke-width:2px;
    classDef evaluation fill:#e2d6f3,stroke:#5a3e8e,stroke-width:2px;
    classDef serving fill:#fff3cd,stroke:#856404,stroke-width:2px;
    classDef experimental fill:#f8d7da,stroke:#721c24,stroke-width:2px;
    classDef config fill:#d1d1e0,stroke:#333399,stroke-width:2px;
```
