How xiuzbl/tree-instruct works
Overall Project Components
a. Data Storage and Collection:
• A “DATA” component that stores the original instruction datasets and their metadata. Note the subdirectories (alpaca1k_original, alpaca3k_original, alpaca4k_original, and various tree datasets like tree1k_10nodes) that contain raw data files (JSON, parquet, JSONL).
b. Data Generation Pipeline:
• Scripts “generate_tree_instructions.py” and “generate_tree_responses.py”. These form the data transformation pipeline that takes the raw instructions and generates complex instructions using the semantic tree modifications.
c. Model Training Infrastructure:
• A Training component that uses the “ochat” modules. Under ochat/training_deepspeed, you have configuration files (e.g., deepspeed_config.json, mydeepspeed_config.json), dataset preparation via “parquet_dataset.py”, and the main training script (“train.py”). This part is based on advanced training setups likely built upon deepspeed to support LLaMA2-13B training.
d. Serving / Inference Layer:
• Another component is for serving/inference, located in “ochat/serving”. Here you have the “openai_api_server.py”, “openai_api_protocol.py”, and an “async_tokenizer.py” module that together support asynchronous queries to the trained model, implementing an OpenAI-like API interface.
e. Evaluation and Experimental Modules:
• The “ochat/evaluation” directory contains multiple scripts (such as run_eval.py, match_answer.py) and a rich collection of evaluation data (fs_cothub, zs, etc.) which shows that evaluation of the model’s performance is a key part of the workflow.
• The “ochat/experimental” directory includes notebooks and older dataset generation experiments. This can be drawn as an optional experimental sandbox that feeds into model refinements or additional research experiments.
f. Configuration and Model Modules:
• The “ochat/config” and “ochat/models” folders support configuration (including model parameters) and model implementation details (for example, the custom LLaMA variant in “unpadded_llama.py”).
Determining Relationships and Data Flow
• Lastly, include an inference/serving block (ochat/serving) that wraps the model into an OpenAI-style API interface, making it accessible for real-time queries.
Architectural Patterns and Technologies
• That the system uses a pipeline-like architecture: Data ingestion → Data processing/transformation → Model training → Evaluation → Inference.
• Use of deepspeed for scalable training, which is a key design decision.
• That the integration with OpenChat (and indirectly the LLaMA2-13B model) is a crucial component in both training and inference.
• That the evaluation component is modular and extensible, featuring various datasets for testing different facets (logical reasoning, arithmetic, etc.) of model performance.
Additional Details
• Incorporate configuration files if needed (like those found under ochat/config) that detail parameters for training and inference.
• Since this is research-oriented, you might also note the iterative nature of the pipeline where experimental outcomes feed back into data generation improvements.