How thviet79/asm1-development-llm-model works
File structure
• Top-level Python modules:
• base_llama.py – base classes and shared utilities for the model
• llama.py – your transformer block implementation and full forward/inference pipeline
• rope.py – rotary positional embeddings
• tokenizer.py – tokenization (vocab load, encode/decode)
• classifier.py – a classification head wrapping the LLM’s outputs
• optimizer.py – simple SGD/Adam-like routines + weight updates
• config.py – hyperparameter definitions (layer count, hidden size, etc.)
• utils.py – shared helpers (logging, metrics)
• run_llama.py – CLI entry point for generation, zero-shot, and fine-tuning workflows
• sanity_check.py & tests – smoke tests for your components
• data/ – raw train/dev/test splits + label maps for two sentiment datasets
README insights
• You load pretrained TinyStories weights (stories42M.pt), then run three workflows:
a. Text generation
b. Zero-shot prompting for sentiment
c. Fine-tuning with the classification head
• README emphasizes gaining understanding of model internals by implementing core classes.
Detail
• Every Python module as its own component
• How run_llama.py orchestrates workflows by invoking tokenizer → model → classifier/LM head → optimizer