How gfratmct/llm-siciliano-vero works
This repository is a local Italian-language decoder-only LLM project. Its implemented workflow prepares Italian text, optionally trains a corpus-specific BPE tokenizer, trains and saves a dense or MoE model, then loads a checkpoint to generate text from a prompt. Training also evaluates held-out data and can report metrics to Weights & Biases. The README describes conversational and Sicilian fine-tuning and teacher alignment as future stages; no implementation for those stages appears in the tree. Some source excerpts are partial, so the graph omits uncertain internal calls.
Main entry points: train_transformer.py Training entrypoint · app.py Inference entrypoint · factory.py Model factory · tokenizer.py Tokenizer
Components 20
Data preparation
- Dataset reader
lib/dataset.py - BPE training
train_tokenizer.py - Tokenizer artifact
- Tokenizer
lib/tokenizer.py - Token datasets
lib/dataset.py
Model architecture
- Model factory
lib/models/factory.py - Dense transformer
lib/models/dense.py - MoE transformer
lib/models/moe.py - Shared model base
lib/models/base.py - Transformer components
lib/models/components.py - Model configuration
lib/config.py
Training
- Training entrypoint
train_transformer.py - Train and evaluate
lib/training.py - Model checkpoints
- Device utilities
lib/utils.py
Text generation
- Inference entrypoint
app.py - Generation helpers
lib/training.py
Other components
- Project operator
- Italian corpus files
- Weights & Biases
Connections 29
- Project operator → Italian corpus files: provides text
- Project operator → BPE training: starts training
- Italian corpus files → Dataset reader: reads files
- BPE training → Dataset reader: reads corpus
- BPE training → Tokenizer artifact: writes tokenizer
- Training entrypoint → Dataset reader: reads corpus
- Training entrypoint → Tokenizer: loads tokenizer
- Training entrypoint → Token datasets: prepares splits
- Token datasets → Tokenizer: encodes text
- Training entrypoint → Model factory: builds model
- Training entrypoint → Model configuration: configures model
- Model factory → Dense transformer: builds dense
- Model factory → MoE transformer: builds MoE
- Dense transformer → Shared model base: extends base
- MoE transformer → Shared model base: extends base
- Shared model base → Transformer components: uses components
- Training entrypoint → Train and evaluate: trains and evaluates
- Train and evaluate → Model checkpoints: updates weights
- Training entrypoint → Model checkpoints: saves checkpoints
- Training entrypoint → Weights & Biases: logs metrics
- Training entrypoint → Device utilities: selects device
- Project operator → Inference entrypoint: supplies prompt
- Inference entrypoint → Model configuration: loads config
- Inference entrypoint → Tokenizer: loads tokenizer
- Inference entrypoint → Model factory: loads checkpoint model
- Model factory → Model checkpoints: loads weights
- Inference entrypoint → Generation helpers: generates text
- Generation helpers → Tokenizer: encodes and decodes
- Inference entrypoint → Device utilities: selects device