# gfratmct/llm-siciliano-vero architecture

> Architecture diagram of the GitHub repository gfratmct/llm-siciliano-vero, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/gfratmct/llm-siciliano-vero
- Repository: https://github.com/gfratmct/llm-siciliano-vero

## Overview

This repository is a local Italian-language decoder-only LLM project. Its implemented workflow prepares Italian text, optionally trains a corpus-specific BPE tokenizer, trains and saves a dense or MoE model, then loads a checkpoint to generate text from a prompt. Training also evaluates held-out data and can report metrics to Weights & Biases. The README describes conversational and Sicilian fine-tuning and teacher alignment as future stages; no implementation for those stages appears in the tree. Some source excerpts are partial, so the graph omits uncertain internal calls.

Main entry points: [`train_transformer.py`](https://github.com/gfratmct/llm-siciliano-vero/blob/master/train_transformer.py) (Training entrypoint), [`app.py`](https://github.com/gfratmct/llm-siciliano-vero/blob/master/app.py) (Inference entrypoint), [`lib/models/factory.py`](https://github.com/gfratmct/llm-siciliano-vero/blob/master/lib/models/factory.py) (Model factory), [`lib/tokenizer.py`](https://github.com/gfratmct/llm-siciliano-vero/blob/master/lib/tokenizer.py) (Tokenizer)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Data preparation"]
  node_dataset_reader["Dataset reader<br/>[dataset.py]"]
  node_tokenizer_training["BPE training<br/>[train_tokenizer.py]"]
  node_tokenizer_artifact["Tokenizer artifact"]
  node_tokenizer["Tokenizer<br/>[tokenizer.py]"]
  node_dataset["Token datasets<br/>[dataset.py]"]
end

subgraph group_model["Model architecture"]
  node_factory["Model factory<br/>[factory.py]"]
  node_dense_model["Dense transformer<br/>[dense.py]"]
  node_moe_model["MoE transformer<br/>[moe.py]"]
  node_base_model["Shared model base<br/>[base.py]"]
  node_model_components["Transformer components<br/>[components.py]"]
  node_model_config["Model configuration<br/>[config.py]"]
end

subgraph group_training["Training"]
  node_train_cli["Training entrypoint"]
  node_training_loop["Train and evaluate<br/>[training.py]"]
  node_checkpoint["Model checkpoints"]
  node_device_utils["Device utilities<br/>[utils.py]"]
end

subgraph group_inference["Text generation"]
  node_inference_cli["Inference entrypoint<br/>[app.py]"]
  node_generation["Generation helpers<br/>[training.py]"]
end

node_operator(("Project operator"))
node_italian_corpus["Italian corpus files"]
node_wandb(("Weights &amp; Biases"))

node_operator -->|"provides text"| node_italian_corpus
node_operator -->|"starts training"| node_tokenizer_training
node_italian_corpus -->|"reads files"| node_dataset_reader
node_tokenizer_training -->|"reads corpus"| node_dataset_reader
node_tokenizer_training -->|"writes tokenizer"| node_tokenizer_artifact
node_train_cli -->|"reads corpus"| node_dataset_reader
node_train_cli -->|"loads tokenizer"| node_tokenizer
node_train_cli -->|"prepares splits"| node_dataset
node_dataset -->|"encodes text"| node_tokenizer
node_train_cli -->|"builds model"| node_factory
node_train_cli -->|"configures model"| node_model_config
node_factory -->|"builds dense"| node_dense_model
node_factory -->|"builds MoE"| node_moe_model
node_dense_model -->|"extends base"| node_base_model
node_moe_model -->|"extends base"| node_base_model
node_base_model -->|"uses components"| node_model_components
node_train_cli -->|"trains and evaluates"| node_training_loop
node_training_loop -->|"updates weights"| node_checkpoint
node_train_cli -->|"saves checkpoints"| node_checkpoint
node_train_cli -.->|"logs metrics"| node_wandb
node_train_cli -->|"selects device"| node_device_utils
node_operator -->|"supplies prompt"| node_inference_cli
node_inference_cli -->|"loads config"| node_model_config
node_inference_cli -->|"loads tokenizer"| node_tokenizer
node_inference_cli -->|"loads checkpoint model"| node_factory
node_factory -->|"loads weights"| node_checkpoint
node_inference_cli -->|"generates text"| node_generation
node_generation -->|"encodes and decodes"| node_tokenizer
node_inference_cli -->|"selects device"| node_device_utils

click node_dataset_reader "https://github.com/gfratmct/llm-siciliano-vero/blob/master/lib/dataset.py"
click node_tokenizer_training "https://github.com/gfratmct/llm-siciliano-vero/blob/master/train_tokenizer.py"
click node_tokenizer "https://github.com/gfratmct/llm-siciliano-vero/blob/master/lib/tokenizer.py"
click node_dataset "https://github.com/gfratmct/llm-siciliano-vero/blob/master/lib/dataset.py"
click node_train_cli "https://github.com/gfratmct/llm-siciliano-vero/blob/master/train_transformer.py"
click node_training_loop "https://github.com/gfratmct/llm-siciliano-vero/blob/master/lib/training.py"
click node_factory "https://github.com/gfratmct/llm-siciliano-vero/blob/master/lib/models/factory.py"
click node_dense_model "https://github.com/gfratmct/llm-siciliano-vero/blob/master/lib/models/dense.py"
click node_moe_model "https://github.com/gfratmct/llm-siciliano-vero/blob/master/lib/models/moe.py"
click node_base_model "https://github.com/gfratmct/llm-siciliano-vero/blob/master/lib/models/base.py"
click node_model_components "https://github.com/gfratmct/llm-siciliano-vero/blob/master/lib/models/components.py"
click node_model_config "https://github.com/gfratmct/llm-siciliano-vero/blob/master/lib/config.py"
click node_inference_cli "https://github.com/gfratmct/llm-siciliano-vero/blob/master/app.py"
click node_generation "https://github.com/gfratmct/llm-siciliano-vero/blob/master/lib/training.py"
click node_device_utils "https://github.com/gfratmct/llm-siciliano-vero/blob/master/lib/utils.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_dataset_reader,node_tokenizer_training,node_tokenizer_artifact,node_tokenizer,node_dataset toneBlue
class node_factory,node_dense_model,node_moe_model,node_base_model,node_model_components,node_model_config toneAmber
class node_train_cli,node_training_loop,node_checkpoint,node_device_utils toneMint
class node_inference_cli,node_generation toneRose
class node_operator,node_italian_corpus,node_wandb toneIndigo
```

## Components

### Data preparation

- [Dataset reader](https://github.com/gfratmct/llm-siciliano-vero/blob/master/lib/dataset.py): `lib/dataset.py`
- [BPE training](https://github.com/gfratmct/llm-siciliano-vero/blob/master/train_tokenizer.py): `train_tokenizer.py`
- **Tokenizer artifact**
- [Tokenizer](https://github.com/gfratmct/llm-siciliano-vero/blob/master/lib/tokenizer.py): `lib/tokenizer.py`
- [Token datasets](https://github.com/gfratmct/llm-siciliano-vero/blob/master/lib/dataset.py): `lib/dataset.py`

### Model architecture

- [Model factory](https://github.com/gfratmct/llm-siciliano-vero/blob/master/lib/models/factory.py): `lib/models/factory.py`
- [Dense transformer](https://github.com/gfratmct/llm-siciliano-vero/blob/master/lib/models/dense.py): `lib/models/dense.py`
- [MoE transformer](https://github.com/gfratmct/llm-siciliano-vero/blob/master/lib/models/moe.py): `lib/models/moe.py`
- [Shared model base](https://github.com/gfratmct/llm-siciliano-vero/blob/master/lib/models/base.py): `lib/models/base.py`
- [Transformer components](https://github.com/gfratmct/llm-siciliano-vero/blob/master/lib/models/components.py): `lib/models/components.py`
- [Model configuration](https://github.com/gfratmct/llm-siciliano-vero/blob/master/lib/config.py): `lib/config.py`

### Training

- [Training entrypoint](https://github.com/gfratmct/llm-siciliano-vero/blob/master/train_transformer.py): `train_transformer.py`
- [Train and evaluate](https://github.com/gfratmct/llm-siciliano-vero/blob/master/lib/training.py): `lib/training.py`
- **Model checkpoints**
- [Device utilities](https://github.com/gfratmct/llm-siciliano-vero/blob/master/lib/utils.py): `lib/utils.py`

### Text generation

- [Inference entrypoint](https://github.com/gfratmct/llm-siciliano-vero/blob/master/app.py): `app.py`
- [Generation helpers](https://github.com/gfratmct/llm-siciliano-vero/blob/master/lib/training.py): `lib/training.py`

### Other components

- **Project operator**
- **Italian corpus files**
- **Weights & Biases**

## Connections

- Project operator → Italian corpus files: provides text
- Project operator → BPE training: starts training
- Italian corpus files → Dataset reader: reads files
- BPE training → Dataset reader: reads corpus
- BPE training → Tokenizer artifact: writes tokenizer
- Training entrypoint → Dataset reader: reads corpus
- Training entrypoint → Tokenizer: loads tokenizer
- Training entrypoint → Token datasets: prepares splits
- Token datasets → Tokenizer: encodes text
- Training entrypoint → Model factory: builds model
- Training entrypoint → Model configuration: configures model
- Model factory → Dense transformer: builds dense
- Model factory → MoE transformer: builds MoE
- Dense transformer → Shared model base: extends base
- MoE transformer → Shared model base: extends base
- Shared model base → Transformer components: uses components
- Training entrypoint → Train and evaluate: trains and evaluates
- Train and evaluate → Model checkpoints: updates weights
- Training entrypoint → Model checkpoints: saves checkpoints
- Training entrypoint → Weights & Biases: logs metrics
- Training entrypoint → Device utilities: selects device
- Project operator → Inference entrypoint: supplies prompt
- Inference entrypoint → Model configuration: loads config
- Inference entrypoint → Tokenizer: loads tokenizer
- Inference entrypoint → Model factory: loads checkpoint model
- Model factory → Model checkpoints: loads weights
- Inference entrypoint → Generation helpers: generates text
- Generation helpers → Tokenizer: encodes and decodes
- Inference entrypoint → Device utilities: selects device