# vincent-stragier/llm-text-classifier architecture

> Architecture diagram of the GitHub repository vincent-stragier/llm-text-classifier, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/vincent-stragier/llm-text-classifier
- Repository: https://github.com/vincent-stragier/llm-text-classifier

## Overview

This repository is a script-based LLM text-classification experiment. The main workflow downloads synthetic request datasets, splits them and renders model-specific prompts, then runs each prompt through a configured local LLM and stores class scores alongside ground truth. Model configurations and prompt templates parameterize preprocessing and inference; model weights can also be downloaded separately. The classifier evaluates candidate class-token continuations using logits, while shared utilities aggregate scores and map prompt paths to result paths. Edges follow the README workflow and shown calls; the dataset, model-config, and prompt-template contents were not sampled.

Main entry points: [`do_classification.py`](https://github.com/vincent-stragier/llm-text-classifier/blob/main/do_classification.py) (Classification runner), [`do_preprocess.py`](https://github.com/vincent-stragier/llm-text-classifier/blob/main/do_preprocess.py) (Prompt generation), [`download_datasets.py`](https://github.com/vincent-stragier/llm-text-classifier/blob/main/download_datasets.py) (Dataset downloader), [`classifiers.py`](https://github.com/vincent-stragier/llm-text-classifier/blob/main/classifiers.py) (LLM classifier)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Data acquisition"]
  node_datasets[("Synthetic datasets")]
  node_download_datasets["Dataset downloader"]
end

subgraph group_prep["Prompt preparation"]
  node_preprocess["Prompt generation<br/>[do_preprocess.py]"]
  node_templates["Prompt templates"]
  node_prompts["Generated prompts"]
end

subgraph group_infer["Classification"]
  node_classification["Classification runner"]
  node_classifier["LLM classifier<br/>[classifiers.py]"]
end

subgraph group_support["Configuration and outputs"]
  node_model_configs["Model configurations"]
  node_model_download["Model downloader"]
  node_weights[("LLM weights")]
  node_score_utils["Score aggregation<br/>[utils.py]"]
  node_results[("Classification results")]
  node_path_utils["Result path mapping<br/>[utils.py]"]
end

node_user(("Experimenter"))
node_dataset_host(("Dataset host"))
node_huggingface(("Hugging Face Hub"))
node_llama["llama.cpp runtime"]

node_user -->|"runs"| node_download_datasets
node_download_datasets -->|"downloads from"| node_dataset_host
node_download_datasets -->|"writes datasets"| node_datasets
node_user -->|"runs"| node_preprocess
node_preprocess -->|"loads and splits"| node_datasets
node_preprocess -->|"renders"| node_templates
node_preprocess -->|"loads"| node_model_configs
node_preprocess -->|"writes prompts"| node_prompts
node_user -.->|"runs optionally"| node_model_download
node_model_download -->|"loads"| node_model_configs
node_model_download -->|"downloads from"| node_huggingface
node_huggingface -.->|"provides weights"| node_weights
node_user -->|"runs"| node_classification
node_classification -->|"loads"| node_model_configs
node_classification -->|"reads prompts"| node_prompts
node_classification -->|"loads classes"| node_datasets
node_classification -->|"fetches model"| node_huggingface
node_classification -->|"calls classify"| node_classifier
node_classifier -->|"decodes tokens"| node_llama
node_classifier -->|"aggregates logits"| node_score_utils
node_classification -->|"maps result path"| node_path_utils
node_classification -->|"writes scores"| node_results

click node_download_datasets "https://github.com/vincent-stragier/llm-text-classifier/blob/main/download_datasets.py"
click node_preprocess "https://github.com/vincent-stragier/llm-text-classifier/blob/main/do_preprocess.py"
click node_templates "https://github.com/vincent-stragier/llm-text-classifier/tree/main/prompts"
click node_model_configs "https://github.com/vincent-stragier/llm-text-classifier/tree/main/models_configs"
click node_prompts "https://github.com/vincent-stragier/llm-text-classifier/tree/main/prompts"
click node_model_download "https://github.com/vincent-stragier/llm-text-classifier/blob/main/download_all_llms.py"
click node_classification "https://github.com/vincent-stragier/llm-text-classifier/blob/main/do_classification.py"
click node_classifier "https://github.com/vincent-stragier/llm-text-classifier/blob/main/classifiers.py"
click node_score_utils "https://github.com/vincent-stragier/llm-text-classifier/blob/main/utils.py"
click node_results "https://github.com/vincent-stragier/llm-text-classifier/tree/main/results"
click node_path_utils "https://github.com/vincent-stragier/llm-text-classifier/blob/main/utils.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_datasets,node_download_datasets toneBlue
class node_preprocess,node_templates,node_prompts toneAmber
class node_classification,node_classifier toneMint
class node_model_configs,node_model_download,node_weights,node_score_utils,node_results,node_path_utils toneRose
class node_user,node_dataset_host,node_huggingface,node_llama toneIndigo
```

## Components

### Data acquisition

- **Synthetic datasets**
- [Dataset downloader](https://github.com/vincent-stragier/llm-text-classifier/blob/main/download_datasets.py): `download_datasets.py`

### Prompt preparation

- [Prompt generation](https://github.com/vincent-stragier/llm-text-classifier/blob/main/do_preprocess.py): `do_preprocess.py`
- [Prompt templates](https://github.com/vincent-stragier/llm-text-classifier/tree/main/prompts): `prompts`
- [Generated prompts](https://github.com/vincent-stragier/llm-text-classifier/tree/main/prompts): `prompts`

### Classification

- [Classification runner](https://github.com/vincent-stragier/llm-text-classifier/blob/main/do_classification.py): `do_classification.py`
- [LLM classifier](https://github.com/vincent-stragier/llm-text-classifier/blob/main/classifiers.py): `classifiers.py`

### Configuration and outputs

- [Model configurations](https://github.com/vincent-stragier/llm-text-classifier/tree/main/models_configs): `models_configs`
- [Model downloader](https://github.com/vincent-stragier/llm-text-classifier/blob/main/download_all_llms.py): `download_all_llms.py`
- **LLM weights**
- [Score aggregation](https://github.com/vincent-stragier/llm-text-classifier/blob/main/utils.py): `utils.py`
- [Classification results](https://github.com/vincent-stragier/llm-text-classifier/tree/main/results): `results`
- [Result path mapping](https://github.com/vincent-stragier/llm-text-classifier/blob/main/utils.py): `utils.py`

### Other components

- **Experimenter**
- **Dataset host**
- **Hugging Face Hub**
- **llama.cpp runtime**

## Connections

- Experimenter → Dataset downloader: runs
- Dataset downloader → Dataset host: downloads from (evidence: `download_datasets.py`)
- Dataset downloader → Synthetic datasets: writes datasets (evidence: `download_datasets.py`)
- Experimenter → Prompt generation: runs
- Prompt generation → Synthetic datasets: loads and splits (evidence: `do_preprocess.py`)
- Prompt generation → Prompt templates: renders (evidence: `do_preprocess.py`)
- Prompt generation → Model configurations: loads (evidence: `do_preprocess.py`)
- Prompt generation → Generated prompts: writes prompts (evidence: `do_preprocess.py`)
- Experimenter → Model downloader: runs optionally (evidence: `download_all_llms.py`)
- Model downloader → Model configurations: loads (evidence: `download_all_llms.py`)
- Model downloader → Hugging Face Hub: downloads from (evidence: `download_all_llms.py`)
- Hugging Face Hub → LLM weights: provides weights
- Experimenter → Classification runner: runs
- Classification runner → Model configurations: loads (evidence: `do_classification.py`)
- Classification runner → Generated prompts: reads prompts (evidence: `do_classification.py`)
- Classification runner → Synthetic datasets: loads classes (evidence: `do_classification.py`)
- Classification runner → Hugging Face Hub: fetches model (evidence: `do_classification.py`)
- Classification runner → LLM classifier: calls classify (evidence: `do_classification.py`)
- LLM classifier → llama.cpp runtime: decodes tokens (evidence: `classifiers.py`)
- LLM classifier → Score aggregation: aggregates logits (evidence: `classifiers.py`)
- Classification runner → Result path mapping: maps result path (evidence: `do_classification.py`)
- Classification runner → Classification results: writes scores (evidence: `do_classification.py`)