# posthog/jeeves architecture

> Architecture diagram of the GitHub repository posthog/jeeves, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/posthog/jeeves
- Repository: https://github.com/posthog/jeeves

## Overview

Jeeves is a reasoning-enabled Jev-style classifier: training data is prepared and encoded, model and drafter components are trained, then exported weights are served through a Jev-compatible API. The README also documents CUDA/MPS and FP8 inference, mixed question types, and a Python SDK. The map follows documented capabilities and relationships visible in excerpts or the source index; several training and serving internals were not sampled, so their connections are omitted where unverified.

Main entry points: [`model/model.py`](https://github.com/posthog/jeeves/blob/master/model/model.py) (Qwen model), [`inference/engine.py`](https://github.com/posthog/jeeves/blob/master/inference/engine.py) (Inference engine), [`predictor.py`](https://github.com/posthog/jeeves/blob/master/predictor.py) (Training predictor), [`loader/dataloader.py`](https://github.com/posthog/jeeves/blob/master/loader/dataloader.py) (Tokenization and batches)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Data preparation"]
  node_prep["Dataset preparation<br/>[prep.py]"]
  node_contrast["Contrastive cases<br/>[contrastive.py]"]
  node_format["Record format<br/>[format.py]"]
  node_loader["Tokenization and batches<br/>[dataloader.py]"]
end

subgraph group_train["Training"]
  node_trainer["Model training<br/>[trainer.py]"]
  node_predictor["Training predictor<br/>[predictor.py]"]
  node_checkpoint["Checkpoint handling<br/>[checkpoint.py]"]
  node_distributed["Distributed execution<br/>[distributed.py]"]
  node_drafter_train["Drafter training<br/>[train.py]"]
  node_drafter_data["Drafter data<br/>[data.py]"]
end

subgraph group_model["Model and export"]
  node_drafter_view["Drafter model view<br/>[view.py]"]
  node_model_core["Qwen model<br/>[model.py]"]
  node_config["Model configuration<br/>[config.py]"]
  node_lora["LoRA adapters<br/>[lora.py]"]
  node_head["Pointer head<br/>[head.py]"]
  node_metal["Metal kernels<br/>[metal.py]"]
  node_export["Checkpoint export<br/>[export.py]"]
  node_fp8_export["FP8 checkpoint support<br/>[fp8_checkpoint.py]"]
end

subgraph group_serve["Inference and serving"]
  node_serve["HTTP serving<br/>[serve.py]"]
  node_api["Jev-compatible API<br/>[api.py]"]
  node_engine["Inference engine<br/>[engine.py]"]
  node_fp8["FP8 inference<br/>[fp8.py]"]
  node_fp8_metal["Metal FP8<br/>[fp8_metal.py]"]
end

subgraph group_tools["Client and evaluation"]
  node_sdk["Python SDK<br/>[__init__.py]"]
  node_bench["JevBench evaluation<br/>[jevbench.py]"]
end

node_user(("API client"))

node_prep -->|"uses generators"| node_contrast
node_prep -->|"builds records"| node_format
node_contrast -->|"formats examples"| node_format
node_loader -->|"reads records"| node_format
node_trainer -->|"loads batches"| node_loader
node_trainer -->|"uses predictor"| node_predictor
node_trainer -->|"saves checkpoints"| node_checkpoint
node_trainer -->|"uses distributed setup"| node_distributed
node_predictor -->|"encodes inputs"| node_loader
node_predictor -->|"runs model"| node_model_core
node_predictor -->|"uses head"| node_head
node_predictor -->|"loads weights"| node_checkpoint
node_drafter_train -->|"uses data"| node_drafter_data
node_drafter_train -->|"trains view"| node_drafter_view
node_drafter_train -->|"uses model"| node_model_core
node_drafter_train -->|"loads batches"| node_loader
node_model_core -->|"uses configuration"| node_config
node_model_core -->|"uses adapters"| node_lora
node_model_core -->|"dispatches kernels"| node_metal
node_export -->|"loads checkpoint"| node_checkpoint
node_export -->|"builds encoder"| node_loader
node_export -->|"fuses model"| node_model_core
node_export -->|"bundles head"| node_head
node_fp8_export -->|"reuses export"| node_export
node_fp8_export -->|"loads drafter view"| node_drafter_view
node_fp8_export -->|"loads model weights"| node_model_core
node_user -->|"sends request"| node_serve
node_serve -->|"registers API"| node_api
node_serve -->|"creates engine"| node_engine
node_api -->|"formats inputs"| node_format
node_api -.->|"dispatches inference"| node_engine
node_engine -->|"runs model"| node_model_core
node_engine -->|"uses drafter"| node_drafter_view
node_engine -->|"dispatches FP8"| node_fp8
node_engine -->|"dispatches Metal FP8"| node_fp8_metal
node_engine -->|"uses Metal kernels"| node_metal
node_sdk -.->|"provides client"| node_user
node_bench -->|"evaluates model"| node_predictor
node_bench -->|"loads data"| node_loader
node_bench -->|"uses records"| node_format

click node_prep "https://github.com/posthog/jeeves/blob/master/prep/prep.py"
click node_contrast "https://github.com/posthog/jeeves/blob/master/prep/contrastive.py"
click node_format "https://github.com/posthog/jeeves/blob/master/prep/format.py"
click node_loader "https://github.com/posthog/jeeves/blob/master/loader/dataloader.py"
click node_trainer "https://github.com/posthog/jeeves/blob/master/trainer.py"
click node_predictor "https://github.com/posthog/jeeves/blob/master/predictor.py"
click node_checkpoint "https://github.com/posthog/jeeves/blob/master/checkpoint.py"
click node_distributed "https://github.com/posthog/jeeves/blob/master/distributed.py"
click node_drafter_train "https://github.com/posthog/jeeves/blob/master/drafter/train.py"
click node_drafter_data "https://github.com/posthog/jeeves/blob/master/drafter/data.py"
click node_drafter_view "https://github.com/posthog/jeeves/blob/master/drafter/view.py"
click node_model_core "https://github.com/posthog/jeeves/blob/master/model/model.py"
click node_config "https://github.com/posthog/jeeves/blob/master/model/config.py"
click node_lora "https://github.com/posthog/jeeves/blob/master/model/lora.py"
click node_head "https://github.com/posthog/jeeves/blob/master/model/head.py"
click node_metal "https://github.com/posthog/jeeves/blob/master/model/metal.py"
click node_export "https://github.com/posthog/jeeves/blob/master/export.py"
click node_fp8_export "https://github.com/posthog/jeeves/blob/master/inference/fp8_checkpoint.py"
click node_serve "https://github.com/posthog/jeeves/blob/master/inference/serve.py"
click node_api "https://github.com/posthog/jeeves/blob/master/inference/api.py"
click node_engine "https://github.com/posthog/jeeves/blob/master/inference/engine.py"
click node_fp8 "https://github.com/posthog/jeeves/blob/master/inference/fp8.py"
click node_fp8_metal "https://github.com/posthog/jeeves/blob/master/inference/fp8_metal.py"
click node_sdk "https://github.com/posthog/jeeves/blob/master/sdk/jeeves_sdk/__init__.py"
click node_bench "https://github.com/posthog/jeeves/blob/master/jevbench.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_prep,node_contrast,node_format,node_loader,node_user toneBlue
class node_trainer,node_predictor,node_checkpoint,node_distributed,node_drafter_train,node_drafter_data toneAmber
class node_drafter_view,node_model_core,node_config,node_lora,node_head,node_metal,node_export,node_fp8_export toneMint
class node_serve,node_api,node_engine,node_fp8,node_fp8_metal toneRose
class node_sdk,node_bench toneIndigo
```

## Components

### Data preparation

- [Dataset preparation](https://github.com/posthog/jeeves/blob/master/prep/prep.py): `prep/prep.py`
- [Contrastive cases](https://github.com/posthog/jeeves/blob/master/prep/contrastive.py): `prep/contrastive.py`
- [Record format](https://github.com/posthog/jeeves/blob/master/prep/format.py): `prep/format.py`
- [Tokenization and batches](https://github.com/posthog/jeeves/blob/master/loader/dataloader.py): `loader/dataloader.py`

### Training

- [Model training](https://github.com/posthog/jeeves/blob/master/trainer.py): `trainer.py`
- [Training predictor](https://github.com/posthog/jeeves/blob/master/predictor.py): `predictor.py`
- [Checkpoint handling](https://github.com/posthog/jeeves/blob/master/checkpoint.py): `checkpoint.py`
- [Distributed execution](https://github.com/posthog/jeeves/blob/master/distributed.py): `distributed.py`
- [Drafter training](https://github.com/posthog/jeeves/blob/master/drafter/train.py): `drafter/train.py`
- [Drafter data](https://github.com/posthog/jeeves/blob/master/drafter/data.py): `drafter/data.py`

### Model and export

- [Drafter model view](https://github.com/posthog/jeeves/blob/master/drafter/view.py): `drafter/view.py`
- [Qwen model](https://github.com/posthog/jeeves/blob/master/model/model.py): `model/model.py`
- [Model configuration](https://github.com/posthog/jeeves/blob/master/model/config.py): `model/config.py`
- [LoRA adapters](https://github.com/posthog/jeeves/blob/master/model/lora.py): `model/lora.py`
- [Pointer head](https://github.com/posthog/jeeves/blob/master/model/head.py): `model/head.py`
- [Metal kernels](https://github.com/posthog/jeeves/blob/master/model/metal.py): `model/metal.py`
- [Checkpoint export](https://github.com/posthog/jeeves/blob/master/export.py): `export.py`
- [FP8 checkpoint support](https://github.com/posthog/jeeves/blob/master/inference/fp8_checkpoint.py): `inference/fp8_checkpoint.py`

### Inference and serving

- [HTTP serving](https://github.com/posthog/jeeves/blob/master/inference/serve.py): `inference/serve.py`
- [Jev-compatible API](https://github.com/posthog/jeeves/blob/master/inference/api.py): `inference/api.py`
- [Inference engine](https://github.com/posthog/jeeves/blob/master/inference/engine.py): `inference/engine.py`
- [FP8 inference](https://github.com/posthog/jeeves/blob/master/inference/fp8.py): `inference/fp8.py`
- [Metal FP8](https://github.com/posthog/jeeves/blob/master/inference/fp8_metal.py): `inference/fp8_metal.py`

### Client and evaluation

- [Python SDK](https://github.com/posthog/jeeves/blob/master/sdk/jeeves_sdk/__init__.py): `sdk/jeeves_sdk/__init__.py`
- [JevBench evaluation](https://github.com/posthog/jeeves/blob/master/jevbench.py): `jevbench.py`

### Other components

- **API client**

## Connections

- Dataset preparation → Contrastive cases: uses generators (evidence: `prep/prep.py`)
- Dataset preparation → Record format: builds records (evidence: `prep/prep.py`)
- Contrastive cases → Record format: formats examples (evidence: `prep/contrastive.py`)
- Tokenization and batches → Record format: reads records (evidence: `loader/dataloader.py`)
- Model training → Tokenization and batches: loads batches (evidence: `trainer.py`)
- Model training → Training predictor: uses predictor (evidence: `trainer.py`)
- Model training → Checkpoint handling: saves checkpoints (evidence: `trainer.py`)
- Model training → Distributed execution: uses distributed setup (evidence: `trainer.py`)
- Training predictor → Tokenization and batches: encodes inputs (evidence: `predictor.py`)
- Training predictor → Qwen model: runs model (evidence: `predictor.py`)
- Training predictor → Pointer head: uses head (evidence: `predictor.py`)
- Training predictor → Checkpoint handling: loads weights (evidence: `predictor.py`)
- Drafter training → Drafter data: uses data (evidence: `drafter/train.py`)
- Drafter training → Drafter model view: trains view (evidence: `drafter/train.py`)
- Drafter training → Qwen model: uses model (evidence: `drafter/train.py`)
- Drafter training → Tokenization and batches: loads batches (evidence: `drafter/train.py`)
- Qwen model → Model configuration: uses configuration (evidence: `model/model.py`)
- Qwen model → LoRA adapters: uses adapters (evidence: `model/model.py`)
- Qwen model → Metal kernels: dispatches kernels (evidence: `model/model.py`)
- Checkpoint export → Checkpoint handling: loads checkpoint (evidence: `export.py`)
- Checkpoint export → Tokenization and batches: builds encoder (evidence: `export.py`)
- Checkpoint export → Qwen model: fuses model (evidence: `export.py`)
- Checkpoint export → Pointer head: bundles head (evidence: `export.py`)
- FP8 checkpoint support → Checkpoint export: reuses export (evidence: `inference/fp8_checkpoint.py`)
- FP8 checkpoint support → Drafter model view: loads drafter view (evidence: `inference/fp8_checkpoint.py`)
- FP8 checkpoint support → Qwen model: loads model weights (evidence: `inference/fp8_checkpoint.py`)
- API client → HTTP serving: sends request (evidence: `README.md`)
- HTTP serving → Jev-compatible API: registers API (evidence: `inference/serve.py`)
- HTTP serving → Inference engine: creates engine (evidence: `inference/serve.py`)
- Jev-compatible API → Record format: formats inputs (evidence: `inference/api.py`)
- Jev-compatible API → Inference engine: dispatches inference
- Inference engine → Qwen model: runs model (evidence: `inference/engine.py`)
- Inference engine → Drafter model view: uses drafter (evidence: `inference/engine.py`)
- Inference engine → FP8 inference: dispatches FP8 (evidence: `inference/engine.py`)
- Inference engine → Metal FP8: dispatches Metal FP8 (evidence: `inference/engine.py`)
- Inference engine → Metal kernels: uses Metal kernels (evidence: `inference/engine.py`)
- Python SDK → API client: provides client
- JevBench evaluation → Training predictor: evaluates model (evidence: `jevbench.py`)
- JevBench evaluation → Tokenization and batches: loads data (evidence: `jevbench.py`)
- JevBench evaluation → Record format: uses records (evidence: `jevbench.py`)
