# yinsongxu/llm2jev architecture

> Architecture diagram of the GitHub repository yinsongxu/llm2jev, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/yinsongxu/llm2jev
- Repository: https://github.com/yinsongxu/llm2jev

## Overview

LLM2Jev exposes Jev-style Choice, Score, and Noul decisions through a Python API or compatible HTTP service. The main workflow parses a request, turns each question into binary candidate-scoring prompts, scores them with a prefill-only backend, then assembles structured answers. The README also documents multimodal input, prefix reuse, and demos. The tree and excerpts establish the major modules, but do not expose every server or inference call; uncertain links are omitted.

Main entry points: [`src/llm2jev/inference/__init__.py`](https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/inference/__init__.py) (Inference API), [`src/llm2jev/backend/base.py`](https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/backend/base.py) (Backend interface), [`src/llm2jev/server/server.py`](https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/server/server.py) (Server CLI), [`src/llm2jev/inference/request_parser.py`](https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/inference/request_parser.py) (Request parser)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_request["Request modeling"]
  node_parser["Request parser<br/>[request_parser.py]"]
  node_questions["Question types<br/>[questions.py]"]
  node_requestmodel["Jev request<br/>[request.py]"]
  node_multimodal["Multimodal content<br/>[multimodal.py]"]
end

subgraph group_inference["Scoring and answers"]
  node_engine["Inference API<br/>[__init__.py]"]
  node_binary["Binary questions<br/>[binary.py]"]
  node_prompt["Prompt rendering<br/>[prompt.py]"]
  node_scoring["Probability normalization<br/>[normalization.py]"]
  node_assembler["Answer assembly<br/>[assembler.py]"]
  node_answers["Structured answers<br/>[answers.py]"]
end

subgraph group_runtime["Serving and backends"]
  node_http["HTTP API<br/>[http_server.py]"]
  node_cli["Server CLI<br/>[server.py]"]
  node_sgserver["SGLang service<br/>[sglang_server.py]"]
  node_mlxserver["MLX service<br/>[mlx_server.py]"]
  node_tfserver["Transformers service"]
  node_backendbase["Backend interface<br/>[base.py]"]
  node_sglang["SGLang scorer<br/>[backend.py]"]
  node_mlx["MLX scorer<br/>[backend.py]"]
  node_mlxvision["MLX vision scoring<br/>[vision.py]"]
  node_transformers["Transformers scorer<br/>[backend.py]"]
  node_tfimages["Image preparation<br/>[image_inputs.py]"]
end

subgraph group_demos["Interactive demos"]
  node_web["Web demo UI<br/>[app.js]"]
  node_proxy["Demo proxy<br/>[server.py]"]
  node_snake["Snake demo<br/>[snake.py]"]
  node_robot["Pick-place demo<br/>[mujoco_demo.py]"]
end

node_user(("API consumer"))
node_jevapi["System One API"]

node_user -->|"submits request"| node_http
node_http -->|"parses body"| node_parser
node_parser -->|"builds questions"| node_questions
node_parser -->|"builds request"| node_requestmodel
node_requestmodel -->|"provides input"| node_engine
node_engine -->|"compiles questions"| node_binary
node_binary -->|"builds prompts"| node_prompt
node_engine -->|"scores prompts"| node_backendbase
node_backendbase -->|"implemented by"| node_sglang
node_backendbase -->|"implemented by"| node_mlx
node_backendbase -->|"implemented by"| node_transformers
node_engine -->|"normalizes scores"| node_scoring
node_engine -->|"assembles response"| node_assembler
node_assembler -->|"creates answers"| node_answers
node_cli -->|"dispatches"| node_sgserver
node_cli -->|"dispatches"| node_mlxserver
node_cli -->|"dispatches"| node_tfserver
node_mlx -->|"scores images"| node_mlxvision
node_transformers -->|"prepares images"| node_tfimages
node_web -->|"calls demo API"| node_proxy
node_proxy -->|"proxies requests"| node_jevapi
node_snake -->|"submits decisions"| node_jevapi
node_robot -->|"requests actions"| node_jevapi
node_jevapi -->|"serves requests"| node_http

click node_http "https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/server/http_server.py"
click node_cli "https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/server/server.py"
click node_sgserver "https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/server/sglang_server.py"
click node_mlxserver "https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/server/mlx_server.py"
click node_tfserver "https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/server/transformers_server.py"
click node_parser "https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/inference/request_parser.py"
click node_questions "https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/core/questions.py"
click node_requestmodel "https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/core/request.py"
click node_multimodal "https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/core/multimodal.py"
click node_engine "https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/inference/__init__.py"
click node_binary "https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/inference/binary.py"
click node_prompt "https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/inference/prompt.py"
click node_scoring "https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/inference/normalization.py"
click node_assembler "https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/inference/assembler.py"
click node_answers "https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/core/answers.py"
click node_backendbase "https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/backend/base.py"
click node_sglang "https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/backend/sglang/backend.py"
click node_mlx "https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/backend/mlx/backend.py"
click node_mlxvision "https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/backend/mlx/vision.py"
click node_transformers "https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/backend/transformers/backend.py"
click node_tfimages "https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/backend/transformers/image_inputs.py"
click node_web "https://github.com/yinsongxu/llm2jev/blob/main/demos/web/app.js"
click node_proxy "https://github.com/yinsongxu/llm2jev/blob/main/demos/web/server.py"
click node_snake "https://github.com/yinsongxu/llm2jev/blob/main/demos/snake.py"
click node_robot "https://github.com/yinsongxu/llm2jev/blob/main/demos/pick_place/mujoco_demo.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_parser,node_questions,node_requestmodel,node_multimodal toneBlue
class node_engine,node_binary,node_prompt,node_scoring,node_assembler,node_answers toneAmber
class node_http,node_cli,node_sgserver,node_mlxserver,node_tfserver,node_backendbase,node_sglang,node_mlx,node_mlxvision,node_transformers,node_tfimages,node_user,node_jevapi toneMint
class node_web,node_proxy,node_snake,node_robot toneRose
```

## Components

### Request modeling

- [Request parser](https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/inference/request_parser.py): `src/llm2jev/inference/request_parser.py`
- [Question types](https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/core/questions.py): `src/llm2jev/core/questions.py`
- [Jev request](https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/core/request.py): `src/llm2jev/core/request.py`
- [Multimodal content](https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/core/multimodal.py): `src/llm2jev/core/multimodal.py`

### Scoring and answers

- [Inference API](https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/inference/__init__.py): `src/llm2jev/inference/__init__.py`
- [Binary questions](https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/inference/binary.py): `src/llm2jev/inference/binary.py`
- [Prompt rendering](https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/inference/prompt.py): `src/llm2jev/inference/prompt.py`
- [Probability normalization](https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/inference/normalization.py): `src/llm2jev/inference/normalization.py`
- [Answer assembly](https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/inference/assembler.py): `src/llm2jev/inference/assembler.py`
- [Structured answers](https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/core/answers.py): `src/llm2jev/core/answers.py`

### Serving and backends

- [HTTP API](https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/server/http_server.py): `src/llm2jev/server/http_server.py`
- [Server CLI](https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/server/server.py): `src/llm2jev/server/server.py`
- [SGLang service](https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/server/sglang_server.py): `src/llm2jev/server/sglang_server.py`
- [MLX service](https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/server/mlx_server.py): `src/llm2jev/server/mlx_server.py`
- [Transformers service](https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/server/transformers_server.py): `src/llm2jev/server/transformers_server.py`
- [Backend interface](https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/backend/base.py): `src/llm2jev/backend/base.py`
- [SGLang scorer](https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/backend/sglang/backend.py): `src/llm2jev/backend/sglang/backend.py`
- [MLX scorer](https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/backend/mlx/backend.py): `src/llm2jev/backend/mlx/backend.py`
- [MLX vision scoring](https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/backend/mlx/vision.py): `src/llm2jev/backend/mlx/vision.py`
- [Transformers scorer](https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/backend/transformers/backend.py): `src/llm2jev/backend/transformers/backend.py`
- [Image preparation](https://github.com/yinsongxu/llm2jev/blob/main/src/llm2jev/backend/transformers/image_inputs.py): `src/llm2jev/backend/transformers/image_inputs.py`

### Interactive demos

- [Web demo UI](https://github.com/yinsongxu/llm2jev/blob/main/demos/web/app.js): `demos/web/app.js`
- [Demo proxy](https://github.com/yinsongxu/llm2jev/blob/main/demos/web/server.py): `demos/web/server.py`
- [Snake demo](https://github.com/yinsongxu/llm2jev/blob/main/demos/snake.py): `demos/snake.py`
- [Pick-place demo](https://github.com/yinsongxu/llm2jev/blob/main/demos/pick_place/mujoco_demo.py): `demos/pick_place/mujoco_demo.py`

### Other components

- **API consumer**
- **System One API**

## Connections

- API consumer → HTTP API: submits request
- HTTP API → Request parser: parses body
- Request parser → Question types: builds questions
- Request parser → Jev request: builds request
- Jev request → Inference API: provides input
- Inference API → Binary questions: compiles questions
- Binary questions → Prompt rendering: builds prompts
- Inference API → Backend interface: scores prompts
- Backend interface → SGLang scorer: implemented by
- Backend interface → MLX scorer: implemented by
- Backend interface → Transformers scorer: implemented by
- Inference API → Probability normalization: normalizes scores
- Inference API → Answer assembly: assembles response
- Answer assembly → Structured answers: creates answers
- Server CLI → SGLang service: dispatches
- Server CLI → MLX service: dispatches
- Server CLI → Transformers service: dispatches
- MLX scorer → MLX vision scoring: scores images
- Transformers scorer → Image preparation: prepares images
- Web demo UI → Demo proxy: calls demo API
- Demo proxy → System One API: proxies requests
- Snake demo → System One API: submits decisions
- Pick-place demo → System One API: requests actions
- System One API → HTTP API: serves requests