How yinsongxu/llm2jev works
LLM2Jev exposes Jev-style Choice, Score, and Noul decisions through a Python API or compatible HTTP service. The main workflow parses a request, turns each question into binary candidate-scoring prompts, scores them with a prefill-only backend, then assembles structured answers. The README also documents multimodal input, prefix reuse, and demos. The tree and excerpts establish the major modules, but do not expose every server or inference call; uncertain links are omitted.
Main entry points: __init__.py Inference API · base.py Backend interface · server.py Server CLI · request_parser.py Request parser
Components 27
Request modeling
- Request parser
src/llm2jev/inference/request_parser.py - Question types
src/llm2jev/core/questions.py - Jev request
src/llm2jev/core/request.py - Multimodal content
src/llm2jev/core/multimodal.py
Scoring and answers
- Inference API
src/llm2jev/inference/__init__.py - Binary questions
src/llm2jev/inference/binary.py - Prompt rendering
src/llm2jev/inference/prompt.py - Probability normalization
src/llm2jev/inference/normalization.py - Answer assembly
src/llm2jev/inference/assembler.py - Structured answers
src/llm2jev/core/answers.py
Serving and backends
- HTTP API
src/llm2jev/server/http_server.py - Server CLI
src/llm2jev/server/server.py - SGLang service
src/llm2jev/server/sglang_server.py - MLX service
src/llm2jev/server/mlx_server.py - Transformers service
src/llm2jev/server/transformers_server.py - Backend interface
src/llm2jev/backend/base.py - SGLang scorer
src/llm2jev/backend/sglang/backend.py - MLX scorer
src/llm2jev/backend/mlx/backend.py - MLX vision scoring
src/llm2jev/backend/mlx/vision.py - Transformers scorer
src/llm2jev/backend/transformers/backend.py - Image preparation
src/llm2jev/backend/transformers/image_inputs.py
Interactive demos
- Web demo UI
demos/web/app.js - Demo proxy
demos/web/server.py - Snake demo
demos/snake.py - Pick-place demo
demos/pick_place/mujoco_demo.py
Other components
- API consumer
- System One API
Connections 24
- API consumer → HTTP API: submits request
- HTTP API → Request parser: parses body
- Request parser → Question types: builds questions
- Request parser → Jev request: builds request
- Jev request → Inference API: provides input
- Inference API → Binary questions: compiles questions
- Binary questions → Prompt rendering: builds prompts
- Inference API → Backend interface: scores prompts
- Backend interface → SGLang scorer: implemented by
- Backend interface → MLX scorer: implemented by
- Backend interface → Transformers scorer: implemented by
- Inference API → Probability normalization: normalizes scores
- Inference API → Answer assembly: assembles response
- Answer assembly → Structured answers: creates answers
- Server CLI → SGLang service: dispatches
- Server CLI → MLX service: dispatches
- Server CLI → Transformers service: dispatches
- MLX scorer → MLX vision scoring: scores images
- Transformers scorer → Image preparation: prepares images
- Web demo UI → Demo proxy: calls demo API
- Demo proxy → System One API: proxies requests
- Snake demo → System One API: submits decisions
- Pick-place demo → System One API: requests actions
- System One API → HTTP API: serves requests