How preferencemodel/karotte works
Karotte is a framework for building and running task-based RL environments. Its main workflow is to create an environment from a template, define and load tasks, build an execution image, run an agent in a sandbox, and inspect the resulting transcript and score. The graph includes documented agent, judge, runtime, and dashboard capabilities. Several connections are documented in the README; source excerpts are partial, so unverified internal call paths are omitted.
Main entry points: run.py Run CLI · run_helpers.py Task runner · agents Agent implementations · app.py Terminal dashboard
Components 20
Environment setup
- Environment templates
src/karotte/templates - Task definitions
src/karotte/task.py - Task discovery
src/karotte/load_tasks.py
Run orchestration
- Run CLI
src/karotte/cli/run.py - Run configuration
src/karotte/schemas/evaluation_run_config.py - Image builder
src/karotte/build.py - Container runtimes
src/karotte/container.py - Firecracker VM
src/karotte/firecracker/vm.py - Task runner
src/karotte/run_helpers.py
Agents and tools
- Agent implementations
src/karotte/agents - Model providers
src/karotte/providers.py - MCP tool servers
src/karotte/mcp_servers/http_mcp_server.py - Environment tools
src/karotte/tools
Evaluation results
- Judges and scoring
src/karotte/judges - Transcript events
src/karotte/schemas/transcript.py - Run artifacts
src/karotte/save_artifact.py
Run inspection
- Terminal dashboard
src/karotte/terminal/app.py - Run backend
src/karotte/backend_client.py
Other components
- Environment author
- Model API
Connections 21
- Environment author → Environment templates: creates from
README.md - Environment author → Task definitions: defines tasks
README.md - Run CLI → Run configuration: parses config
src/karotte/cli/run.py - Run CLI → Task discovery: loads tasks
src/karotte/cli/run.py - Task discovery → Task definitions: discovers classes
src/karotte/load_tasks.py - Run CLI → Image builder: builds image
README.md - Run CLI → Container runtimes: runs in runtime
README.md - Container runtimes → Firecracker VM: can use
README.md - Run CLI → Task runner: starts task run
src/karotte/cli/run.py - Task runner → Agent implementations: runs agent
- Agent implementations → Model providers: requests completions
src/karotte/agents/builtin_source.py - Model providers → Model API: calls API
- Agent implementations → MCP tool servers: uses tools server
src/karotte/agents/grok_build/agent.py - MCP tool servers → Environment tools: exposes tools
- Task runner → Judges and scoring: evaluates run
- Task runner → Transcript events: records events
- Agent implementations → Run artifacts: saves native logs
src/karotte/agents/grok_build/agent.py - Run CLI → Terminal dashboard: opens run UI
src/karotte/cli/run.py - Terminal dashboard → Transcript events: displays events
src/karotte/terminal/app.py - Environment author → Terminal dashboard: inspects results
README.md - Run backend → Transcript events: appends events
src/karotte/backend_client.py