How renagao/jev-dataops works
JEV DataOps is a self-hosted data screening, model-training, and evaluation workbench, with a separate browser-only demo. Users upload JSONL/CSV through the workbench or run batch jobs with the CLI; the runtime screens and partitions records, reports decisions, trains on retained data, and evaluates against held-out records. The map also includes domain metrics and portable training-bundle export/launch capabilities documented in the README. Relationships for unsampled subsystems are limited to documented capabilities; their internal wiring is not inferred.
Main entry points: cli.py CLI · runner.py Run orchestration · server.py HTTP service · training_launch.py Bundle validation and launch
Components 19
User interfaces
- Browser workbench
jev_dataops/static/app.js - CLI
jev_dataops/cli.py - HTTP service
jev_dataops/server.py - Browser-only demo runtime
public_demo/runtime.js
Data pipeline
- Run orchestration
jev_dataops/runner.py - Run and dataset store
jev_dataops/store.py - Screen and partition
jev_dataops/screening.py - JEV decision client
jev_dataops/jev.py
Training and export
- Train and evaluate
jev_dataops/training.py - Training bundle export
jev_dataops/training_export.py - Bundle validation and launch
jev_dataops/training_launch.py - Distributed SFT
jev_dataops/training_distributed.py - verl rewards
jev_dataops/rewards.py
Domain metrics
- Metric commands
jev_dataops/cli.py - Metric evaluator
jev_dataops/domain_metrics.py - Domain metric packs
jev_dataops/metric_packs
Other components
- User
- JEV provider
- Configured base model
Connections 22
- User → Browser workbench: uses
- Browser workbench → HTTP service: calls API
- User → CLI: runs jobs
- CLI → HTTP service: starts service
- HTTP service → Run orchestration: dispatches runs
- HTTP service → Run and dataset store: reads and writes
- Run orchestration → Screen and partition: screens records
- Run orchestration → Train and evaluate: trains and evaluates
- Run orchestration → Run and dataset store: persists run data
- Screen and partition → JEV decision client: requests decisions
- JEV decision client → JEV provider: sends records
- Train and evaluate → Configured base model: loads model
- CLI → Screen and partition: screens batch
- CLI → Train and evaluate: trains batch
- User → Browser-only demo runtime: uses locally
- User → Metric commands: scores predictions
- Metric commands → Metric evaluator: evaluates metrics
- Metric commands → Domain metric packs: loads or copies
- User → Training bundle export: requests export
- User → Bundle validation and launch: requests launch
- Bundle validation and launch → Distributed SFT: dispatches SFT
- Bundle validation and launch → verl rewards: bundles rewards