How yuxuanjiang123/lerobotdatalabel works
LerobotDataLabel combines automated LeRobot episode labeling, segmentation and quality filtering with a Flask-based manual annotation application. The automated workflow configures dataset paths, generates visual inputs, uses a selected VLM backend to label or classify episodes, then materializes segments or applies quality gates. Manual users browse and annotate episodes through the web app, which reads dataset/video data and persists annotations. README capabilities are included, but several referenced pipeline scripts and configuration directories are absent from the supplied tree; their internal wiring is therefore not expanded.
Main entry points: app.py Annotation web app · label_core.py Labeling core · classify_microwave_phase.py Phase classifier · boundary_pipeline.py Boundary detection
Components 26
VLM Labeling
- Label entrypoints
scripts/label_episode_fixed_subtasks_vllm.py - Labeling core
scripts/label_core.py - DashScope API
- Local Qwen VL
microwave_split/qwen3vl_inference.py - vLLM service
- Contact sheets
scripts/gen_contact_sheets.py - Phase classifier
scripts/classify_microwave_phase.py
Segmentation
- Frame index export
scripts/results_to_frames_jsonl.py - Phase slicing
scripts/slice_by_phase.py - Boundary detection
microwave_split/boundary_pipeline.py - Segment materialization
microwave_split/materialize.py - Subtask video slicing
scripts/segment_subtask_videos.py
Quality Screening
- Dead-time cleanup
scripts/clean_dead_segments.py - Action quality scoring
movement_detection_core/runner.py - HQ dataset filter
scripts/filter_hq_dataset.py - Semantic success gate
scripts/quantity_judge_core.py
Manual Annotation
- Annotation web app
manual_labeling/web/app.py - Annotator UI
manual_labeling/static/js/annotator.js - Dataset loader
manual_labeling/core/data_loader.py - Video handler
manual_labeling/core/video_handler.py - Annotation store
manual_labeling/core/annotation_store.py - Dataset manager
manual_labeling/core/dataset_manager.py - LeRobot v0.1 export
manual_labeling/core/v01_format_handler.py
Shared Data Services
- Path resolver
paths.py
Other components
- Dataset operator
- LeRobot dataset
Connections 39
- Dataset operator → Path resolver: sets paths
- Path resolver → Label entrypoints: configures inputs
- Dataset operator → Contact sheets: starts labeling
- Contact sheets → LeRobot dataset: reads videos
- Label entrypoints → Labeling core: dispatches mode
- Labeling core → DashScope API: calls API
- Labeling core → Local Qwen VL: runs inference
- Labeling core → vLLM service: calls service
- Labeling core → LeRobot dataset: reads episode inputs
- Labeling core → Subtask video slicing: produces boundaries
- Dataset operator → Phase classifier: starts classification
- Phase classifier → LeRobot dataset: reads episodes
- Phase classifier → DashScope API: requests phase
- Phase classifier → Frame index export: writes results
- Frame index export → Phase slicing: provides frame index
- Phase slicing → LeRobot dataset: reads and slices
- Boundary detection → Local Qwen VL: requests boundaries
- Boundary detection → LeRobot dataset: samples video frames
- Boundary detection → Segment materialization: returns manifest
- Dataset operator → Dead-time cleanup: starts cleanup
- Dead-time cleanup → LeRobot dataset: rewrites segments
- Dataset operator → Action quality scoring: starts scoring
- Action quality scoring → LeRobot dataset: reads action data
- Action quality scoring → HQ dataset filter: provides scores
- HQ dataset filter → LeRobot dataset: writes HQ subset
- Dataset operator → Semantic success gate: starts judgement
- Semantic success gate → vLLM service: requests judgement
- Semantic success gate → LeRobot dataset: reads episodes
- Dataset operator → Annotation web app: opens annotator
- Annotation web app → Annotator UI: serves UI
- Annotator UI → Annotation web app: calls APIs
- Annotation web app → Dataset loader: loads episode data
- Annotation web app → Video handler: serves video details
- Annotation web app → Annotation store: loads and saves
- Annotation web app → Dataset manager: manages dataset
- Dataset manager → LeRobot v0.1 export: syncs task labels
- Dataset loader → LeRobot dataset: reads dataset
- Video handler → LeRobot dataset: reads videos
- Annotation store → LeRobot dataset: persists labels