How dbiswas55/vlmflow works
vlmflow runs config-defined, multi-step vision-language experiments: the task entry point resolves settings, prepares ScienceQA samples, sends each prompt step to a vlmhub model, parses intermediate outputs for later steps, then scores and saves each result while recording resumable progress. The graph separates task orchestration, dataset handling, workflow execution, and persistence. The README documents the model-provider options and configurable workflows; unsampled configuration and submodule internals are not expanded beyond those documented capabilities.
Main entry points: scienceqa_task.py ScienceQA task · runner.py Workflow runner · config.py Experiment config · scienceqa.py Sample preparation
Components 17
Task orchestration
- ScienceQA task
src/scienceqa_task.py - Experiment config
src/utils/config.py - Workflow definitions
configs/experiment.json
ScienceQA data
- Sample preparation
src/dataset/scienceqa.py - Local sample files
src/dataset/scienceqa.py - Prompt tag builder
src/dataset/scienceqa.py
Workflow engine
- Prompt templates
src/prompts/scienceqa - Workflow runner
src/engine/runner.py - vlmhub model
src/_libs/vlmhub - Response parser
src/engine/parser.py
Results and state
- Result scoring
src/scienceqa_task.py - JSON writer
src/engine/io_utils.py - Results and progress
src/scienceqa_task.py - Input debug dumps
src/engine/runner.py
Other components
- Experiment operator
- Hugging Face dataset
- Model providers
Connections 19
- Experiment operator → ScienceQA task: starts run
README.md - ScienceQA task → Experiment config: loads settings
src/scienceqa_task.py - Experiment config → Workflow definitions: reads config
src/utils/config.py - ScienceQA task → Sample preparation: prepares samples
src/scienceqa_task.py - Sample preparation → Local sample files: reads and writes
src/dataset/scienceqa.py - Sample preparation → Hugging Face dataset: downloads samples
src/dataset/scienceqa.py - ScienceQA task → Workflow runner: runs workflow
src/scienceqa_task.py - Workflow runner → Experiment config: loads steps
src/engine/runner.py - Experiment config → Prompt templates: resolves prompt paths
README.md - Workflow runner → Prompt tag builder: builds tag values
src/engine/runner.py - Workflow runner → vlmhub model: generates response
src/engine/runner.py - vlmhub model → Model providers: uses configured backend
- Workflow runner → Response parser: parses responses
src/engine/runner.py - Workflow runner → Input debug dumps: dumps model input
src/engine/runner.py - ScienceQA task → Result scoring: scores sample
src/scienceqa_task.py - Result scoring → JSON writer: writes JSON
src/scienceqa_task.py - ScienceQA task → Results and progress: resumes and records
src/scienceqa_task.py - Result scoring → Results and progress: saves result
src/scienceqa_task.py - ScienceQA task → JSON writer: writes progress
src/scienceqa_task.py