How bobbibao/ocsd-thesis works
OCSD is a sketch-and-text-guided scene image generation research system: it turns object descriptions and a scene sketch into an image intended to preserve object identity, count, layout, and relationships. The interactive path runs from user input through sketch parsing and OCSD preparation/generation to a consistency check and displayed image. The broader experiment path builds benchmark scenes, runs competing generation methods, evaluates images, and reports comparisons. The README documents additional baselines, ablations, and quality metrics; unsampled implementation details are shown only where supported by that documentation.
Main entry points: metrics.py Image metrics · method.py OCSD preparation and generation · stages.py Experiment stages · app.py Gradio interface
Components 18
Scene input
- Gradio interface
ocsd/app.py - Sketch and prompt parsing
ocsd/sketch.py - Benchmark scenes
ocsd/data.py
Generation
- OCSD preparation and generation
ocsd/method.py - Baseline methods
ocsd/methods.py - Diffusion engine
ocsd/engine.py - Attention control
ocsd/attention.py - Vision models
ocsd/vision.py - Generation configuration
ocsd/config.py
Evaluation
- Object consistency matching
ocsd/matching.py - Image metrics
ocsd/metrics.py - Statistical reports
ocsd/report.py
Experiments
- Experiment runner
ocsd/runner.py - Generated images and logs
ocsd/runner.py - Experiment stages
ocsd/stages.py - Experiment result files
ocsd/stages.py
Other components
- Sketch and text user
- QuickDraw and COCO
Connections 28
- Sketch and text user → Gradio interface: provides input
- Gradio interface → Sketch and prompt parsing: parses sketch
- Gradio interface → OCSD preparation and generation: prepares and generates
- Gradio interface → Vision models: checks detections
- Gradio interface → Object consistency matching: checks consistency
- QuickDraw and COCO → Benchmark scenes: provides samples
- Experiment stages → Benchmark scenes: builds benchmarks
- Experiment stages → Experiment runner: runs experiments
- Experiment stages → Image metrics: evaluates images
- Experiment stages → Statistical reports: builds reports
- Experiment runner → Benchmark scenes: loads scenes
- Experiment runner → OCSD preparation and generation: runs OCSD
- Experiment runner → Baseline methods: runs baselines
- Experiment runner → Generated images and logs: writes images and logs
- OCSD preparation and generation → Sketch and prompt parsing: uses scene prompts
- OCSD preparation and generation → Vision models: selects and checks objects
- OCSD preparation and generation → Diffusion engine: denoises and composes
- OCSD preparation and generation → Generation configuration: uses settings
- Diffusion engine → Attention control: uses processors
- Baseline methods → Diffusion engine: uses diffusion engine
- Image metrics → Generated images and logs: reads generated images
- Image metrics → Benchmark scenes: loads references
- Image metrics → Vision models: detects and scores
- Image metrics → Object consistency matching: computes consistency
- Image metrics → Experiment result files: writes per-image metrics
- Statistical reports → Experiment result files: reads metrics
- Statistical reports → Experiment result files: writes summaries and tables
- Experiment stages → Experiment result files: records experiment outputs