How bytedance/dolphin works
Dolphin is a document-image parsing application with page-level, layout-only, and element-level entry points. Its primary workflow detects layout and reading order, crops and classifies elements, parses their content, then saves structured and Markdown outputs. PDF handling and Markdown conversion are visible in sampled utilities; photographed-page handling and other model capabilities are documented, but their complete implementation is not shown in the excerpts.
Main entry points: demo_page.py Page stages · demo_layout.py Layout stage · utils.py Save outputs · demo_element.py Element stage
Components 18
Parsing workflows
- Page parser
demo_page.py - Layout parser
demo_layout.py - Element parser
demo_element.py - Page stages
demo_page.py - Layout stage
demo_layout.py - Element stage
demo_element.py - Layout decoding
utils/utils.py - Crop and group
demo_page.py - Batch element parsing
demo_page.py
Model inference
- Page VLM inference
demo_page.py - Layout VLM inference
demo_layout.py - Element VLM inference
demo_element.py
Image and output utilities
- PDF rendering
utils/utils.py - Save outputs
utils/utils.py - Markdown conversion
utils/markdown_utils.py
Parsed results
- JSON and Markdown
Other components
- Document user
- Pretrained model
Connections 25
- Document user → Page parser: runs page parse
- Document user → Layout parser: runs layout parse
- Document user → Element parser: runs element parse
- Page parser → Page stages: processes documents
- Layout parser → Layout stage: processes documents
- Element parser → Element stage: processes images
- Page stages → PDF rendering: renders PDF pages
- Layout stage → PDF rendering: renders PDF pages
- Page stages → Page VLM inference: requests layout
- Page stages → Crop and group: dispatches layout
- Crop and group → Layout decoding: parses layout string
- Crop and group → Batch element parsing: dispatches element batches
- Batch element parsing → Page VLM inference: requests element text
- Layout stage → Layout VLM inference: requests layout
- Layout stage → Layout decoding: parses layout string
- Element stage → Element VLM inference: requests element text
- Page VLM inference → Pretrained model: loads model
- Layout VLM inference → Pretrained model: loads model
- Element VLM inference → Pretrained model: loads model
- Page stages → Save outputs: saves parsed results
- Layout stage → Save outputs: saves layout results
- Element stage → Save outputs: saves element results
- Save outputs → Markdown conversion: converts results
- Save outputs → JSON and Markdown: writes JSON and Markdown
- Page stages → JSON and Markdown: writes combined PDF results