How opendatalab/mineru-vl-utils works
MinerU VL Utils is a Python client library for document vision-language inference. The principal workflow accepts an image, runs layout detection, prepares region images and prompts, recognizes block content through a selected inference backend, and post-processes the resulting structured blocks. The README documents seven deployment backends; the excerpts show the client’s post-processing and image preparation responsibilities, shared client abstractions, and several backend implementations. Some workflow stages and backend selection details are documented rather than visible in the excerpts, so those links cite the README.
Main entry points: mineru_client.py MinerU client · base_client.py VLM client contract · __init__.py Block post-processing · http_client.py HTTP backend
Components 23
Document workflow
- MinerU client
mineru_vl_utils/mineru_client.py - Layout detection
mineru_vl_utils/mineru_client.py - Region preparation
mineru_vl_utils/mineru_client.py - Content blocks
mineru_vl_utils/structs.py
Inference backends
- HTTP backend
mineru_vl_utils/vlm_client/http_client.py - Transformers backend
mineru_vl_utils/vlm_client/transformers_client.py - MLX backend
mineru_vl_utils/vlm_client/mlx_client.py - LMDeploy backend
mineru_vl_utils/vlm_client/lmdeploy_engine_client.py - vLLM backend
mineru_vl_utils/vlm_client/vllm_engine_client.py - Async vLLM backend
mineru_vl_utils/vlm_client/vllm_async_engine_client.py - llama.cpp backend
mineru_vl_utils/vlm_client/llama_cpp_engine_client.py
Output processing
- Block post-processing
mineru_vl_utils/post_process/__init__.py - Image analysis cleanup
mineru_vl_utils/post_process/image_analysis_postprocess.py - Table image handling
mineru_vl_utils/post_process/table_image_processor.py - Cross-page tables
mineru_vl_utils/post_process/cross_page_table.py - Equation cleanup
mineru_vl_utils/post_process/equation_big.py - Chart repetition cleanup
mineru_vl_utils/post_process/chart_invalid_repetition.py - Mermaid cleanup
mineru_vl_utils/post_process/mermaid_repetition.py
Shared contracts
- VLM client contract
mineru_vl_utils/vlm_client/base_client.py - Image utilities
mineru_vl_utils/vlm_client/utils.py - No-repeat sampling
mineru_vl_utils/logits_processor/vllm_v1_no_repeat_ngram.py
Other components
- Application caller
- OpenAI-compatible server
Connections 30
- Application caller → MinerU client: calls
README.md - MinerU client → Layout detection: detects layout
mineru_vl_utils/mineru_client.py - MinerU client → Region preparation: prepares regions
mineru_vl_utils/mineru_client.py - Region preparation → Table image handling: masks table images
mineru_vl_utils/mineru_client.py - MinerU client → HTTP backend: selects backend
README.md - MinerU client → Transformers backend: selects backend
README.md - MinerU client → MLX backend: selects backend
README.md - MinerU client → LMDeploy backend: selects backend
README.md - MinerU client → vLLM backend: selects backend
README.md - MinerU client → Async vLLM backend: selects backend
README.md - MinerU client → llama.cpp backend: selects backend
README.md - MinerU client → Block post-processing: processes blocks
mineru_vl_utils/mineru_client.py - Block post-processing → Content blocks: updates blocks
mineru_vl_utils/post_process/__init__.py - Block post-processing → Image analysis cleanup: analyzes images
mineru_vl_utils/post_process/__init__.py - Block post-processing → Table image handling: replaces tokens
mineru_vl_utils/post_process/__init__.py - Block post-processing → Equation cleanup: fixes equations
mineru_vl_utils/post_process/__init__.py - Image analysis cleanup → Chart repetition cleanup: cleans repetitions
mineru_vl_utils/post_process/image_analysis_postprocess.py - Image analysis cleanup → Mermaid cleanup: cleans diagrams
mineru_vl_utils/post_process/image_analysis_postprocess.py - MinerU client → Cross-page tables: handles page tables
mineru_vl_utils/mineru_client.py - HTTP backend → VLM client contract: implements contract
mineru_vl_utils/vlm_client/http_client.py - Transformers backend → VLM client contract: implements contract
mineru_vl_utils/vlm_client/base_client.py - MLX backend → VLM client contract: implements contract
mineru_vl_utils/vlm_client/base_client.py - LMDeploy backend → VLM client contract: implements contract
mineru_vl_utils/vlm_client/lmdeploy_engine_client.py - vLLM backend → VLM client contract: implements contract
mineru_vl_utils/vlm_client/vllm_engine_client.py - Async vLLM backend → VLM client contract: implements contract
mineru_vl_utils/vlm_client/vllm_async_engine_client.py - llama.cpp backend → VLM client contract: implements contract
mineru_vl_utils/vlm_client/llama_cpp_engine_client.py - HTTP backend → Image utilities: loads images
mineru_vl_utils/vlm_client/http_client.py - vLLM backend → No-repeat sampling: supports sampling
README.md - HTTP backend → OpenAI-compatible server: sends requests
README.md - MinerU client → Content blocks: returns results
README.md