How openbmb/minicpm-o-demo works
This repository is a MiniCPM-o 4.5 demo service for turn-based multimodal chat and audio/video full-duplex conversations. The main path is browser interaction through the Gateway and a GPU Worker to a runtime session, backend inference, and streamed results. The map also shows the model-processing layer and the Gateway’s documented queue/session-recording role. README statements establish the service topology; SOURCE INDEX references and excerpts support several internal links. Unsampled implementations and their finer-grained connections are not inferred.
Main entry points: omni-app.js Omni duplex UI · worker.py GPU worker · gateway.py Gateway · unified.py Unified processor
Components 19
User interfaces
- Mobile app
frontend/mobile/src/App.tsx - Omni duplex UI
static/omni/omni-app.js - Audio mixer
static/duplex/lib/mixer-controller.js - Video recording
static/duplex/lib/session-video-recorder.js
Request service
- Gateway
gateway.py - Worker pool
gateway_modules/worker_pool.py - Session recording
gateway_modules/session_recording.py
Inference runtime
- GPU worker
worker.py - Worker state
worker_state.py - Runtime session
runtime/session.py - Backend client
runtime/backend_client.py - Backend server
py_backend/server.py
Model processing
- Backend factory
core/processors/backend_factory.py - Mode capabilities
core/capabilities.py - Unified processor
core/processors/unified.py - Omni model
MiniCPMO45/modeling_minicpmo_unified.py - Vision encoder
MiniCPMO45/modeling_navit_siglip.py - Media processing
MiniCPMO45/processing_minicpmo.py
Other components
- User
Connections 19
- User → Mobile app: uses
README.md - User → Omni duplex UI: uses
README.md - Mobile app → Gateway: sends requests
README.md - Omni duplex UI → Gateway: sends requests
README.md - Omni duplex UI → Audio mixer: uses
static/omni/omni-app.js - Omni duplex UI → Video recording: uses
static/omni/omni-app.js - Gateway → Worker pool: routes and queues
README.md - Gateway → Session recording: records sessions
README.md - Worker pool → GPU worker: dispatches requests
README.md - GPU worker → Worker state: tracks state
worker.py - GPU worker → Runtime session: opens sessions
worker.py - Runtime session → Backend client: uses
runtime/session.py - GPU worker → Backend server: forwards protocol
README.md - Backend server → Backend factory: creates backend
py_backend/server.py - Backend factory → Unified processor: selects backend
core/processors/backend_factory.py - Unified processor → Mode capabilities: uses mode definitions
core/processors/unified.py - Unified processor → Omni model: runs inference
core/processors/unified.py - Omni model → Vision encoder: uses
MiniCPMO45/modeling_minicpmo_unified.py - Omni model → Media processing: uses
MiniCPMO45/modeling_minicpmo_unified.py