How zju3dv/easyvolcap works
Project Type & Purpose
• EasyVolcap is an open-source Python/PyTorch framework and CLI toolset for neural volumetric video research: dataset preprocessing, sampler/network/renderer pipelines, training, testing, GUI/WebSocket rendering.
Key Architectural Components
A. CLI & Config Layer
• evc-train, evc-test, evc-gui, evc-ws entrypoints
• Configuration manager (easyvolcap/engine/config.py + YAML/JSON handlers)
• Registry pattern (easyvolcap/engine/registry.py)
B. Data Layer
• Config files under configs/ (datasets, exps, models, specs)
• Dataset & Dataloader modules (easyvolcap/dataloaders & datasets)
• File I/O & handlers (easyvolcap/engine/file_client.py, handlers/*.py)
• Third-party data: images, intrinsics/extrinsics, point clouds under data/
C. Core Engine & Pipeline
• Pipeline stages: sampler → network → renderer
• Samplers: uniform, importance, Gaussian Splatting, custom (easyvolcap/models/samplers)
• Networks: multilevel, MLP, hash, NeUS, volumetric video networks (easyvolcap/models/networks)
• Renderers: volume, point, mesh renderers (easyvolcap/models/renderers)
• Supervisors & losses (easyvolcap/models/supervisors)
D. Runners & Visualizers
• Trainer & evaluator (easyvolcap/runners/volumetric_video_runner.py + evaluators)
• GUI viewer (OpenGL-based) (easyvolcap/runners/volumetric_video_viewer.py)
• WebSocket server + client (easyvolcap/runners/websocket_server.py + scripts/client.py)
• Recorders, schedulers, moderators (easyvolcap/runners)
E. Utilities & Extensions
• Math, CUDA, OpenGL, profiling helpers (easyvolcap/utils)
• Shader assets (easyvolcap/utils/shaders)
• External scripts (scripts/) for dataset conversion, fusion, Colmap integration
• Tests (tests/)
Interactions & Data Flow
• CLI → Config Parser → instantiate components via Registry
• File I/O → load dataset configs → Dataset → Dataloader → Batch
• Batch → Sampler → Network → Renderer → Batch.output.rgb_map
• Batch.output → Trainer/Test logic → Metrics/Checkpoints
• For GUI: Batch.output → Visualizer → OpenGL window
• For WebSocket: Visualizer frames → WebSocket server → remote client
Architectural Patterns & Principles
• Modular registry for pluggable samplers, networks, renderers
• Separation of concerns: config vs. engine vs. data vs. visualization
• Pipeline abstraction: sampler/network/renderer stages
• Extension via scripts and utilities