How open3dv/xema works
Project Type & Overview
• This is a full-featured hardware-software integrated system for industrial 3D imaging.
• It encompasses low-level firmware (including MCU code), camera calibration, image capturing & processing, and high-level user interfaces (both CLI and GUI) together with an SDK.
• The architecture supports industrial use cases such as bin picking, assembly, 3D inspection, and integration with deep learning processes for advanced computer vision tasks.
Main Components to Highlight
a. Hardware & Embedded Systems:
– Camera sensor and optics
– Industrial imaging hardware with precision calibration (as depicted in calibration and documentation folders)
– Microcontroller unit (MCU) code and associated startup routines (mcu/ folder)
– External interface hardware (e.g., power controller via firmware/power_controller.cpp)
b. Low-Level Firmware Layer:
– Firmware modules (in the firmware/ folder) that encompass image capture, camera interfacing (drivers for different camera types such as Basler, DH, Galaxy), encoding and GPU computing sections (see .cu and .cuh files for CUDA acceleration)
– Communication and network functions such as TCP socket programming (socket_tcp.cpp in firmware and sdk)
c. Calibration & Image Processing:
– Calibration algorithms (calibration/ folder) which are key for accurate 3D reconstruction
– Reconstruction, triangulation, and encoding functions (src/ and test/ directories) forming the computational backbone for 3D point-cloud generation
d. Software Development Kit (SDK) & Application Layers:
– SDK (sdk/ folder) providing API interfaces for external integration and control
– Command-line tool (cmd/ folder), examples (cpp_example, example), and test projects (functest/ and test/) that help validate functionality
– Graphical user interface (gui/ folder): Provided as a Qt-based or Windows-based GUI for calibration, camera capture and firmware updates
e. Auxiliary Tools & Documentation:
– Python and shell scripts in tools/ for generating gray codes, pattern images, and depth map conversions
– Detailed hardware design files and documentation (docs/ folder and README images) for structural chassis and PCB design
Relationships & Interactions
• The hardware layer interfaces with the MCU firmware (mcu/), which in turn manages low-level sensor control.
• The firmware layer processes real-time image capture and leverages CUDA for GPU-accelerated encoding, filtering, and reconstruction.
• Calibration algorithms and image processing modules (calibration/, src/, test/) are used to correct and reconstruct the 3D point cloud.
• The SDK (sdk/) exposes APIs that allow higher-level applications (CLI in cmd/, examples in cpp_example/ and example/, GUI in gui/) to control the camera, trigger image acquisition, and process images.
• Tools and scripts (tools/) assist in creating necessary calibration patterns and processing depth maps for analysis.
• A testing suite (functest/ and test/) ensures the integration between these modules is stable.
Architectural Patterns & Design Principles
– Layered architecture: Dividing hardware control, real-time processing, calibration/algorithms, and application layers (CLI/GUI/SDK).
– Modular design: Each folder represents a module (calibration, firmware, gui, sdk, etc.) with clear separation of concerns.
– Integration of GPU computing: Using CUDA (.cu files) for efficient image encoding and reconstruction.
– Use of common build systems: The presence of CMakeLists.txt and Visual Studio project files indicates standardized build and deployment pipelines.