# embarktrucks/embag architecture

> Architecture diagram of the GitHub repository embarktrucks/embag, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/embarktrucks/embag
- Repository: https://github.com/embarktrucks/embag

## Overview

##### Project Type and Purpose
- Project Type: A dual API library for reading ROS bag files – it’s a native C++ library (with a CLI demo in embag_echo) and a Python API (packaged via pip and also built as a Bazel target).
- Purpose: To quickly read ROS bag files (even without loading full message definitions) with good performance in both C++ and Python. It also includes components for decompression (e.g., lz4, bz2) and supports cross-platform building using Bazel.

##### File Structure
- Top-Level Build & Configuration: Files such as .bazelproject, WORKSPACE, .bazelversion, and BUILD files in various directories indicate Bazel is used as the primary build system.
- Core Library Code: The lib directory holds the main C++ implementation (embag.cc, message parsing logic, view classes, etc.) and associated headers. This is the “heart” of the reader.
- Auxiliary Tools: The embag_echo directory contains a demo binary using the core library to demonstrate functionality.
- Python API: There are two directories (pip_package and python) which contain code for the Python interface, setup scripts, and tests.
- Compression Components: Directories such as bz2 and lz4 indicate modules for handling possibly compressed data.
- Testing: The test directory includes both C++ tests (.cc files) and Python tests, along with sample bag files.
- Continuous Integration and Deployment configurations in .github and Dockerfile confirm automated build and deployment pipelines.

##### README for Additional Insights
- The README explains usage scenarios for both the C++ and Python APIs.
- It shows how to build different targets (e.g., embag_echo demo, shared libraries, Debian packages, Python wheels) using Bazel.
- Performance benchmarks and usage examples are provided, highlighting performance differences and integration points.

a. Main Components to Include:
- User Interfaces:
- Command-line interface (embag_echo demo) for C++ usage.
- Python API (pip_package and python modules) accessible via pip installation.
- Core Library:
- C++ core library (in lib) which implements parsing logic, message handling, and decompression.
- Compression Modules:
- Distinct modules for handling compressed data: bz2 and lz4.
- Build and Test Infrastructure:
- Bazel build system orchestrates building of C++ binaries, shared objects, Debian packages, wheel packages and running tests.
- Testing module (test) that validates both C++ and Python interfaces using sample ROS bag files.
- External Integration:
- Integration with ROS bag format and compatibility with ROS message structures.
- Deployment pipelines via Dockerfile and GitHub Actions for CI/CD.

b. Relationships and Interactions:
- The core library (lib) is central to Embag – both the embag_echo demo and the Python wrappers use it.
- The Python API wraps around the C++ library (possibly using extension modules or bindings) and is packaged separately.
- Compression components (bz2 and lz4) interact with the core library to decompress bag file content.
- The Bazel build system orchestrates the build process for all these components, ensuring that C++ binaries, shared libraries, and Python wheels are produced in a consistent manner.
- Test suite validates interactions between components by reading bag files and verifying message formats.

c. Important Architectural Patterns and Design Principles:
- Separation of Concerns: The design isolates core parsing logic (in lib) from client-specific interfaces (CLI demo, Python wrappers) and compression logic.
- Modularization: Separate directories for C++ implementation, Python API layers, demo binary, and tests promote maintainability and scalability.
- Build Automation and Consistency: Using Bazel for all build targets ensures reproducibility and cross-language integration.
- Performance Focus: Emphasis on efficient message processing and minimal dependencies indicates a strong focus on performance and low overhead.

d. Technologies, Frameworks, and Libraries:
- Bazel: Used as the build system (files like WORKSPACE, BUILD, .bazelproject).
- C++: The core functionality is implemented in C++ (lib, embag_echo).
- Python: Python API and packaging (pip_package, python directories).
- Compression Libraries: Support for lz4 (and bz2) indicates handling compressed data, either via custom or external libraries.
- CI/CD Tools: GitHub Actions workflows (in .github directory) and Docker for containerized builds hint at continuous integration best practices.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User
    U["User"]:::user

    %% Interface Layer Subgraph
    subgraph "Interface Layer"
        A1["Command-line Interface (embag_echo demo)"]:::interface
        A2["Python API (pip_package)"]:::interface
        A3["Python Adapters (python)"]:::interface
    end

    %% Core System Subgraph
    subgraph "Core System"
        B["Core C++ Library"]:::core
        C1["Compression Module (bz2)"]:::compression
        C2["Compression Module (lz4)"]:::compression
    end

    %% Build & Test Infrastructure Subgraph
    subgraph "Build & Test Infrastructure"
        D["Bazel Build System"]:::build
        E["Test Suite"]:::test
        F["CI/CD Pipeline"]:::cicd
    end

    %% Relationships from User to Interfaces
    U -->|"uses"| A1
    U -->|"uses"| A2
    U -->|"uses"| A3

    %% Interfaces use the Core Library
    A1 -->|"calls"| B
    A2 -->|"wraps"| B
    A3 -->|"binds to"| B

    %% Core Library relies on Compression Modules
    B -->|"decompresses"| C1
    B -->|"decompresses"| C2

    %% Build Infrastructure builds all components
    D -->|"builds"| B
    D -->|"builds"| A1
    D -->|"builds"| A2
    D -->|"builds"| A3
    D -->|"builds"| C1
    D -->|"builds"| C2
    D -->|"runs"| E

    %% CI/CD Pipeline triggers the Build System
    F -->|"triggers"| D

    %% Test Suite validates components
    E -->|"tests"| B
    E -->|"tests"| A1
    E -->|"tests"| A2
    E -->|"tests"| A3

    %% Click Events
    click B "https://github.com/embarktrucks/embag/tree/master/lib"
    click A1 "https://github.com/embarktrucks/embag/tree/master/embag_echo"
    click A2 "https://github.com/embarktrucks/embag/blob/master/pip_package/embag/__init__.py"
    click A3 "https://github.com/embarktrucks/embag/blob/master/python/embag.cc"
    click C1 "https://github.com/embarktrucks/embag/tree/master/bz2"
    click C2 "https://github.com/embarktrucks/embag/tree/master/lz4"
    click E "https://github.com/embarktrucks/embag/tree/master/test"
    click F "https://github.com/embarktrucks/embag/blob/master/.github"

    %% Styles
    classDef user fill:#B3E5FC,stroke:#0277BD,stroke-width:2px;
    classDef interface fill:#C8E6C9,stroke:#388E3C,stroke-width:2px;
    classDef core fill:#FFCDD2,stroke:#C62828,stroke-width:2px;
    classDef compression fill:#FFF9C4,stroke:#F9A825,stroke-width:2px;
    classDef build fill:#D1C4E9,stroke:#7B1FA2,stroke-width:2px;
    classDef test fill:#F0F4C3,stroke:#AFB42B,stroke-width:2px;
    classDef cicd fill:#B2EBF2,stroke:#0097A7,stroke-width:2px;
```
