# griffindore/rosbag_parquet architecture

> Architecture diagram of the GitHub repository griffindore/rosbag_parquet, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/griffindore/rosbag_parquet
- Repository: https://github.com/griffindore/rosbag_parquet

## Overview

##### Project type and purpose
- This is an open-source Python library plus several command-line tools.
- Purpose: read ROS bagfiles, convert them to Pandas DataFrames, and write out Parquet or CSV (and also plot or print message keys).

##### File structure
- Top-level:
– scripts/ : CLI entry-points (bag_parquet, bag_csv, bag_plot, bag_print)
– src/rosbag_parquet/ : core library modules (dataframe.py, utils.py)
– requirements.txt, setup.py : dependencies & packaging
– tests/ : integration/unit tests
– doc/ : documentation assets (e.g. bag_plot.png)
- Patterns: separation of CLI layer (scripts) from core logic (src), standard Python packaging layout.

##### README insights
- Core functions:
– bag_to_dataframes()
– topics_from_keys()
- CLI usage for conversion, plotting, printing.
- External dependencies implied: ROS Python API (rosbag), Pandas, PyArrow (Parquet), Matplotlib (plotting).

##### Main components
a. CLI Layer (“scripts/”)
– bag_parquet
– bag_csv
– bag_plot
– bag_print
Each is a small entrypoint that:
- parses arguments (argparse)
- invokes core library functions
b. Core Library (“rosbag_parquet” package)
– utils.py
- Topic-pattern parsing (msgevalgen patterns)
- Filtering include/exclude lists
– dataframe.py
- Reads messages from ROS bag
- Transforms per-topic messages into Pandas DataFrames
- Handles indexing, optional compression
c. External Services / Libraries
– rosbag Python API (underlying ROS bag file reader)
– Pandas DataFrames
– PyArrow or fastparquet (Parquet engine)
– Matplotlib (for bag_plot)
– Python stdlib (argparse, logging, file I/O)
d. File System
– Input: .bag files
– Outputs: .parquet files, .csv files, plotted graphs, printed text

##### Relationships and data flow
– User invokes a script (e.g. bag_parquet) →
– Script parses CLI args →
– Calls rosbag_parquet.bag_to_dataframes() →
– Inside: utils.py filters topics → dataframe.py reads from rosbag API → builds DataFrames →
– DataFrames passed to write routines (Parquet via PyArrow or CSV via Pandas) →
– Files written to disk
– For bag_plot/bag_print: DataFrames or raw messages → Matplotlib or console output

##### Architectural patterns & design principles
- Clear layering: CLI layer decoupled from core logic
- Single-responsibility modules: utils for parsing/filtering, dataframe for data transformation
- Dependency injection via parameters (include/exclude lists, compression, indexing topic)
- Testability: small modules covered by tests/ directory

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% User actor
    User(("User")):::actor

    %% CLI Layer
    subgraph "CLI Layer"
        direction TB
        BP((bag_parquet)):::cli
        BC((bag_csv)):::cli
        BPL((bag_plot)):::cli
        BPR((bag_print)):::cli
    end

    %% Core Library
    subgraph "Core Library" 
        direction TB
        U["utils.py"]:::core
        DF["dataframe.py"]:::core
    end

    %% External Dependencies
    subgraph "External Libraries"
        direction TB
        ROS(["ROS Bag Reader\n(rosbag API)"]):::external
        PD(["Pandas DataFrame"]):::external
        PA(["Parquet Engine\n(PyArrow/Fastparquet)"]):::external
        MT(["Matplotlib"]):::external
        STDOUT(["Console Output"]):::external
    end

    %% File I/O
    subgraph "File System"
        direction TB
        BAG[(".bag File")]:::io
        PARQ[(".parquet File")]:::io
        CSV[(".csv File")]:::io
    end

    %% Auxiliary Modules
    subgraph "Auxiliary Modules"
        direction TB
        Tests["tests/"]:::aux
        Doc["doc/"]:::aux
    end

    %% Connections
    User --> BP
    User --> BC
    User --> BPL
    User --> BPR

    BP -->|parses args| U
    BC -->|parses args| U
    BPL -->|parses args| U
    BPR -->|parses args| U

    U -->|filters topics| DF
    DF -->|reads messages| ROS
    ROS --> BAG
    DF -->|builds DataFrame| PD

    PD -->|writes Parquet| PA
    PA --> PARQ

    PD -->|writes CSV| CSV

    BPL -->|plots DataFrame| MT
    BPR -->|prints messages| STDOUT

    %% Click events
    click BP "https://github.com/griffindore/rosbag_parquet/tree/master/scripts/bag_parquet"
    click BC "https://github.com/griffindore/rosbag_parquet/tree/master/scripts/bag_csv"
    click BPL "https://github.com/griffindore/rosbag_parquet/tree/master/scripts/bag_plot"
    click BPR "https://github.com/griffindore/rosbag_parquet/tree/master/scripts/bag_print"
    click U "https://github.com/griffindore/rosbag_parquet/blob/master/src/rosbag_parquet/utils.py"
    click DF "https://github.com/griffindore/rosbag_parquet/blob/master/src/rosbag_parquet/dataframe.py"
    click Tests "https://github.com/griffindore/rosbag_parquet/tree/master/tests/"
    click Doc "https://github.com/griffindore/rosbag_parquet/tree/master/doc/"
    click Setup "https://github.com/griffindore/rosbag_parquet/blob/master/setup.py"
    click Req "https://github.com/griffindore/rosbag_parquet/blob/master/requirements.txt"

    %% Packaging & Dependencies nodes
    setup["setup.py"]:::aux
    req["requirements.txt"]:::aux
    TestsTest1["tests/test_import.py"]:::aux
    TestsTest2["tests/test_topics_from_keys.py"]:::aux
    Fixture["tests/data/rosout.bag"]:::aux

    %% Position packaging & tests
    setup -.-> Tests
    req -.-> Tests
    Tests -.-> TestsTest1
    Tests -.-> TestsTest2
    Tests -.-> Fixture

    %% Styles
    classDef actor fill:#ffffff,stroke:#000000,stroke-width:1px
    classDef cli fill:#add8e6,stroke:#333,stroke-width:1px
    classDef core fill:#90ee90,stroke:#333,stroke-width:1px
    classDef external fill:#ffa500,stroke:#333,stroke-width:1px
    classDef io fill:#d3d3d3,stroke:#333,stroke-width:1px
    classDef aux fill:#f0e68c,stroke:#333,stroke-width:1px
```
