How griffindore/rosbag_parquet works
Project type and purpose
• This is an open-source Python library plus several command-line tools.
• Purpose: read ROS bagfiles, convert them to Pandas DataFrames, and write out Parquet or CSV (and also plot or print message keys).
File structure
• Top-level:
– scripts/ : CLI entry-points (bag_parquet, bag_csv, bag_plot, bag_print)
– src/rosbag_parquet/ : core library modules (dataframe.py, utils.py)
– requirements.txt, setup.py : dependencies & packaging
– tests/ : integration/unit tests
– doc/ : documentation assets (e.g. bag_plot.png)
• Patterns: separation of CLI layer (scripts) from core logic (src), standard Python packaging layout.
README insights
• Core functions:
– bag_to_dataframes()
– topics_from_keys()
• CLI usage for conversion, plotting, printing.
• External dependencies implied: ROS Python API (rosbag), Pandas, PyArrow (Parquet), Matplotlib (plotting).
Main components
a. CLI Layer (“scripts/”)
– bag_parquet
– bag_csv
– bag_plot
– bag_print
Each is a small entrypoint that:
• parses arguments (argparse)
• invokes core library functions
b. Core Library (“rosbag_parquet” package)
– utils.py
• Topic-pattern parsing (msgevalgen patterns)
• Filtering include/exclude lists
– dataframe.py
• Reads messages from ROS bag
• Transforms per-topic messages into Pandas DataFrames
• Handles indexing, optional compression
c. External Services / Libraries
– rosbag Python API (underlying ROS bag file reader)
– Pandas DataFrames
– PyArrow or fastparquet (Parquet engine)
– Matplotlib (for bag_plot)
– Python stdlib (argparse, logging, file I/O)
d. File System
– Input: .bag files
– Outputs: .parquet files, .csv files, plotted graphs, printed text
Relationships and data flow
– User invokes a script (e.g. bag_parquet) →
– Script parses CLI args →
– Calls rosbag_parquet.bag_to_dataframes() →
– Inside: utils.py filters topics → dataframe.py reads from rosbag API → builds DataFrames →
– DataFrames passed to write routines (Parquet via PyArrow or CSV via Pandas) →
– Files written to disk
– For bag_plot/bag_print: DataFrames or raw messages → Matplotlib or console output
Architectural patterns & design principles
• Clear layering: CLI layer decoupled from core logic
• Single-responsibility modules: utils for parsing/filtering, dataframe for data transformation
• Dependency injection via parameters (include/exclude lists, compression, indexing topic)
• Testability: small modules covered by tests/ directory