How lvivona/safelz4 works
Project Type and Purpose
• This is an open-source library providing Python bindings to a Rust implementation of the LZ4 compression algorithm (lz4_flex).
• It exposes two main APIs: frame-based compression/decompression and block-based compression/decompression.
Top-Level Architecture Overview
• Python Layer (py/safelz4):
• Entry point ( __main__.py ) for CLI use.
• safelz4/__init__.py: exposes compress_file, open_frame, etc.
• safelz4/block.py and safelz4/frame.py: pure-Python wrappers around the Rust extension.
• Rust Core (src/*.rs):
• lib.rs: Rust library exposing block and frame APIs via FFI.
• block.rs, frame.rs, error.rs: implement core compression/decompression logic.
• Build & Packaging
• pyproject.toml & Cargo.toml: configure Python packaging (setuptools-rust) and Rust crate.
• makefile, GitHub workflows (.github/workflows/*.yml): CI for Python tests, Rust tests, releases.
• Tests & Quality
• tests/*.py: Python unit tests against the FFI extension.
• benches/: benchmarks using sample files.
• fuzz/: fuzz tests to validate error handling and round-trips.
Main Components
• A. Python Application Layer
• CLI module
• safelz4 Python package
• block API
• frame API
• B. Rust Extension Layer
• FFI boundary (C ABI).
• lz4_flex dependency (external Rust crate).
• C. Build & CI Infrastructure
• setuptools-rust binding generator
• Cargo build pipeline
• GitHub Actions workflows
• D. Test & Validation
• Unit tests (pytest)
• Benchmarks (bench harness)
• Fuzz tests (Python fuzz scripts invoking the FFI)
Relationships and Data Flow
• Data Flow A→B: Python calls into the compiled Rust extension via C-ABI (through the wheel/.so).
• lz4_flex: The Rust code imports and uses lz4_flex crate internally.
• Build Flow: pyproject.toml triggers setuptools-rust which invokes cargo to compile src/*.rs into a shared library.
• CI Flow: Code pushed → GitHub Actions → run rust tests, python tests, publish to PyPI.
Architectural Patterns and Principles
• FFI Bridge Pattern: decouples Python API from Rust implementation.
• Modular Design: distinct block vs. frame functionality, each in its own module.
• CI/CD Automation: workflows ensure cross-platform builds and releases.