How anssi-fr/mla works
Project type and purpose
• This is a multi‐crate Rust repository implementing a custom archive format (“Multi Layer Archive” or MLA).
• Core deliverables:
• mla: the Rust library (reader + writer + repair) implementing the layered archive format
• mlar: a command‐line tool built atop the mla library for create/list/extract/convert operations
• curve25519‐parser: a helper library for parsing Ed25519/X25519 keys in PEM/DER
• mla‑fuzz‑afl: an AFL‐based fuzzing harness for testing mla
• bindings/C and bindings/python: language bindings exposing mla to C/C++ and Python
• samples: test archives and key files
• .github: CI pipelines (build, test, bench, release)
File structure
• Top‐level crates (each with Cargo.toml): mla, mlar, curve25519‐parser, mla‑fuzz‑afl
• bindings/ subdirectories for C and Python wrappers
• samples/ for shipped artifacts to test backward compatibility
• .github/ for CI definitions (benchmarks, tests, releases)
• FORMAT.md and README.md describe the layered “pipeline” architecture of the archive format
Extract architecture insights from the README
• MLA is built as a chain of optional/mandatory “layers” (Raw → Compression → Encryption → Position → File storage)
• Each layer provides a Writer, Reader, and FailSafeReader for repairability
• ArchiveWriterConfig / ArchiveReaderConfig govern which layers are enabled
• Cryptographic services: AES‑GCM (RustCrypto), ECIES over Curve25519 (x25519‑dalek), random nonces (getrandom + ChaChaRng)
• Compression service: Brotli chunks via rust‑brotli
• File storage: chunked, interleaved, indexed blocks with SHA‑256 file hashes
• CLI (mlar) and language bindings consume the mla library’s public API
Main components and their interactions
A. MLA Core Library (crate: mla)
• Modules:
• config – defines ArchiveWriterConfig, ArchiveReaderConfig
• layers – submodules: raw, compress, encrypt, position, file storage (plus traits)
• crypto – aesgcm.rs, ecc.rs, hash.rs
• helpers – StreamWriter, linear_extract
• errors, tests, benches
B. CLI Tool (crate: mlar)
• Uses mla::{ArchiveWriter, ArchiveReader} + curve25519‑parser to perform user‐facing commands
• Exposes subcommands: keygen, create, list, extract, convert, keyderive
C. Curve25519 Parser (crate: curve25519-parser)
• Provides parse_openssl_25519_pubkey / privkey for key ingestion
• Dependency of mla and mlar
D. Fuzzer (crate: mla‑fuzz‑afl)
• Invokes mla’s API in randomized scenarios to test robustness + repair
E. Language Bindings (bindings/C and bindings/python)
• C: cbindgen → C header + C++ header, wraps mla functions via FFI
• Python: pyo3‐based binding crate → Python module exposing writer/reader
F. CI/Benchmarking/Fuzz pipelines (.github workflows)
• Build & test all crates, run Criterion benchmarks, release artifacts, run AFL fuzzing, publish Python/C packages
Architectural patterns & design principles
• Layered pipeline: each processing layer implements the same IO traits, enforces a strict order, and is repairable without footer data
• Modularity: separate crates for core format, CLI, parsing, fuzzing, language bindings
• Single‐source‐of‐truth: all logic lives in Rust; bindings and tools are thin layers
• Streaming & seekable design: enables interleaved writes and random access reads
Essential elements
• Annotation of optional vs mandatory layers