How 5igi0/masstuffy works
Project type and purpose
• Masstuffy is a Rust-based object-storage solution that exposes both:
• A command-line interface (CLI) for local management of collections and records.
• An HTTP server for remote management of collections and records.
• It uses SQLite (via SQLx) for metadata (CDX indexes) and WARC files on the filesystem for object payloads.
Main architectural components
a. CLI component (src/bin/cli.rs + cli_logic modules)
b. HTTP Server component (src/bin/server.rs + server_logic/endpoints)
c. Core library (src/lib.rs + modules: config, constants, database, filesystem, warc, utils)
d. Database layer (migrations/ + src/database)
e. WARC storage layer (src/filesystem + src/warc)
f. External interfaces (users calling CLI or sending HTTP requests)
Relationships and data flows
• The CLI and Server both sit “on top” of the same Core Library.
• Both use the Database module to run SQLx queries against the SQLite DB.
• Both use the Filesystem + WARC modules to read/write WARC files on disk.
• Server exposes HTTP endpoints (collections, dictionaries, record_getters, record_search) that map to server_logic handlers, which call Core Library functions.
• CLI logic modules map commands (create_collection, push_records, get_record, search) to Core Library calls.
Architectural patterns & design principles
• Layered architecture:
• Presentation layer: CLI & HTTP Server
• Application logic layer: server_logic & cli_logic modules
• Core services layer: config, constants, database, filesystem, warc, utils
• Infrastructure: SQLite via SQLx, migrations, raw files on disk.
• Shared library approach: Avoid duplication by centralizing business logic in src/lib.rs and its submodules.
• Migration-based database versioning (migrations/*.sql files).
Key technologies/frameworks
• Rust & Cargo (Cargo.toml)
• SQLx for async DB access (Cargo.toml + .sqlx files)
• SQLite as the metadata store (migrations folder)
• WARC file format for object storage
• Tokio async runtime (implied by async modules)
• CLI parsing (structopt or clap in cli.rs)
• HTTP framework (e.g. Axum or Warp in server.rs)
Final checklist
[ ] Annotations for config (env vars, config/mod.rs), utils (seek.rs)
[ ] Callouts for HTTP endpoints and CLI commands