How openzim/nautilus works
Project type and purpose
• Nautilus is an open-source, Python‐based CLI application that ingests a ZIP archive or a JSON collection file of multimedia documents and produces a browsable ZIM file. It orchestrates downloading, optional video recompression, static HTML generation (using Handlebars templates), and final ZIM packaging.
File structure
• Top-level:
• .github/ – CI/CD workflows for tests, Docker build, publishing.
• Dockerfile – containerizes the CLI tool.
• pyproject.toml – Python packaging and dependencies.
• src/nautiluszim/ – core package:
– entrypoint.py – CLI argument parsing and main pipeline kick-off.
– scraper.py – handles downloading videos, subtitles, thumbnails, profiles.
– templates/ – Handlebars (.handlebars), HTML, JS and CSS for the static UI.
– locale/ – i18n support (PO/MO files).
– constants.py – configuration defaults.
– VERSION and __about__.py – metadata.
– tasks.py – helper tasks for development (e.g., building docs or releases).
• contrib/ – extension scripts (e.g., diksha importer; recompress scripts to lower video size).
• tests/ – simple stub, pointing to an extensible test suite.
README for additional insights
• Outlines usage modes (--archive, --collection), required JSON schema, optional about page, environment quirks, and linking to openZIM packaging conventions. It also highlights translation support and Docker publishing.
a. Main components
1. CLI Frontend (entrypoint)
2. Input Loader
• Archive Handler (unzips and reads collection.json/about.html)
• Remote Collection Fetcher (HTTP GET for URLs)
3. Content Pipeline
• Scraper Module (downloads media, thumbnails, subtitles, profiles)
• Optional Recompress Module (ffmpeg invocation or contrib/recompress scripts)
• Static Page Generator (applies Handlebars templates + CSS/JS + locale files)
4. ZIM Builder (invokes zimwriter or libzim to bundle static files into .zim)
5. Output – ZIM File written to filesystem or STDOUT
6. Extensions/Plugins (contrib/diksha, etc.)
7. External Services – HTTP endpoints (media URLs), Docker registry (for container), PyPI (for distribution)
b. Relationships & data flow
• User → CLI Frontend → parses args → routes to Input Loader
• Archive Handler / Remote Fetcher → produces document list → feeds Scraper
• Scraper → raw media files → (optionally) Recompress → processed media
• Static Page Generator consumes media + metadata → writes HTML/JS/CSS tree
• ZIM Builder → reads static tree → outputs ZIM
• Extensions hook into the Content Pipeline before or after Scraper
c. Architectural patterns & design principles
• Modular pipeline: each stage is a separate Python module/class.
• Plugin extensibility: contrib/ scripts illustrate how new inputs or processing stages can be added.
• Separation of concerns:
• I/O (Archive vs. HTTP)
• Processing (download vs. recompress vs. render)
• Packaging (ZIM)
• Declarative templating for UI, i18n support in locale/.
d. Key technologies & frameworks
• Python 3, Poetry/pyproject
• Handlebars templates for static UI
• GNU gettext for translations
• ffmpeg (external) or Python recompress scripts
• zimwriterfs or libzim for ZIM creation
• Docker for containerized distribution
• GitHub Actions for CI/CD
Essential architectural elements
• Don’t overcrowd: stick to the main flow (Input → Scraper → Generator → ZIM) but show optional branches (recompress, plugins).