# openzim/nautilus architecture

> Architecture diagram of the GitHub repository openzim/nautilus, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/openzim/nautilus
- Repository: https://github.com/openzim/nautilus

## Overview

##### Project type and purpose
- Nautilus is an open-source, Python‐based CLI application that ingests a ZIP archive or a JSON collection file of multimedia documents and produces a browsable ZIM file. It orchestrates downloading, optional video recompression, static HTML generation (using Handlebars templates), and final ZIM packaging.

##### File structure
- Top-level:
- `.github/` – CI/CD workflows for tests, Docker build, publishing.
- `Dockerfile` – containerizes the CLI tool.
- `pyproject.toml` – Python packaging and dependencies.
- `src/nautiluszim/` – core package:
– `entrypoint.py` – CLI argument parsing and main pipeline kick-off.
– `scraper.py` – handles downloading videos, subtitles, thumbnails, profiles.
– `templates/` – Handlebars (`.handlebars`), HTML, JS and CSS for the static UI.
– `locale/` – i18n support (PO/MO files).
– `constants.py` – configuration defaults.
– `VERSION` and `__about__.py` – metadata.
– `tasks.py` – helper tasks for development (e.g., building docs or releases).
- `contrib/` – extension scripts (e.g., `diksha` importer; `recompress` scripts to lower video size).
- `tests/` – simple stub, pointing to an extensible test suite.

##### README for additional insights
- Outlines usage modes (`--archive`, `--collection`), required JSON schema, optional about page, environment quirks, and linking to openZIM packaging conventions. It also highlights translation support and Docker publishing.

a. Main components
1. CLI Frontend (entrypoint)
2. Input Loader
- Archive Handler (unzips and reads `collection.json`/`about.html`)
- Remote Collection Fetcher (HTTP GET for URLs)
3. Content Pipeline
- Scraper Module (downloads media, thumbnails, subtitles, profiles)
- Optional Recompress Module (ffmpeg invocation or `contrib/recompress` scripts)
- Static Page Generator (applies Handlebars templates + CSS/JS + locale files)
4. ZIM Builder (invokes zimwriter or libzim to bundle static files into `.zim`)
5. Output – ZIM File written to filesystem or STDOUT
6. Extensions/Plugins (`contrib/diksha`, etc.)
7. External Services – HTTP endpoints (media URLs), Docker registry (for container), PyPI (for distribution)

b. Relationships & data flow
- User → CLI Frontend → parses args → routes to Input Loader
- Archive Handler / Remote Fetcher → produces document list → feeds Scraper
- Scraper → raw media files → (optionally) Recompress → processed media
- Static Page Generator consumes media + metadata → writes HTML/JS/CSS tree
- ZIM Builder → reads static tree → outputs ZIM
- Extensions hook into the Content Pipeline before or after Scraper

c. Architectural patterns & design principles
- Modular pipeline: each stage is a separate Python module/class.
- Plugin extensibility: `contrib/` scripts illustrate how new inputs or processing stages can be added.
- Separation of concerns:
- I/O (Archive vs. HTTP)
- Processing (download vs. recompress vs. render)
- Packaging (ZIM)
- Declarative templating for UI, i18n support in `locale/`.

d. Key technologies & frameworks
- Python 3, Poetry/pyproject
- Handlebars templates for static UI
- GNU gettext for translations
- ffmpeg (external) or Python recompress scripts
- zimwriterfs or libzim for ZIM creation
- Docker for containerized distribution
- GitHub Actions for CI/CD

##### Essential architectural elements
- Don’t overcrowd: stick to the main flow (Input → Scraper → Generator → ZIM) but show optional branches (recompress, plugins).

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Input Layer"
        CLI["CLI Frontend\n(entrypoint)"]:::cli
        Archive["Archive Handler"]:::module
        Fetcher["Remote Collection Fetcher"]:::module
    end

    subgraph "Processing Pipeline"
        Scraper["Scraper Module"]:::module
        Recompress["Optional Recompress Module"]:::module
        subgraph "Static Page Generator"
            Generator["Static Page Generator"]:::module
            Templates["Handlebars Templates\n+ CSS/JS/HTML"]:::data
            Locale["Localization Files"]:::data
        end
    end

    subgraph "Output Layer"
        ZIMBuilder["ZIM Builder"]:::module
        ZIMFile["ZIM File"]:::data
    end

    subgraph "Extensions / Plugins"
        Diksha["Diksha Importer"]:::module
        RecompressExt["Recompress Scripts"]:::module
    end

    subgraph "Infrastructure"
        PYProject["pyproject.toml"]:::infra
        DockerNode["Dockerfile"]:::infra
        CI["GitHub Workflows"]:::infra
        Tasks["Development Tasks\n(tasks.py)"]:::infra
        OpenZIM["openzim.toml"]:::infra
        VersionFile["VERSION"]:::infra
        About["__about__.py"]:::infra
        Config["Constants"]:::infra
        Tests["Test Stub\n(tests/test_stub.py)"]:::infra
    end

    Media["HTTP Media Servers"]:::external

    CLI --> Archive
    CLI --> Fetcher
    Archive --> Scraper
    Fetcher --> Scraper
    Scraper --> Recompress
    Scraper --> Generator
    Recompress --> Generator
    Templates --> Generator
    Locale --> Generator
    Generator --> ZIMBuilder
    ZIMBuilder --> ZIMFile

    Archive --> Diksha
    Scraper --> RecompressExt

    Media --> Scraper

    CLI -.-> PYProject
    DockerNode -.-> CLI
    CI -.-> DockerNode
    CI -.-> PYProject
    Tasks -.-> CI
    Tests -.-> CI
    VersionFile -.-> PYProject
    About -.-> PYProject
    Config -.-> Scraper
    OpenZIM -.-> ZIMBuilder

    click CLI "https://github.com/openzim/nautilus/blob/main/src/nautiluszim/entrypoint.py"
    click Scraper "https://github.com/openzim/nautilus/blob/main/src/nautiluszim/scraper.py"
    click Config "https://github.com/openzim/nautilus/blob/main/src/nautiluszim/constants.py"
    click Templates "https://github.com/openzim/nautilus/tree/main/src/nautiluszim/templates/"
    click Locale "https://github.com/openzim/nautilus/tree/main/src/nautiluszim/locale/"
    click VersionFile "https://github.com/openzim/nautilus/tree/main/src/nautiluszim/VERSION"
    click About "https://github.com/openzim/nautilus/blob/main/src/nautiluszim/__about__.py"
    click Tasks "https://github.com/openzim/nautilus/blob/main/tasks.py"
    click PYProject "https://github.com/openzim/nautilus/blob/main/pyproject.toml"
    click DockerNode "https://github.com/openzim/nautilus/tree/main/Dockerfile"
    click CI "https://github.com/openzim/nautilus/tree/main/.github/workflows/"
    click OpenZIM "https://github.com/openzim/nautilus/blob/main/openzim.toml"
    click Tests "https://github.com/openzim/nautilus/blob/main/tests/test_stub.py"
    click Diksha "https://github.com/openzim/nautilus/blob/main/contrib/diksha/xls2collection.py"
    click Diksha "https://github.com/openzim/nautilus/tree/main/contrib/diksha/Dockerfile"
    click Diksha "https://github.com/openzim/nautilus/blob/main/contrib/diksha/requirements.txt"
    click RecompressExt "https://github.com/openzim/nautilus/blob/main/contrib/recompress/scripts/recompress_file.py"
    click RecompressExt "https://github.com/openzim/nautilus/blob/main/contrib/recompress/scripts/recompress_tree.py"
    click RecompressExt "https://github.com/openzim/nautilus/tree/main/contrib/recompress/Dockerfile"
    click RecompressExt "https://github.com/openzim/nautilus/blob/main/contrib/recompress/requirements.txt"

    classDef cli fill:#f9f,stroke:#333,stroke-width:2px
    classDef module fill:#cff,stroke:#333,stroke-width:2px
    classDef data fill:#cfc,stroke:#333,stroke-width:2px
    classDef external fill:#ccf,stroke:#333,stroke-width:2px
    classDef infra fill:#ffc,stroke:#333,stroke-width:2px
```
