# thetorproject/lepidopter architecture

> Architecture diagram of the GitHub repository thetorproject/lepidopter, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/thetorproject/lepidopter
- Repository: https://github.com/thetorproject/lepidopter

## Overview

##### Project type and purpose
- Lepidopter is an open-source image-building tool.
- It automates creation of a Raspberry Pi (armel) Debian Jessie SD-card image pre-configured with OONI Probe and an auto-updater.

##### File structure
- Top-level “scripts/” and “lepidopter-vmdebootstrap_build.sh” implement the build logic.
- “conf/” (lepidopter-image.conf, tor-pt.conf) defines the build-time configuration.
- “pkglist/” lists OS and Python packages to install.
- “lepidopter-fh/” is a complete filesystem hierarchy template (boot/, etc/, opt/, usr/, var/) used to assemble the rootfs.
- “images/” is where the final .img(.xz) artifacts land.
- Sign-and-release scripts under “scripts/” handle packaging and distribution.
- Build depends on external tool vmdebootstrap and standard Unix tooling (bash, dd, xz, bmaptool).

##### README for additional insights
- The build is driven by vmdebootstrap + shell wrappers (setup.sh, customize, cleanup).
- Runtime on-device services: Avahi (mDNS), systemd units (ooniprobe, lepidopter-update), cron jobs, logrotate, sshd.
- OONI Probe runs a local web UI on port 80; updater.py (Python) pulls and applies signed updates.

##### Detail
- Every directory or file group that influences the build (conf/, pkglist/, lepidopter-fh/, customize, cleanup).
- Systemd units and runtime jobs (cron, logrotate, watchdog).
- External dependencies (vmdebootstrap, network, update server).
- Critical config files (e.g., tor-pt.conf for Tor bridges, ooniprobe.conf).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Build Environment
    subgraph "Build Environment"
        direction TB
        SetupScript["setup.sh"]:::build
        VMDBDriver["lepidopter-vmdebootstrap_build.sh"]:::build
        VMDBEngine["vmdebootstrap engine"]:::build
        subgraph "Configuration Inputs"
            direction TB
            Conf1["lepidopter-image.conf"]:::config
            Conf2["tor-pt.conf"]:::config
        end
        subgraph "Package Lists"
            direction TB
            Pkg1["lepidopter-v1.0.0-armel-apt-packages"]:::config
            Pkg2["lepidopter-v1.0.0-armel-pip-packages"]:::config
        end
        Customize["customize"]:::build
        Cleanup["cleanup.sh"]:::build
        RootfsTemplateDir["lepidopter-fh/"]:::input
        ImageOutput["images/"]:::input
        SigningScript["lepidopter-sign.sh"]:::build
        ReleaseScript["release"]:::build

        SetupScript --> VMDBDriver
        VMDBDriver --> VMDBEngine
        VMDBEngine --> Conf1
        VMDBEngine --> Conf2
        VMDBEngine --> Pkg1
        VMDBEngine --> Pkg2
        VMDBEngine --> Customize
        VMDBEngine --> Cleanup
        VMDBEngine --> RootfsTemplateDir
        VMDBEngine --> ImageOutput
        ImageOutput --> SigningScript
        SigningScript --> ReleaseScript
    end

    %% Device Runtime
    subgraph "Device Runtime"
        direction TB
        BootConfig["config.txt"]:::config
        APTConfig["02compress-indexes"]:::config
        subgraph "Avahi Services"
            direction TB
            AvahiProbe["ooniprobe.service"]:::config
            AvahiSSH["ssh.service"]:::config
        end
        subgraph "Cron Jobs"
            direction TB
            CronDaily["remove_old_logs"]:::config
            CronTab["crontab"]:::config
        end
        subgraph "Systemd Units"
            direction TB
            SDProbe["ooniprobe.service"]:::runtime
            SDUpdate["lepidopter-update.service"]:::runtime
        end
        subgraph "Update Mechanism"
            direction TB
            Updater["updater.py"]:::runtime
            PublicKey["public.asc"]:::config
            Versions["versions/"]:::input
        end
        subgraph "Logrotate Configs"
            direction TB
            LRProbe["ooniprobe"]:::config
            LRUpdate["lepidopter-update"]:::config
        end
        subgraph "OONI Config"
            direction TB
            OONIConf["ooniprobe.conf"]:::config
            OONIScript["ooniconfig.sh"]:::config
        end
        WatchdogCfg["watchdog.conf"]:::config
        SSHRegen["regenerate_ssh_host_keys"]:::config
        UI["OONI Web UI (HTTP)"]:::runtime
        SSHD["sshd"]:::runtime
        AvahiDaemon["Avahi Daemon"]:::runtime
        Watchdog["watchdog"]:::runtime

        RootfsTemplateDir --> BootConfig
        RootfsTemplateDir --> APTConfig
        RootfsTemplateDir --> AvahiProbe
        RootfsTemplateDir --> AvahiSSH
        RootfsTemplateDir --> CronDaily
        RootfsTemplateDir --> CronTab
        RootfsTemplateDir --> SDProbe
        RootfsTemplateDir --> SDUpdate
        RootfsTemplateDir --> Updater
        RootfsTemplateDir --> PublicKey
        RootfsTemplateDir --> Versions
        RootfsTemplateDir --> LRProbe
        RootfsTemplateDir --> LRUpdate
        RootfsTemplateDir --> OONIConf
        RootfsTemplateDir --> OONIScript
        RootfsTemplateDir --> WatchdogCfg
        RootfsTemplateDir --> SSHRegen

        SDProbe --> UI
        SDUpdate --> Updater
        Updater -->|"fetch updates"| Internet
        CronDaily -->|"remove logs"| VarLog
        VarLog["/var/log"]:::runtime
        LogRotate["logrotate"]:::runtime
        CronTab --> LogRotate
        AvahiDaemon --> UI
        AvahiDaemon --> SSHD
        Watchdog -->|monitors| SDProbe
    end

    %% External
    Internet(("Internet")):::cloud
    User(("User")):::external

    Internet -->|"update server"| Updater
    User -->|HTTP| UI
    User -->|SSH| SSHD

    %% Click Events
    click SetupScript "https://github.com/thetorproject/lepidopter/blob/master/scripts/setup.sh"
    click VMDBDriver "https://github.com/thetorproject/lepidopter/blob/master/lepidopter-vmdebootstrap_build.sh"
    click Conf1 "https://github.com/thetorproject/lepidopter/blob/master/conf/lepidopter-image.conf"
    click Conf2 "https://github.com/thetorproject/lepidopter/blob/master/conf/tor-pt.conf"
    click Pkg1 "https://github.com/thetorproject/lepidopter/blob/master/pkglist/lepidopter-v1.0.0-armel-apt-packages"
    click Pkg2 "https://github.com/thetorproject/lepidopter/blob/master/pkglist/lepidopter-v1.0.0-armel-pip-packages"
    click Customize "https://github.com/thetorproject/lepidopter/tree/master/customize"
    click Cleanup "https://github.com/thetorproject/lepidopter/blob/master/lepidopter-fh/cleanup.sh"
    click RootfsTemplateDir "https://github.com/thetorproject/lepidopter/tree/master/lepidopter-fh/"
    click ImageOutput "https://github.com/thetorproject/lepidopter/tree/master/images/"
    click SigningScript "https://github.com/thetorproject/lepidopter/blob/master/scripts/lepidopter-sign.sh"
    click ReleaseScript "https://github.com/thetorproject/lepidopter/tree/master/scripts/release"
    click BootConfig "https://github.com/thetorproject/lepidopter/blob/master/lepidopter-fh/boot/config.txt"
    click APTConfig "https://github.com/thetorproject/lepidopter/tree/master/lepidopter-fh/etc/apt/apt.conf.d/02compress-indexes"
    click AvahiProbe "https://github.com/thetorproject/lepidopter/blob/master/lepidopter-fh/etc/avahi/services/ooniprobe.service"
    click AvahiSSH "https://github.com/thetorproject/lepidopter/blob/master/lepidopter-fh/etc/avahi/services/ssh.service"
    click CronDaily "https://github.com/thetorproject/lepidopter/tree/master/lepidopter-fh/etc/cron.daily/remove_old_logs"
    click CronTab "https://github.com/thetorproject/lepidopter/tree/master/lepidopter-fh/etc/crontab"
    click SDProbe "https://github.com/thetorproject/lepidopter/blob/master/lepidopter-fh/etc/systemd/system/ooniprobe.service"
    click SDUpdate "https://github.com/thetorproject/lepidopter/blob/master/lepidopter-fh/etc/systemd/system/lepidopter-update.service"
    click Updater "https://github.com/thetorproject/lepidopter/blob/master/lepidopter-fh/opt/ooni/lepidopter-update/updater.py"
    click PublicKey "https://github.com/thetorproject/lepidopter/blob/master/lepidopter-fh/opt/ooni/lepidopter-update/public.asc"
    click Versions "https://github.com/thetorproject/lepidopter/tree/master/lepidopter-fh/opt/ooni/lepidopter-update/versions/"
    click LRProbe "https://github.com/thetorproject/lepidopter/tree/master/lepidopter-fh/etc/logrotate.d/ooniprobe"
    click LRUpdate "https://github.com/thetorproject/lepidopter/tree/master/lepidopter-fh/etc/logrotate.d/lepidopter-update"
    click OONIConf "https://github.com/thetorproject/lepidopter/blob/master/lepidopter-fh/etc/ooniprobe.conf"
    click OONIScript "https://github.com/thetorproject/lepidopter/blob/master/lepidopter-fh/etc/ooniprobe/ooniconfig.sh"
    click WatchdogCfg "https://github.com/thetorproject/lepidopter/blob/master/lepidopter-fh/etc/watchdog.conf"
    click SSHRegen "https://github.com/thetorproject/lepidopter/tree/master/lepidopter-fh/etc/init.d/regenerate_ssh_host_keys"

    %% Styles
    classDef build fill:#bbdefb,stroke:#1e88e5;
    classDef input fill:#c8e6c9,stroke:#2e7d32;
    classDef runtime fill:#ffe0b2,stroke:#ef6c00;
    classDef config fill:#eeeeee,stroke:#757575;
    classDef cloud fill:#b3e5fc,stroke:#039be5;
    classDef external fill:#f3e5f5,stroke:#8e24aa;
```
