How dusty-nv/jetson-containers works
Project Type & Purpose
• This is a modular, CLI-driven container build system (“jetson-containers”) written in Python, augmented by shell scripts, for assembling and running AI/ML Docker images on NVIDIA Jetson hardware.
• It exposes commands like jetson-containers build, run, and autotag, reads package definitions from the packages/ folder, and orchestrates Docker builds, pulls, and runs with configured defaults (e.g., NVIDIA runtime, mounted volumes, CUDA versions).
Top-Level Components
a. CLI Frontend
• Entry point scripts (run.sh, build.sh, autotag, install.sh) wrap the Python package.
• Shell commands parse user flags & environment variables, invoke Python modules.
b. Core Python Engine (jetson_containers/)
• container.py: interfaces with Docker CLI/daemon (build, tag, run)
• build.py: dependency resolution, multi-stage build orchestration
• db.py: local metadata/graph of package definitions (packages/*/*.json, data/containers.json)
• network.py: pulls/pushes images from registries, pip server interactions
• packages.py: plugin loader for each package subdirectory (defines Dockerfiles, scripts)
• ci.py: helpers for GitHub Actions workflows
• utils.py & logging.py: shared utilities, standardized logging
c. Package Definitions (packages/…)
• Hundreds of subfolders grouping categories (llm, ml, cv, llm, speech, robotics, etc.)
• Each contains Dockerfile(s), build/install scripts, config metadata, and tests.
d. Local Data & Config
• .env, .env_sample for version pins (CUDA, Python, cuDNN)
• data/ for caching: model files, audio, prompts, logs, container DB snapshots
• docs/ for end-user documentation; .github/workflows/ for CI pipelines
e. External Services & Dependencies
• Docker daemon (local)
• Container registries (Docker Hub, GHCR, custom Pip index)
• GitHub Actions for CI building/testing on commits
• NVIDIA Jetson hardware (edge target)
Interactions & Data Flow
• User invokes CLI script → Frontend shell parses env/args → Python CLI entrypoint
• CLI reads .env, resolves package names → queries db.py for dependency graph
• build.py orchestrates build order, invokes Docker builds (via container.py)
• Docker daemon performs image layers, caches, tags images → optional push to registry
• network.py handles pulls of base images and pushes of completed images
• Data/metadata updates written back to data/containers.json and logs in data/logs/
• run command uses container.py to launch docker run with GPU runtime and mounts
Architectural Patterns & Principles
• Plugin-based architecture: each package folder self-describes build steps
• Layered abstraction over Docker CLI: container.py sits between build logic & Docker
• Dependency graph resolution: build.py ensures correct build order and caching
• Environment-driven configuration: .env, CLI flags, and scripts configure behavior
• Composition over monolith: users “compose” containers by name (e.g., pytorch + ros)
Relevant Technologies & Frameworks
• Language: Python 3.x
• Containers: Docker Engine with NVIDIA runtime
• CI: GitHub Actions (YAML workflows in .github/workflows/)
• Shell: bash build/install wrappers
• Data format: JSON for metadata, YAML for system configs (docker-compose.yaml)
• External APIs: Docker REST/CLI, GitHub, custom PyPI index