How instill-ai/package works
Project type and purpose
• This is not a web app or service; it’s a CI/CD packaging utility.
• Purpose: automate building/testing of MLC-AI Python wheels and Conda packages on Linux (manylinux, Ubuntu, AlmaLinux), macOS (x86_64 & Apple Silicon), Windows, and handle nightly pruning and uploads.
File structure
• .github/workflows/: GitHub Actions definitions (triggers and pipelines).
• docker/: Dockerfiles and install scripts to create build environments.
• conda/: Conda build recipes and environment definitions.
• scripts/: Bash/Batch/Python scripts coordinating builds, pruning, syncing, uploads.
• 3rdparty/tlcpack: bundled third-party component.
• Root files (.gitignore, LICENSE, README.md, .gitmodules) manage governance and submodules.
Insights from README.md
• Badges show nightly workflows for wheel builds and prune jobs.
• Confirms CI-driven packaging across OS/architectures.
A. Source Repository (GitHub)
– Hosts code, workflows, Dockerfiles, conda recipes, scripts.
B. GitHub Actions CI/CD
– Workflow triggers (pushes, schedules).
– Jobs: build_docker_ci, wheel_*_nightly, prune_nightly, package builds.
C. Docker Build Environments
– Multiple Dockerfiles (CI-CPU, GPU variants, ROCm, etc.).
– docker/install/* scripts to install CUDA/ROCm/conda/python libs.
D. Conda Recipes & Environments
– Environment YAMLs to define build and deploy environments.
– Recipes (.recipe/) to assemble mlc-llm packages.
E. Packaging Scripts
– Build scripts (build_mlc_*.sh/.bat), wheel prune, sync, upload.
F. External Services
– Docker registry (for base images).
– Conda channels (for dependencies and upload).
– PyPI (for wheel publishing).
– Artifact storage or GitHub Packages (for intermediate artifacts).
Relationships & data flow
1. Trigger: GitHub Actions scheduled or on-push →
2. Spin up Docker Containers using specific Dockerfile →
3. Run Conda Env Setup inside container →
4. Invoke build scripts to compile C/C++ libs, package Python wheels & Conda tars →
5. Prune old wheels (prune_nightly job) →
6. Upload artifacts to PyPI / Conda channels / GitHub Packages.
Architectural patterns & principles
• CICD as code: pipelines defined in YAML (GitHub Actions).
• Containerization: deterministic build environments via Docker.
• Modular recipes: separate conda envs and recipes for variations.
• Cross-platform builds: use scripts and Docker to abstract OS differences.