How ethereum/solc-bin works
Project Type & Purpose
• This is an open-source “static asset management” tool, hosting current and historical builds of the Solidity compiler for multiple platforms (Emscripten-ASM.js, Emscripten-WASM, Linux, macOS, Windows, pure WASM).
• README makes clear it’s a repository of static binaries, mirrored to GitHub Pages and an S3 bucket (binaries.soliditylang.org), with automated workflows to fetch, generate indexes, and publish.
Key Architectural Components
A. Source Repository (GitHub)
• Stores:
– Platform directories (e.g. linux-amd64/, macosx-amd64/, emscripten-asmjs/, wasm/…) containing versioned binaries and list.{js,json,txt}
– “bin/” folder mirroring latest soljson.js builds
– CI config (.circleci/, .github/workflows/)
– Publishing scripts (add-nightly-and-push.sh, sync-s3.sh, update.mjs)
– Jekyll config (_config.yml) for GitHub Pages
• Exposes code and static assets via GitHub Pages (ethereum.github.io/solc-bin)
B. Continuous Integration / Automation
• CircleCI & GitHub Actions workflows:
– nightly-emscripten.yml: builds nightly Emscripten versions
– random-macosx-build.yml: periodically builds random macOS versions
– s3-mirror.yml: syncs content to S3 bucket
– t-bytecode-compare.yml: binary smoke tests/comparison
• Scripts invoked by CI:
– update.mjs: regenerates list.json/js/txt indexes per directory
– sync-s3.sh: pushes folder structure to AWS S3 (binaries.soliditylang.org)
– add-nightly-and-push.sh: adds new nightly build to “bin/” and commits
C. Package & Build Tooling
• package.json: Node.js dependencies for update.mjs & related scripts
• soljson.js: entry point for latest compiler distribution in top-level
D. Static Hosting / Distribution
• GitHub Pages: serves static directory structure & list files
• AWS S3 + CloudFront (binaries.soliditylang.org): primary HTTPS distribution mirror
Relationships & Data Flows
1. Developer or upstream Solidity release triggers CI workflows in GitHub Actions or CircleCI.
2. CI jobs fetch or build compiler binaries for each platform.
3. Binaries get checked into respective platform directories or into “bin/”, and update.mjs regenerates list.js/json/txt indexes.
4. CI commits updated index + binaries back to GitHub (or pushes a release tag).
5. sync-s3.sh workflow (triggered by push or schedule) uploads the complete tree to the AWS S3 bucket.
6. Jekyll (_config.yml) on GitHub Pages picks up directory tree to serve via HTTPS.
7. Client tools (npm-solc, Truffle, Hardhat, users) fetch compilers directly from GitHub Pages or binaries.soliditylang.org.
Architectural Patterns & Design Principles
• Static asset hosting (no dynamic back-end servers)
• “Infrastructure as code” for publishing pipelines
• Single repo, multi-platform directory layout with standard naming conventions per version
• Idempotent list generation (update.mjs) to ensure consistency of index files
• Separation of concerns:
• CI workflows handle build/test/publish
• Scripts handle indexing & synchronization
• GitHub Pages & S3 handle serving