How explosion/spacy-models works
Project type and purpose
• This is an open-source “model registry” repository for spaCy NLP models. It does not contain typical frontend/backend code but stores:
• metadata JSON files (in /meta/) describing each language/model release
• a compatibility index (compatibility.json) for matching spaCy versions to models
• automated test suites under /tests/
• release artifacts (wheels and tar.gz) published via GitHub Releases
• Its purpose is to serve as the authoritative index and delivery mechanism for spaCy model packages, enabling python -m spacy download [model] to locate, verify and install the correct model wheel.
File structure
• Top-level:
• README.md – overview and usage
• compatibility.json – spaCy-to-model mapping
• /meta – a large collection of <lang>_<pipeline>_<genre>_<size>-<version>.json files describing each model’s metadata
• /tests – pytest suites validating metadata, compatibility logic and download/install flows
• There are no distinct “frontend” or “backend” directories; this repo functions as a static registry with CI/CD to publish artifacts.
• Configuration & build: GitHub Actions (in .github/) package and upload wheels/tar.gz to the GitHub Releases page for each model.
README for additional insights
• It outlines:
• Naming conventions (lang, type, genre, size)
• Versioning semantics (spaCy major.minor + model version)
• Download/install commands via spaCy CLI and pip
• It describes how spaCy uses compatibility.json internally to choose the best-matching model release.
a. Main components
1. spaCy CLI module (user environment)
2. compatibility.json (local or fetched from repo)
3. GitHub Releases (artifact store of .whl/.tar.gz)
4. /meta JSON files (model metadata)
5. CI/CD pipeline (GitHub Actions building and publishing releases)
6. Local Python environment / pip installer
b. Relationships & interactions
• User invokes python -m spacy download en_core_web_sm
• spaCy CLI reads compatibility.json to determine compatible model version
• CLI queries GitHub Releases API for model wheel URL
• pip installs the wheel into the user’s environment
• spaCy.load() imports the installed model package
c. Architectural patterns & design principles
• Registry pattern: static metadata store (/meta) indexed by compatibility.json
• Continuous Delivery: GitHub Actions automatically package and publish new model versions
• Separation of concerns: metadata files separated from binary artifacts
d. Technologies & frameworks
• GitHub & GitHub Releases (artifact hosting)
• JSON metadata files
• Python wheels (.whl) and source archives (.tar.gz)
• spaCy CLI module
• pytest for testing download/compatibility logic
Detail & coverage
• Be exhaustive: include the roles of /meta files vs. compatibility.json vs. CI/CD.
• Do not overthink subdivisions—separate into as many logical components as needed for clarity.