# explosion/spacy-models architecture

> Architecture diagram of the GitHub repository explosion/spacy-models, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/explosion/spacy-models
- Repository: https://github.com/explosion/spacy-models

## Overview

##### Project type and purpose
- This is an open-source “model registry” repository for spaCy NLP models. It does not contain typical frontend/backend code but stores:
- metadata JSON files (in /meta/) describing each language/model release
- a compatibility index (compatibility.json) for matching spaCy versions to models
- automated test suites under /tests/
- release artifacts (wheels and tar.gz) published via GitHub Releases
- Its purpose is to serve as the authoritative index and delivery mechanism for spaCy model packages, enabling `python -m spacy download [model]` to locate, verify and install the correct model wheel.

##### File structure
- Top-level:
- README.md – overview and usage
- compatibility.json – spaCy-to-model mapping
- /meta – a large collection of `<lang>_<pipeline>_<genre>_<size>-<version>.json` files describing each model’s metadata
- /tests – pytest suites validating metadata, compatibility logic and download/install flows
- There are no distinct “frontend” or “backend” directories; this repo functions as a static registry with CI/CD to publish artifacts.
- Configuration & build: GitHub Actions (in .github/) package and upload wheels/tar.gz to the GitHub Releases page for each model.

##### README for additional insights
- It outlines:
- Naming conventions (lang, type, genre, size)
- Versioning semantics (spaCy major.minor + model version)
- Download/install commands via spaCy CLI and pip
- It describes how spaCy uses `compatibility.json` internally to choose the best-matching model release.

a. Main components
1. spaCy CLI module (user environment)
2. compatibility.json (local or fetched from repo)
3. GitHub Releases (artifact store of .whl/.tar.gz)
4. /meta JSON files (model metadata)
5. CI/CD pipeline (GitHub Actions building and publishing releases)
6. Local Python environment / pip installer
b. Relationships & interactions
- User invokes `python -m spacy download en_core_web_sm`
- spaCy CLI reads compatibility.json to determine compatible model version
- CLI queries GitHub Releases API for model wheel URL
- pip installs the wheel into the user’s environment
- spaCy.load() imports the installed model package
c. Architectural patterns & design principles
- Registry pattern: static metadata store (/meta) indexed by compatibility.json
- Continuous Delivery: GitHub Actions automatically package and publish new model versions
- Separation of concerns: metadata files separated from binary artifacts
d. Technologies & frameworks
- GitHub & GitHub Releases (artifact hosting)
- JSON metadata files
- Python wheels (`.whl`) and source archives (`.tar.gz`)
- spaCy CLI module
- pytest for testing download/compatibility logic

##### Detail & coverage
- Be exhaustive: include the roles of /meta files vs. compatibility.json vs. CI/CD.
- Do not overthink subdivisions—separate into as many logical components as needed for clarity.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% User Environment
    subgraph "User Environment"
        spaCyCLI["spaCy CLI"]:::external
        pipInstaller["pip Installer"]:::external
        spaCyLoad["spaCy.load()"]:::external
    end

    %% Registry Repository
    subgraph "Registry Repository (GitHub)"
        compatibilityJSON["compatibility.json"]:::metadata
        metaStore["Model Metadata Store (meta/)"]:::metadata
        README["README.md"]:::doc
        CI_CD["CI/CD Pipeline (.github/)"]:::ci
        subgraph "Test Suites"
            TSData["tests/data/"]:::tests
            TSLang["tests/lang/"]:::tests
            TSUtil["tests/util.py"]:::tests
            TSPytestIni["tests/pytest.ini"]:::tests
            TSReq["tests/requirements.txt"]:::tests
            TSFlake["tests/.flake8"]:::tests
            TSIgnore["tests/.gitignore"]:::tests
        end
    end

    %% External Services
    subgraph "External Services"
        GitHubReleases["GitHub Releases\n(model artifacts)"]:::artifacts
    end

    %% Interactions
    spaCyCLI -->|"lookup"| compatibilityJSON
    compatibilityJSON -->|"indexes"| metaStore
    spaCyCLI -->|"query API"| GitHubReleases
    GitHubReleases -->|"download wheel"| pipInstaller
    pipInstaller -->|"install model"| spaCyLoad
    CI_CD -->|"publish artifacts"| GitHubReleases

    %% Testing interactions
    TSData -.-> compatibilityJSON
    TSLang -.-> metaStore
    TSUtil -.-> compatibilityJSON
    TSUtil -.-> metaStore
    TSPytestIni -.-> compatibilityJSON
    TSReq -.-> compatibilityJSON

    %% Click Events
    click compatibilityJSON "https://github.com/explosion/spacy-models/blob/master/compatibility.json"
    click metaStore "https://github.com/explosion/spacy-models/tree/master/meta/"
    click CI_CD "https://github.com/explosion/spacy-models/tree/master/.github/"
    click TSData "https://github.com/explosion/spacy-models/tree/master/tests/data/"
    click TSLang "https://github.com/explosion/spacy-models/tree/master/tests/lang/"
    click TSUtil "https://github.com/explosion/spacy-models/blob/master/tests/util.py"
    click TSPytestIni "https://github.com/explosion/spacy-models/blob/master/tests/pytest.ini"
    click TSReq "https://github.com/explosion/spacy-models/blob/master/tests/requirements.txt"
    click TSFlake "https://github.com/explosion/spacy-models/blob/master/tests/.flake8"
    click TSIgnore "https://github.com/explosion/spacy-models/blob/master/tests/.gitignore"
    click README "https://github.com/explosion/spacy-models/blob/master/README.md"

    %% Styles
    classDef metadata fill:#FFEEAA,stroke:#E38B0C,color:#000
    classDef artifacts fill:#EEEFFF,stroke:#7F8EFA,color:#000
    classDef external fill:#E0E0E0,stroke:#888,color:#000
    classDef ci fill:#EEFFEE,stroke:#29A329,color:#000
    classDef tests fill:#FFEFEF,stroke:#E04A4A,color:#000
    classDef doc fill:#FFFFE0,stroke:#CCCC00,color:#000
```
