# linus-berg/backpack architecture

> Architecture diagram of the GitHub repository linus-berg/backpack, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/linus-berg/backpack
- Repository: https://github.com/linus-berg/backpack

## Overview

Backpack is a distributed artifact collection and tracking system, not a repository-serving platform. The documented flow begins when a user submits an artifact through the API: ecosystem processors resolve versions and dependencies, Core.Gateway records changes and routes collection work, protocol-specific collectors fetch missing files into S3-compatible storage, and Tracker.Scheduler periodically re-tracks artifacts. The graph preserves the documented stages and major ecosystem/protocol branches; several links are based on the README because their service implementations were not sampled.

Main entry points: [`Collector.Kernel/FileSystem.cs`](https://github.com/linus-berg/backpack/blob/master/Collector.Kernel/FileSystem.cs) (Protocol collectors), [`Core.Gateway/GatewayProcessingService.cs`](https://github.com/linus-berg/backpack/blob/master/Core.Gateway/GatewayProcessingService.cs) (Gateway processing), [`Integration.API/Program.cs`](https://github.com/linus-berg/backpack/blob/master/Integration.API/Program.cs) (REST API), [`Integration.API/Controllers/ArtifactController.cs`](https://github.com/linus-berg/backpack/blob/master/Integration.API/Controllers/ArtifactController.cs) (Artifact operations)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_access["API and management"]
  node_api["REST API<br/>[Program.cs]"]
  node_artifact_api["Artifact operations"]
  node_api_auth["API-key authentication"]
end

subgraph group_orchestration["Artifact orchestration"]
  node_gateway["Gateway processing"]
  node_gateway_consumers["Gateway consumers<br/>[Consumers]"]
  node_router["Collector router<br/>[Router.cs]"]
end

subgraph group_processors["Ecosystem processors"]
  node_processor_fleet["Ecosystem processors<br/>[Processor.Npm]"]
  node_npm["Npm processor<br/>[Npm.cs]"]
  node_nuget["NuGet processor<br/>[Nuget.cs]"]
  node_pypi["PyPI processor<br/>[Pypi.cs]"]
  node_containers["Container processor<br/>[Worker.cs]"]
  node_helm["Helm processor<br/>[Helm.cs]"]
  node_terraform["Terraform processor<br/>[Terraform.cs]"]
  node_huggingface["Hugging Face processor<br/>[HuggingFace.cs]"]
  node_github["GitHub releases processor<br/>[GithubReleases.cs]"]
  node_jetbrains["JetBrains processors<br/>[Jetbrains.cs]"]
  node_other_processors["Other ecosystem processors<br/>[Rancher.cs]"]
end

subgraph group_collectors["File collectors"]
  node_collector_fleet["Protocol collectors<br/>[FileSystem.cs]"]
  node_http_collectors["HTTP and mirror<br/>[WebMirror.cs]"]
  node_git_collector["Git collector<br/>[Git.cs]"]
  node_container_collector["Container collectors<br/>[Docker.cs]"]
  node_other_collectors["Other collectors<br/>[RSync.cs]"]
end

subgraph group_storage["Persistence and scheduling"]
  node_filesystem["Storage filesystem<br/>[FileSystem.cs]"]
  node_s3[("S3-compatible storage<br/>[MinioStorage.cs]")]
  node_mongo[("MongoDB metadata<br/>[MongoDatabase.cs]")]
  node_redis[("Redis dependency cache<br/>[CoreCache.cs]")]
  node_scheduler["Schedule tracking<br/>[ScheduleManager.cs]"]
end

node_user(("Artifact submitter"))
node_rabbitmq["RabbitMQ"]
node_upstream["Artifact registries"]

node_user -->|"submits artifact"| node_api
node_api -->|"authenticates requests"| node_api_auth
node_api -->|"routes artifact requests"| node_artifact_api
node_artifact_api -.->|"reads and writes metadata"| node_mongo
node_artifact_api -.->|"queues work"| node_rabbitmq
node_rabbitmq -.->|"delivers messages"| node_gateway_consumers
node_gateway_consumers -.->|"invokes processing"| node_gateway
node_gateway -->|"stores changes"| node_mongo
node_gateway -->|"routes files"| node_router
node_gateway -->|"dispatches dependencies"| node_processor_fleet
node_processor_fleet -->|"resolves versions"| node_upstream
node_npm -.->|"queries packages"| node_upstream
node_nuget -.->|"queries packages"| node_upstream
node_pypi -.->|"queries packages"| node_upstream
node_containers -.->|"queries images"| node_upstream
node_helm -.->|"queries charts"| node_upstream
node_terraform -.->|"queries providers"| node_upstream
node_huggingface -.->|"queries models"| node_upstream
node_github -.->|"queries releases"| node_upstream
node_jetbrains -.->|"queries products"| node_upstream
node_other_processors -.->|"queries ecosystems"| node_upstream
node_router -->|"dispatches collection"| node_collector_fleet
node_collector_fleet -.->|"selects protocol collector"| node_http_collectors
node_collector_fleet -.->|"selects protocol collector"| node_git_collector
node_collector_fleet -.->|"selects protocol collector"| node_container_collector
node_collector_fleet -.->|"selects protocol collector"| node_other_collectors
node_collector_fleet -->|"stores collected files"| node_filesystem
node_filesystem -->|"reads and writes objects"| node_s3
node_scheduler -->|"reads schedules"| node_mongo
node_scheduler -.->|"retracks artifacts"| node_rabbitmq
node_gateway -.->|"tracks dependencies"| node_redis

click node_api "https://github.com/linus-berg/backpack/blob/master/Integration.API/Program.cs"
click node_artifact_api "https://github.com/linus-berg/backpack/blob/master/Integration.API/Controllers/ArtifactController.cs"
click node_api_auth "https://github.com/linus-berg/backpack/blob/master/Integration.API/ApiKeyAuthenticationHandler.cs"
click node_gateway "https://github.com/linus-berg/backpack/blob/master/Core.Gateway/GatewayProcessingService.cs"
click node_gateway_consumers "https://github.com/linus-berg/backpack/tree/master/Core.Gateway/Consumers"
click node_processor_fleet "https://github.com/linus-berg/backpack/tree/master/Processor.Npm"
click node_npm "https://github.com/linus-berg/backpack/blob/master/Processor.Npm/Npm.cs"
click node_nuget "https://github.com/linus-berg/backpack/blob/master/Processor.Nuget/Nuget.cs"
click node_pypi "https://github.com/linus-berg/backpack/blob/master/Processor.Pypi/Pypi.cs"
click node_containers "https://github.com/linus-berg/backpack/blob/master/Processor.Container/Worker.cs"
click node_helm "https://github.com/linus-berg/backpack/blob/master/Processor.Helm/Helm.cs"
click node_terraform "https://github.com/linus-berg/backpack/blob/master/Processor.Terraform/Terraform.cs"
click node_huggingface "https://github.com/linus-berg/backpack/blob/master/Processor.HuggingFace/HuggingFace.cs"
click node_github "https://github.com/linus-berg/backpack/blob/master/Processor.Github.Releases/GithubReleases.cs"
click node_jetbrains "https://github.com/linus-berg/backpack/blob/master/Processor.Jetbrains.IDE/Jetbrains.cs"
click node_other_processors "https://github.com/linus-berg/backpack/blob/master/Processor.Rancher/Rancher.cs"
click node_router "https://github.com/linus-berg/backpack/blob/master/Collector.Router/Router.cs"
click node_collector_fleet "https://github.com/linus-berg/backpack/blob/master/Collector.Kernel/FileSystem.cs"
click node_http_collectors "https://github.com/linus-berg/backpack/blob/master/Collector.Wget/WebMirror.cs"
click node_git_collector "https://github.com/linus-berg/backpack/blob/master/Collector.Git/Git.cs"
click node_container_collector "https://github.com/linus-berg/backpack/blob/master/Collector.DockerArchive/Docker.cs"
click node_other_collectors "https://github.com/linus-berg/backpack/blob/master/Collector.Rsync/RSync.cs"
click node_filesystem "https://github.com/linus-berg/backpack/blob/master/Collector.Kernel/FileSystem.cs"
click node_s3 "https://github.com/linus-berg/backpack/blob/master/Collector.Kernel/Storage/Minio/MinioStorage.cs"
click node_mongo "https://github.com/linus-berg/backpack/blob/master/Core.Infrastructure/MongoDatabase.cs"
click node_redis "https://github.com/linus-berg/backpack/blob/master/Core.Infrastructure/CoreCache.cs"
click node_scheduler "https://github.com/linus-berg/backpack/blob/master/Tracker.Scheduler/ScheduleManager.cs"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_api,node_artifact_api,node_api_auth toneBlue
class node_gateway,node_gateway_consumers,node_router toneAmber
class node_processor_fleet,node_npm,node_nuget,node_pypi,node_containers,node_helm,node_terraform,node_huggingface,node_github,node_jetbrains,node_other_processors toneMint
class node_collector_fleet,node_http_collectors,node_git_collector,node_container_collector,node_other_collectors toneRose
class node_filesystem,node_s3,node_mongo,node_redis,node_scheduler,node_user,node_rabbitmq,node_upstream toneIndigo
```

## Components

### API and management

- [REST API](https://github.com/linus-berg/backpack/blob/master/Integration.API/Program.cs): `Integration.API/Program.cs`
- [Artifact operations](https://github.com/linus-berg/backpack/blob/master/Integration.API/Controllers/ArtifactController.cs): `Integration.API/Controllers/ArtifactController.cs`
- [API-key authentication](https://github.com/linus-berg/backpack/blob/master/Integration.API/ApiKeyAuthenticationHandler.cs): `Integration.API/ApiKeyAuthenticationHandler.cs`

### Artifact orchestration

- [Gateway processing](https://github.com/linus-berg/backpack/blob/master/Core.Gateway/GatewayProcessingService.cs): `Core.Gateway/GatewayProcessingService.cs`
- [Gateway consumers](https://github.com/linus-berg/backpack/tree/master/Core.Gateway/Consumers): `Core.Gateway/Consumers`
- [Collector router](https://github.com/linus-berg/backpack/blob/master/Collector.Router/Router.cs): `Collector.Router/Router.cs`

### Ecosystem processors

- [Ecosystem processors](https://github.com/linus-berg/backpack/tree/master/Processor.Npm): `Processor.Npm`
- [Npm processor](https://github.com/linus-berg/backpack/blob/master/Processor.Npm/Npm.cs): `Processor.Npm/Npm.cs`
- [NuGet processor](https://github.com/linus-berg/backpack/blob/master/Processor.Nuget/Nuget.cs): `Processor.Nuget/Nuget.cs`
- [PyPI processor](https://github.com/linus-berg/backpack/blob/master/Processor.Pypi/Pypi.cs): `Processor.Pypi/Pypi.cs`
- [Container processor](https://github.com/linus-berg/backpack/blob/master/Processor.Container/Worker.cs): `Processor.Container/Worker.cs`
- [Helm processor](https://github.com/linus-berg/backpack/blob/master/Processor.Helm/Helm.cs): `Processor.Helm/Helm.cs`
- [Terraform processor](https://github.com/linus-berg/backpack/blob/master/Processor.Terraform/Terraform.cs): `Processor.Terraform/Terraform.cs`
- [Hugging Face processor](https://github.com/linus-berg/backpack/blob/master/Processor.HuggingFace/HuggingFace.cs): `Processor.HuggingFace/HuggingFace.cs`
- [GitHub releases processor](https://github.com/linus-berg/backpack/blob/master/Processor.Github.Releases/GithubReleases.cs): `Processor.Github.Releases/GithubReleases.cs`
- [JetBrains processors](https://github.com/linus-berg/backpack/blob/master/Processor.Jetbrains.IDE/Jetbrains.cs): `Processor.Jetbrains.IDE/Jetbrains.cs`
- [Other ecosystem processors](https://github.com/linus-berg/backpack/blob/master/Processor.Rancher/Rancher.cs): `Processor.Rancher/Rancher.cs`

### File collectors

- [Protocol collectors](https://github.com/linus-berg/backpack/blob/master/Collector.Kernel/FileSystem.cs): `Collector.Kernel/FileSystem.cs`
- [HTTP and mirror](https://github.com/linus-berg/backpack/blob/master/Collector.Wget/WebMirror.cs): `Collector.Wget/WebMirror.cs`
- [Git collector](https://github.com/linus-berg/backpack/blob/master/Collector.Git/Git.cs): `Collector.Git/Git.cs`
- [Container collectors](https://github.com/linus-berg/backpack/blob/master/Collector.DockerArchive/Docker.cs): `Collector.DockerArchive/Docker.cs`
- [Other collectors](https://github.com/linus-berg/backpack/blob/master/Collector.Rsync/RSync.cs): `Collector.Rsync/RSync.cs`

### Persistence and scheduling

- [Storage filesystem](https://github.com/linus-berg/backpack/blob/master/Collector.Kernel/FileSystem.cs): `Collector.Kernel/FileSystem.cs`
- [S3-compatible storage](https://github.com/linus-berg/backpack/blob/master/Collector.Kernel/Storage/Minio/MinioStorage.cs): `Collector.Kernel/Storage/Minio/MinioStorage.cs`
- [MongoDB metadata](https://github.com/linus-berg/backpack/blob/master/Core.Infrastructure/MongoDatabase.cs): `Core.Infrastructure/MongoDatabase.cs`
- [Redis dependency cache](https://github.com/linus-berg/backpack/blob/master/Core.Infrastructure/CoreCache.cs): `Core.Infrastructure/CoreCache.cs`
- [Schedule tracking](https://github.com/linus-berg/backpack/blob/master/Tracker.Scheduler/ScheduleManager.cs): `Tracker.Scheduler/ScheduleManager.cs`

### Other components

- **Artifact submitter**
- **RabbitMQ**
- **Artifact registries**

## Connections

- Artifact submitter → REST API: submits artifact (evidence: `README.md`)
- REST API → API-key authentication: authenticates requests (evidence: `Integration.API/ApiKeyAuthenticationHandler.cs`)
- REST API → Artifact operations: routes artifact requests
- Artifact operations → MongoDB metadata: reads and writes metadata
- Artifact operations → RabbitMQ: queues work
- RabbitMQ → Gateway consumers: delivers messages
- Gateway consumers → Gateway processing: invokes processing
- Gateway processing → MongoDB metadata: stores changes (evidence: `README.md`)
- Gateway processing → Collector router: routes files (evidence: `README.md`)
- Gateway processing → Ecosystem processors: dispatches dependencies (evidence: `README.md`)
- Ecosystem processors → Artifact registries: resolves versions (evidence: `README.md`)
- Npm processor → Artifact registries: queries packages
- NuGet processor → Artifact registries: queries packages
- PyPI processor → Artifact registries: queries packages
- Container processor → Artifact registries: queries images
- Helm processor → Artifact registries: queries charts
- Terraform processor → Artifact registries: queries providers
- Hugging Face processor → Artifact registries: queries models
- GitHub releases processor → Artifact registries: queries releases
- JetBrains processors → Artifact registries: queries products
- Other ecosystem processors → Artifact registries: queries ecosystems
- Collector router → Protocol collectors: dispatches collection (evidence: `README.md`)
- Protocol collectors → HTTP and mirror: selects protocol collector
- Protocol collectors → Git collector: selects protocol collector
- Protocol collectors → Container collectors: selects protocol collector
- Protocol collectors → Other collectors: selects protocol collector
- Protocol collectors → Storage filesystem: stores collected files (evidence: `README.md`)
- Storage filesystem → S3-compatible storage: reads and writes objects (evidence: `Collector.Kernel/FileSystem.cs`)
- Schedule tracking → MongoDB metadata: reads schedules (evidence: `Tracker.Scheduler/ScheduleManager.cs`)
- Schedule tracking → RabbitMQ: retracks artifacts
- Gateway processing → Redis dependency cache: tracks dependencies