# paperswithcode/sotabench-api architecture

> Architecture diagram of the GitHub repository paperswithcode/sotabench-api, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/paperswithcode/sotabench-api
- Repository: https://github.com/paperswithcode/sotabench-api

## Overview

sotabench-api is a Python client and CLI for interacting with the SotaBench benchmarking service. Users configure credentials, then use CLI/API operations to validate benchmark metadata, manage repositories and builds, and list or upload datasets; benchmark results can be checked remotely or serialized locally. The graph shows documented capabilities whose command implementations were not excerpted, but omits unverified internal call links. The principal edges are supported by the sampled source and SOURCE INDEX; the service is external.

Main entry points: [`sotabenchapi/client.py`](https://github.com/paperswithcode/sotabench-api/blob/develop/sotabenchapi/client.py) (API client), [`sotabenchapi/uploader/upload.py`](https://github.com/paperswithcode/sotabench-api/blob/develop/sotabenchapi/uploader/upload.py) (Multipart upload), [`sotabenchapi/http.py`](https://github.com/paperswithcode/sotabench-api/blob/develop/sotabenchapi/http.py) (HTTP requests), [`sotabenchapi/commands/cli.py`](https://github.com/paperswithcode/sotabench-api/blob/develop/sotabenchapi/commands/cli.py) (CLI commands)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["CLI and configuration"]
  node_cli["CLI commands<br/>[cli.py]"]
  node_config["Profiles and settings<br/>[config.py]"]
  node_checkcmd["Setup validation<br/>[check.py]"]
end

subgraph group_benchmark["Benchmark results"]
  node_results["Benchmark result model<br/>[results.py]"]
  node_inputs["Benchmark inputs<br/>[inputs.py]"]
  node_checkmode["Check-mode utilities<br/>[check.py]"]
end

subgraph group_operations["Service operations"]
  node_repo["Repository management<br/>[repo.py]"]
  node_build["Build operations<br/>[build.py]"]
  node_dataset["Dataset management<br/>[dataset.py]"]
  node_benchcmd["Benchmark commands<br/>[benchmark.py]"]
end

subgraph group_transport["Transport and upload"]
  node_client["API client<br/>[client.py]"]
  node_http["HTTP requests<br/>[http.py]"]
  node_upload["Multipart upload<br/>[upload.py]"]
  node_uploadmodels["Upload parts<br/>[models.py]"]
  node_uploadutils["File hashing<br/>[utils.py]"]
  node_errors["API errors<br/>[errors.py]"]
end

node_user(("CLI user"))
node_service{{"SotaBench service"}}

node_user -->|"runs commands"| node_cli
node_cli -->|"loads profile"| node_config
node_cli -->|"authenticates"| node_client
node_checkcmd -.->|"validates parameters"| node_client
node_repo -->|"manages repositories"| node_client
node_build -->|"requests builds"| node_client
node_dataset -->|"manages datasets"| node_client
node_benchcmd -.->|"uses API"| node_client
node_client -->|"sends requests"| node_http
node_http -->|"calls API"| node_service
node_client -->|"delegates upload"| node_upload
node_upload -->|"starts and tracks"| node_http
node_upload -->|"transfers file parts"| node_service
node_upload -->|"serializes parts"| node_uploadmodels
node_upload -->|"hashes file data"| node_uploadutils
node_results -->|"reads mode"| node_checkmode
node_results -->|"checks remotely"| node_client
node_inputs -->|"uses result type"| node_results
node_http -->|"raises errors"| node_errors

click node_cli "https://github.com/paperswithcode/sotabench-api/blob/develop/sotabenchapi/commands/cli.py"
click node_config "https://github.com/paperswithcode/sotabench-api/blob/develop/sotabenchapi/config.py"
click node_checkcmd "https://github.com/paperswithcode/sotabench-api/blob/develop/sotabenchapi/commands/check.py"
click node_repo "https://github.com/paperswithcode/sotabench-api/blob/develop/sotabenchapi/commands/repo.py"
click node_build "https://github.com/paperswithcode/sotabench-api/blob/develop/sotabenchapi/commands/build.py"
click node_dataset "https://github.com/paperswithcode/sotabench-api/blob/develop/sotabenchapi/commands/dataset.py"
click node_benchcmd "https://github.com/paperswithcode/sotabench-api/blob/develop/sotabenchapi/commands/benchmark.py"
click node_client "https://github.com/paperswithcode/sotabench-api/blob/develop/sotabenchapi/client.py"
click node_http "https://github.com/paperswithcode/sotabench-api/blob/develop/sotabenchapi/http.py"
click node_upload "https://github.com/paperswithcode/sotabench-api/blob/develop/sotabenchapi/uploader/upload.py"
click node_uploadmodels "https://github.com/paperswithcode/sotabench-api/blob/develop/sotabenchapi/uploader/models.py"
click node_uploadutils "https://github.com/paperswithcode/sotabench-api/blob/develop/sotabenchapi/uploader/utils.py"
click node_results "https://github.com/paperswithcode/sotabench-api/blob/develop/sotabenchapi/core/results.py"
click node_inputs "https://github.com/paperswithcode/sotabench-api/blob/develop/sotabenchapi/core/inputs.py"
click node_checkmode "https://github.com/paperswithcode/sotabench-api/blob/develop/sotabenchapi/check.py"
click node_errors "https://github.com/paperswithcode/sotabench-api/blob/develop/sotabenchapi/errors.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_config,node_checkcmd,node_user toneBlue
class node_results,node_inputs,node_checkmode toneAmber
class node_repo,node_build,node_dataset,node_benchcmd toneMint
class node_client,node_http,node_upload,node_uploadmodels,node_uploadutils,node_errors toneRose
class node_service toneIndigo
```

## Components

### CLI and configuration

- [CLI commands](https://github.com/paperswithcode/sotabench-api/blob/develop/sotabenchapi/commands/cli.py): `sotabenchapi/commands/cli.py`
- [Profiles and settings](https://github.com/paperswithcode/sotabench-api/blob/develop/sotabenchapi/config.py): `sotabenchapi/config.py`
- [Setup validation](https://github.com/paperswithcode/sotabench-api/blob/develop/sotabenchapi/commands/check.py): `sotabenchapi/commands/check.py`

### Benchmark results

- [Benchmark result model](https://github.com/paperswithcode/sotabench-api/blob/develop/sotabenchapi/core/results.py): `sotabenchapi/core/results.py`
- [Benchmark inputs](https://github.com/paperswithcode/sotabench-api/blob/develop/sotabenchapi/core/inputs.py): `sotabenchapi/core/inputs.py`
- [Check-mode utilities](https://github.com/paperswithcode/sotabench-api/blob/develop/sotabenchapi/check.py): `sotabenchapi/check.py`

### Service operations

- [Repository management](https://github.com/paperswithcode/sotabench-api/blob/develop/sotabenchapi/commands/repo.py): `sotabenchapi/commands/repo.py`
- [Build operations](https://github.com/paperswithcode/sotabench-api/blob/develop/sotabenchapi/commands/build.py): `sotabenchapi/commands/build.py`
- [Dataset management](https://github.com/paperswithcode/sotabench-api/blob/develop/sotabenchapi/commands/dataset.py): `sotabenchapi/commands/dataset.py`
- [Benchmark commands](https://github.com/paperswithcode/sotabench-api/blob/develop/sotabenchapi/commands/benchmark.py): `sotabenchapi/commands/benchmark.py`

### Transport and upload

- [API client](https://github.com/paperswithcode/sotabench-api/blob/develop/sotabenchapi/client.py): `sotabenchapi/client.py`
- [HTTP requests](https://github.com/paperswithcode/sotabench-api/blob/develop/sotabenchapi/http.py): `sotabenchapi/http.py`
- [Multipart upload](https://github.com/paperswithcode/sotabench-api/blob/develop/sotabenchapi/uploader/upload.py): `sotabenchapi/uploader/upload.py`
- [Upload parts](https://github.com/paperswithcode/sotabench-api/blob/develop/sotabenchapi/uploader/models.py): `sotabenchapi/uploader/models.py`
- [File hashing](https://github.com/paperswithcode/sotabench-api/blob/develop/sotabenchapi/uploader/utils.py): `sotabenchapi/uploader/utils.py`
- [API errors](https://github.com/paperswithcode/sotabench-api/blob/develop/sotabenchapi/errors.py): `sotabenchapi/errors.py`

### Other components

- **CLI user**
- **SotaBench service**

## Connections

- CLI user → CLI commands: runs commands (evidence: `sotabenchapi/commands/cli.py`)
- CLI commands → Profiles and settings: loads profile (evidence: `sotabenchapi/commands/cli.py`)
- CLI commands → API client: authenticates (evidence: `sotabenchapi/commands/cli.py`)
- Setup validation → API client: validates parameters
- Repository management → API client: manages repositories (evidence: `sotabenchapi/commands/repo.py`)
- Build operations → API client: requests builds (evidence: `sotabenchapi/commands/build.py`)
- Dataset management → API client: manages datasets (evidence: `sotabenchapi/commands/dataset.py`)
- Benchmark commands → API client: uses API
- API client → HTTP requests: sends requests (evidence: `sotabenchapi/client.py`)
- HTTP requests → SotaBench service: calls API (evidence: `sotabenchapi/http.py`)
- API client → Multipart upload: delegates upload (evidence: `sotabenchapi/client.py`)
- Multipart upload → HTTP requests: starts and tracks (evidence: `sotabenchapi/uploader/upload.py`)
- Multipart upload → SotaBench service: transfers file parts (evidence: `sotabenchapi/uploader/upload.py`)
- Multipart upload → Upload parts: serializes parts (evidence: `sotabenchapi/uploader/upload.py`)
- Multipart upload → File hashing: hashes file data (evidence: `sotabenchapi/uploader/upload.py`)
- Benchmark result model → Check-mode utilities: reads mode (evidence: `sotabenchapi/core/results.py`)
- Benchmark result model → API client: checks remotely (evidence: `sotabenchapi/core/results.py`)
- Benchmark inputs → Benchmark result model: uses result type (evidence: `sotabenchapi/core/inputs.py`)
- HTTP requests → API errors: raises errors (evidence: `sotabenchapi/http.py`)
