# brain-score/vision architecture

> Architecture diagram of the GitHub repository brain-score/vision, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/brain-score/vision
- Repository: https://github.com/brain-score/vision

## Overview

##### Key architectural components
- Core Package: The central code is in the “brainscore_vision” directory. This package contains submodules for benchmarks, data handling, metrics, model helpers, model interface definitions, submission endpoints, and utilities.
- Benchmark Engine: The “benchmarks” subdirectory organizes different experimental benchmarks (e.g. “MajajHong2015public”, “Baker2022”, etc.). This is where benchmark definitions, testing scripts, and configuration for running experiments are defined.
- Data Layer: The “data” directory holds code for assembling, pre-processing, and packaging experimental data. It is the input source to the benchmark engine.
- Metrics: The “metrics” module contains various scoring methods (e.g., accuracy, error consistency, distribution similarity) that quantify the match between the model’s output and brain data.
- Model Helpers & Interfaces: The “model_helpers” folder (with submodules for activations, temporal processing, brain transformation, etc.) supports model evaluation. The “model_interface.py” file defines the required abstraction (BrainModel interface) that candidate models must implement.
- Submission and Integration: The “submission” folder and accompanying scripts (and files in the root such as “setup.py”, “pyproject.toml”, etc.) provide the means to package models, integrate with caching (see the “result_caching” references in README), and expose endpoints for model submission and testing.
- External Integration and CI/CD: The “.github” directory contains workflow configurations (e.g. for automerge, version bumping, integration testing) and continuous integration scripts that ensure integrity over the repository.
- Documentation & Examples: The “docs” and “examples” directories serve external documentation (and live notebooks) that outline usage, integration examples, and tutorials on scoring models.
- Testing: The “tests” directory has unit and integration tests in separate subfolders for benchmarking, metric helpers, model helpers, submission endpoints, etc.

##### Relationships and data flows between components
- The Data Providers module extracts and formats experimental stimuli and neural recordings (from the “data” folder) and passes these to the benchmark definitions.
- The Benchmark Engine (within “benchmarks”) orchestrates the evaluation by loading a candidate model (which adheres to “BrainModel”) via the Model Interface and then running the model on prepared input data.
- The model’s output is “post-processed” using Model Helpers (activations, temporal processing) before being compared against experimental data with Metrics modules (accuracy, CKA, etc.).
- Intermediate results, scores, and activations are cached (using the result caching mechanism configured by environment variables such as RESULTCACHING_HOME) to avoid redundant computations.
- There is a submission pipeline (in “submission”) that possibly integrates with an external website (brain-score.org) for full benchmark evaluation.
- The CI/CD pipelines (under “.github/workflows”) ensure that updates in any of these components are automatically tested and documented.
- Documentation and examples act as the user-facing component that explains how full-stack benchmarking is performed (linking conceptual elements to code examples).

##### Relevant technologies and design patterns
- The project is implemented in Python (requiring Python 3.11+ as specified).
- It leverages a modular design where each functional domain (data, processing, metrics) is cleanly separated.
- The caching pattern (using a dedicated result caching library) is essential to optimize repeated benchmark evaluations.
- The use of interfaces (BrainModel interface) ensures extensibility: new models need only abide by the standard interface.
- CI/CD and version management scripts ensure reproducibility and continuous integration of benchmarks and models.

## Diagram (Mermaid)

```mermaid
graph TD
    subgraph "BrainScore Core Package"
        Data["Data Providers"]:::data
        Bench["Benchmark Engine"]:::benchmark
        MI["Model Interface"]:::interface
        MH["Model Helpers"]:::helper
        Met["Metrics"]:::metrics
    end

    Caching(("Caching System")):::cache
    Sub["Submission & Integration"]:::submission
    CID["CI/CD Pipelines"]:::ci
    Test["Testing"]:::testing
    Docs["Documentation"]:::docs
    Ex["Examples"]:::docs
    Ext["External Integration (brain-score.org)"]:::external

    Data -->|"feeds"| Bench
    Bench -->|"loadsModel"| MI
    Bench -->|"processesOutput"| MH
    MH -->|"preparesScoring"| Met
    Met -->|"storesResults"| Caching
    Bench -->|"submitsResults"| Sub
    Sub -->|"integratesWith"| Ext
    CID -->|"triggers"| Test
    Docs -->|"guidesUsage"| Bench
    Ex -->|"guidesUsage"| Bench

    click Data "https://github.com/brain-score/vision/tree/master/brainscore_vision/data/"
    click Bench "https://github.com/brain-score/vision/tree/master/brainscore_vision/benchmarks/"
    click MI "https://github.com/brain-score/vision/blob/master/brainscore_vision/model_interface.py"
    click MH "https://github.com/brain-score/vision/tree/master/brainscore_vision/model_helpers/"
    click Met "https://github.com/brain-score/vision/tree/master/brainscore_vision/metrics/"
    click Sub "https://github.com/brain-score/vision/tree/master/brainscore_vision/submission/"
    click CID "https://github.com/brain-score/vision/tree/master/.github/"
    click Docs "https://github.com/brain-score/vision/tree/master/docs/"
    click Ex "https://github.com/brain-score/vision/tree/master/examples/"
    click Test "https://github.com/brain-score/vision/tree/master/tests/"

    classDef data fill:#AED6F1,stroke:#1B4F72,stroke-width:2px;
    classDef benchmark fill:#F9E79F,stroke:#9A7D0A,stroke-width:2px;
    classDef interface fill:#D6EAF8,stroke:#1B4F72,stroke-width:2px;
    classDef helper fill:#FCF3CF,stroke:#B7950B,stroke-width:2px;
    classDef metrics fill:#FADBD8,stroke:#922B21,stroke-width:2px;
    classDef submission fill:#D5F5E3,stroke:#196F3D,stroke-width:2px;
    classDef ci fill:#F0B27A,stroke:#7D6608,stroke-width:2px;
    classDef testing fill:#D7BDE2,stroke:#6C3483,stroke-width:2px;
    classDef docs fill:#E8DAEF,stroke:#512E5F,stroke-width:2px;
    classDef external fill:#F5B7B1,stroke:#943126,stroke-width:2px;
    classDef cache fill:#ABEBC6,stroke:#145A32,stroke-width:2px;
```
