# tensorbit-labs/tensorbit-core architecture

> Architecture diagram of the GitHub repository tensorbit-labs/tensorbit-core, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-05-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tensorbit-labs/tensorbit-core
- Repository: https://github.com/tensorbit-labs/tensorbit-core

## Overview

1. **Overall shape**
   Tensorbit Core is a single C++20 codebase with an optional CUDA acceleration layer. Its main job is to ingest model weights, compute pruning importance scores, apply structured sparsity, and serialize the result into a compact `.tb` artifact for downstream stages.

2. **Primary pipeline**
   The central data flow is:
   **`.safetensors` model weights → loader/parse → EHAP scoring → CORING mask selection/pruning → `.tb` serialization**
   This is the core architecture to reflect in a diagram; everything else supports one of these stages.

3. **Loader boundary**
   `include/tensorbit/core/loader.hpp` and related code provide the model ingestion boundary. The README says it supports header-only parsing of HuggingFace safetensors weights, including F32/F16/BF16/I64. This is the input interface between external model files and Tensorbit’s internal tensor representation.

4. **Tensor and common core types**
   `tensor.hpp`, `common.hpp`, and `serialization.hpp` define the shared in-memory abstractions and utility types used across the pipeline. These are the “internal contract” layer between parsing, scoring, pruning, and output. The repo emphasizes modern C++ utilities such as `Result<T,E>`, spans, and concepts.

5. **EHAP subsystem: importance estimation**
   `ehap.hpp` / `src/ehap.cpp` implement the Hessian-aware pruning logic. This is the analytical stage that estimates parameter importance using second-order information, Fisher/EMA-style approximation, and OBS/OBD-style scoring. Architecturally, it consumes tensors and gradient/statistical state, then emits importance scores or pruning decisions.

6. **CORING subsystem: structured sparsity enforcement**
   `coring.hpp` / `src/coring.cpp` implement N:M structured sparsity selection and refinement. This is the masking/pruning enforcement boundary: it converts importance information into a hardware-friendly sparse pattern, especially 2:4. It is the stage that transforms “which weights matter” into “which weights remain.”

7. **CUDA execution layer**
   `kernels.hpp`, `src/kernels.cu`, and `src/kernels_stubs.cpp` form the acceleration boundary. The codebase supports GPU-optimized kernels when CUDA is enabled, with CPU-only fallback via stub implementations. For architecture diagrams, treat this as an optional compute backend under the pruning/masking stages, not as a separate product surface.

8. **Serialization/output boundary**
   `src/serialization.cpp` and `include/tensorbit/core/serialization.hpp` convert the pruned model into the `.tb` format. The README highlights a fixed 4096-byte header, packed N:M bitmasks, and FP32/FP16/BF16 storage. This is the persistence boundary between Tensorbit Core and later pipeline stages.

9. **CLI / orchestration entry point**
   `src/main.cpp` is the executable entry point for the pruning tool. It likely coordinates model loading, optional mock/demo sizing, algorithm selection, pruning execution, and output writing. In the architecture diagram, this is the top-level control plane that sequences the pipeline stages.

10. **Build and platform model**
   The project uses CMake and is designed for portable C++20 builds with optional CUDA 12 support. The dependency footprint is intentionally small: Eigen3 on the CPU side and CUDA on supported systems. That makes the repository a self-contained engine rather than a distributed service.

11. **Testing structure**
   Tests are split by subsystem: `tests/test_coring.cpp` and `tests/test_ehap.cpp` cover the algorithmic cores, while `tests/merge` and `tests/multi_tensor` exercise serialization/model-composition workflows via scripts. This suggests the repo validates both mathematical correctness and end-to-end format behavior.

12. **Utility scripts and operational aids**
   `scripts/download_model.py`, `fix_shapes.py`, `merge_tbm.py`, and setup scripts support data preparation and environment setup rather than the core runtime. They sit around the engine to make local and cloud workflows practical, especially for large-model experiments.

13. **Documentation as architecture source**
   `docs/ARCHITECTURE.md`, `docs/ALGORITHMS.md`, `docs/EHAP.md`, and `docs/CORING.md` indicate that the repo is split into clear mathematical, format, and systems concerns. The documentation mirrors the code boundaries and is a good source for naming the diagram’s main subsystems.

14. **What to draw as the major boundaries**
   The most important architectural boundaries are:
   - **External model format ingestion** vs **internal tensor representation**
   - **EHAP scoring** vs **CORING mask selection**
   - **CPU fallback** vs **CUDA accelerated kernels**
   - **internal pruned model state** vs **serialized `.tb` artifact**
   - **core library code** vs **test and utility tooling**

Main entry points: [`src/coring.cpp`](https://github.com/tensorbit-labs/tensorbit-core/blob/main/src/coring.cpp) (CORING impl), [`include/tensorbit/core/ehap.hpp`](https://github.com/tensorbit-labs/tensorbit-core/blob/main/include/tensorbit/core/ehap.hpp) (EHAP), [`include/tensorbit/core/kernels.hpp`](https://github.com/tensorbit-labs/tensorbit-core/blob/main/include/tensorbit/core/kernels.hpp) (Kernels), [`include/tensorbit/core/loader.hpp`](https://github.com/tensorbit-labs/tensorbit-core/blob/main/include/tensorbit/core/loader.hpp) (Loader)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_grp_core["Core pipeline"]
  node_cli["CLI<br/>entrypoint<br/>[main.cpp]"]
  node_loader["Loader<br/>ingest<br/>[loader.hpp]"]
  node_tensor["Tensor core<br/>types<br/>[tensor.hpp]"]
  node_common["Common types<br/>contracts<br/>[common.hpp]"]
  node_ehap{{"EHAP<br/>scoring<br/>[ehap.hpp]"}}
  node_ehap_impl["EHAP impl<br/>analysis<br/>[ehap.cpp]"]
  node_coring{{"CORING<br/>masking<br/>[coring.hpp]"}}
  node_coring_impl["CORING impl<br/>pruning<br/>[coring.cpp]"]
  node_serialization["Serializer<br/>output<br/>[serialization.hpp]"]
  node_serialization_impl["TB writer<br/>formatting<br/>[serialization.cpp]"]
end

subgraph group_grp_backend["Execution backend"]
  node_kernels["Kernels<br/>backend-api<br/>[kernels.hpp]"]
  node_kernels_cuda["CUDA kernels<br/>gpu<br/>[kernels.cu]"]
  node_kernels_stub["CPU stubs<br/>fallback<br/>[kernels_stubs.cpp]"]
end

subgraph group_grp_tooling["Tooling and validation"]
  node_tests_ehap["EHAP tests<br/>[test_ehap.cpp]"]
  node_tests_coring["CORING tests<br/>[test_coring.cpp]"]
  node_scripts["Scripts<br/>ops<br/>[download_model.py]"]
  node_docs["Docs<br/>specification<br/>[ARCHITECTURE.md]"]
end

node_cli -->|"loads"| node_loader
node_loader -->|"produces"| node_tensor
node_tensor -->|"feeds"| node_ehap
node_common -->|"supports"| node_ehap
node_ehap -->|"implemented by"| node_ehap_impl
node_ehap_impl -->|"scores"| node_coring
node_coring -->|"implemented by"| node_coring_impl
node_coring_impl -->|"accelerates via"| node_kernels
node_kernels -->|"gpu path"| node_kernels_cuda
node_kernels -->|"fallback"| node_kernels_stub
node_coring_impl -->|"outputs"| node_serialization
node_serialization -->|"implemented by"| node_serialization_impl
node_serialization_impl -->|"writes"| node_cli
node_tests_ehap -.->|"validates"| node_ehap_impl
node_tests_coring -.->|"validates"| node_coring_impl
node_scripts -.->|"prepares input"| node_loader
node_docs -.->|"describes"| node_ehap
node_docs -.->|"describes"| node_coring

click node_cli "https://github.com/tensorbit-labs/tensorbit-core/blob/main/src/main.cpp"
click node_loader "https://github.com/tensorbit-labs/tensorbit-core/blob/main/include/tensorbit/core/loader.hpp"
click node_tensor "https://github.com/tensorbit-labs/tensorbit-core/blob/main/include/tensorbit/core/tensor.hpp"
click node_common "https://github.com/tensorbit-labs/tensorbit-core/blob/main/include/tensorbit/core/common.hpp"
click node_ehap "https://github.com/tensorbit-labs/tensorbit-core/blob/main/include/tensorbit/core/ehap.hpp"
click node_ehap_impl "https://github.com/tensorbit-labs/tensorbit-core/blob/main/src/ehap.cpp"
click node_coring "https://github.com/tensorbit-labs/tensorbit-core/blob/main/include/tensorbit/core/coring.hpp"
click node_coring_impl "https://github.com/tensorbit-labs/tensorbit-core/blob/main/src/coring.cpp"
click node_serialization "https://github.com/tensorbit-labs/tensorbit-core/blob/main/include/tensorbit/core/serialization.hpp"
click node_serialization_impl "https://github.com/tensorbit-labs/tensorbit-core/blob/main/src/serialization.cpp"
click node_kernels "https://github.com/tensorbit-labs/tensorbit-core/blob/main/include/tensorbit/core/kernels.hpp"
click node_kernels_cuda "https://github.com/tensorbit-labs/tensorbit-core/blob/main/src/kernels.cu"
click node_kernels_stub "https://github.com/tensorbit-labs/tensorbit-core/blob/main/src/kernels_stubs.cpp"
click node_tests_ehap "https://github.com/tensorbit-labs/tensorbit-core/blob/main/tests/test_ehap.cpp"
click node_tests_coring "https://github.com/tensorbit-labs/tensorbit-core/blob/main/tests/test_coring.cpp"
click node_scripts "https://github.com/tensorbit-labs/tensorbit-core/blob/main/scripts/download_model.py"
click node_docs "https://github.com/tensorbit-labs/tensorbit-core/blob/main/docs/ARCHITECTURE.md"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_loader,node_tensor,node_common,node_ehap,node_ehap_impl,node_coring,node_coring_impl,node_serialization,node_serialization_impl toneBlue
class node_kernels,node_kernels_cuda,node_kernels_stub toneAmber
class node_tests_ehap,node_tests_coring,node_scripts,node_docs toneMint
```

## Components

### Core pipeline

Primary model ingestion, scoring, pruning, and serialization flow.

- [CLI](https://github.com/tensorbit-labs/tensorbit-core/blob/main/src/main.cpp): entrypoint · Top-level pruning tool that orchestrates the pipeline. · `src/main.cpp`
- [Loader](https://github.com/tensorbit-labs/tensorbit-core/blob/main/include/tensorbit/core/loader.hpp): ingest · Boundary for parsing safetensors model weights into internal tensors. · `include/tensorbit/core/loader.hpp`
- [Tensor core](https://github.com/tensorbit-labs/tensorbit-core/blob/main/include/tensorbit/core/tensor.hpp): types · Shared tensor and utility abstractions used across the pipeline. · `include/tensorbit/core/tensor.hpp`
- [Common types](https://github.com/tensorbit-labs/tensorbit-core/blob/main/include/tensorbit/core/common.hpp): contracts · Result, span, concept, and shared helper definitions. · `include/tensorbit/core/common.hpp`
- [EHAP](https://github.com/tensorbit-labs/tensorbit-core/blob/main/include/tensorbit/core/ehap.hpp): scoring · Hessian-aware importance estimation and pruning score computation. · `include/tensorbit/core/ehap.hpp`
- [EHAP impl](https://github.com/tensorbit-labs/tensorbit-core/blob/main/src/ehap.cpp): analysis · Implementation of second-order scoring and statistical approximation. · `src/ehap.cpp`
- [CORING](https://github.com/tensorbit-labs/tensorbit-core/blob/main/include/tensorbit/core/coring.hpp): masking · Structured sparsity selection and refinement boundary. · `include/tensorbit/core/coring.hpp`
- [CORING impl](https://github.com/tensorbit-labs/tensorbit-core/blob/main/src/coring.cpp): pruning · Implementation of N:M mask selection and pruning refinement. · `src/coring.cpp`
- [Serializer](https://github.com/tensorbit-labs/tensorbit-core/blob/main/include/tensorbit/core/serialization.hpp): output · Converts pruned model state into compact .tb artifacts. · `include/tensorbit/core/serialization.hpp`
- [TB writer](https://github.com/tensorbit-labs/tensorbit-core/blob/main/src/serialization.cpp): formatting · Implementation of header packing, mask encoding, and weight emission. · `src/serialization.cpp`

### Execution backend

Optional compute acceleration layer for pruning kernels.

- [Kernels](https://github.com/tensorbit-labs/tensorbit-core/blob/main/include/tensorbit/core/kernels.hpp): backend-api · Acceleration boundary for compute kernels used by pruning stages. · `include/tensorbit/core/kernels.hpp`
- [CUDA kernels](https://github.com/tensorbit-labs/tensorbit-core/blob/main/src/kernels.cu): gpu · GPU implementation of accelerated compute paths. · `src/kernels.cu`
- [CPU stubs](https://github.com/tensorbit-labs/tensorbit-core/blob/main/src/kernels_stubs.cpp): fallback · CPU fallback implementations when CUDA is unavailable. · `src/kernels_stubs.cpp`

### Tooling and validation

Supporting scripts, tests, and documentation around the engine.

- [EHAP tests](https://github.com/tensorbit-labs/tensorbit-core/blob/main/tests/test_ehap.cpp): test · Unit coverage for importance estimation logic. · `tests/test_ehap.cpp`
- [CORING tests](https://github.com/tensorbit-labs/tensorbit-core/blob/main/tests/test_coring.cpp): test · Unit coverage for structured sparsity and pruning selection. · `tests/test_coring.cpp`
- [Scripts](https://github.com/tensorbit-labs/tensorbit-core/blob/main/scripts/download_model.py): ops · Workflow helpers for model download, shape fixing, merge, and setup. · `scripts/download_model.py`
- [Docs](https://github.com/tensorbit-labs/tensorbit-core/blob/main/docs/ARCHITECTURE.md): specification · Architecture and algorithm documentation that mirrors the code boundaries. · `docs/ARCHITECTURE.md`

## Connections

- CLI → Loader: loads · CLI starts by ingesting source model weights.
- Loader → Tensor core: produces · Parsed weights become internal tensor objects.
- Tensor core → EHAP: feeds · Tensor state is consumed by importance estimation.
- Common types → EHAP: supports · Shared contracts underpin the analysis stage.
- EHAP → EHAP impl: implemented by · Public scoring boundary maps to the EHAP engine.
- EHAP impl → CORING: scores · Importance estimates drive sparsity selection.
- CORING → CORING impl: implemented by · Public masking boundary maps to CORING logic.
- CORING impl → Kernels: accelerates via · Pruning can dispatch into backend kernels.
- Kernels → CUDA kernels: gpu path · CUDA backend is used when enabled.
- Kernels → CPU stubs: fallback · CPU stubs provide the non-CUDA path.
- CORING impl → Serializer: outputs · Pruned weights are prepared for serialization.
- Serializer → TB writer: implemented by · Output boundary maps to TB format writer.
- TB writer → CLI: writes · CLI completes the pipeline by emitting the artifact.
- EHAP tests → EHAP impl: validates · EHAP tests check analytical scoring behavior.
- CORING tests → CORING impl: validates · CORING tests check structured sparsity behavior.
- Scripts → Loader: prepares input · Scripts help obtain and reshape source models.
- Docs → EHAP: describes · Documentation captures the analytical subsystem.
- Docs → CORING: describes · Documentation captures the sparsity subsystem.
