# sysprog21/vpipe architecture

> Architecture diagram of the GitHub repository sysprog21/vpipe, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-05-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sysprog21/vpipe
- Repository: https://github.com/sysprog21/vpipe

## Overview

1. **What this repo is**
   - `vpipe` is a Linux-only V4L2 mem2mem prototype, not a vision stack.
   - Its goal is to measure frame-path costs: copies, queueing, context switches, scheduler jitter, cache behavior, and a small deterministic kernel-side image transform.

2. **Top-level architecture**
   - The repo splits into three main areas:
     - `kmod/` — kernel module implementation
     - `user/` — userspace capture, benchmark, validation, and reference tools
     - `scripts/`, `tests/`, `docs/` — automation, fixtures, and design/benchmark documentation
   - The architecture is intentionally narrow: single-plane `V4L2_PIX_FMT_GREY`, 640×480, one mem2mem node, and one metadata sideband device.

3. **Kernel transport path**
   - `kmod/vpipe-m2m.c` implements the V4L2 mem2mem device with OUTPUT and CAPTURE queues.
   - Frames arrive on OUTPUT via either `V4L2_MEMORY_DMABUF` or `V4L2_MEMORY_MMAP`, are processed in `device_run()`, then emitted on CAPTURE.
   - This is the core transport boundary the repo is measuring.

4. **Kernel image-processing path**
   - `kmod/vpipe-cv.c` holds the “Tiny CV” logic: a bounded threshold operation over a clamped ROI.
   - The algorithm is deliberately simple and deterministic, with no floating point and no hot-path allocation.
   - This keeps algorithm cost small and auditable so transport effects remain visible.

5. **Kernel metadata sideband**
   - `kmod/vpipe-meta.c` exposes `/dev/vpipe-meta` as a miscdevice carrying per-frame metadata.
   - Metadata is stored in a shared ring buffer with per-open reader cursors, so multiple readers can drain it independently.
   - This sideband avoids overloading the pixel payload and lets timing/state be correlated frame-by-frame.

6. **Shared UAPI boundary**
   - `kmod/vpipe.h` defines the user/kernel contract: ioctls, control IDs, and metadata layout.
   - Per-buffer controls such as source sequence, algorithm selection, threshold, and ROI are snapshotted at QBUF time.
   - That design prevents control races for frames already in flight.

7. **Userspace orchestration**
   - The `user/` programs drive the scenarios rather than implement the kernel logic.
   - Key binaries include:
     - `vpipe-capture-*` for baseline capture modes (`read`, `mmap`, `dmabuf`, `m2m`)
     - `vpipe-bench-fixture` for deterministic fixture transport runs
     - `vpipe-meta-drain` to collect metadata CSV
     - `vpipe-cv-ref` and `vpipe-pgm-diff` for reference comparison
   - Userspace owns sequence correlation, CSV logging, and artifact capture.

8. **Fixture-driven validation path**
   - The deterministic path uses fixture PGM files from `tests/fixtures/`.
   - `vpipe-fixture-feed` and the benchmark scripts load fixture bytes into `dma-heap` memory, then transport them via DMABUF into vpipe.
   - That intentionally includes one explicit userspace `memcpy()` so copy counts stay measurable and there are no accidental zero-copy claims.

9. **Measurement and artifact flow**
   - Benchmark scripts in `scripts/` run capture loops, perf collection, tracing, and summary generation.
   - Outputs go into a gitignored `bench/` directory:
     - per-frame CSV logs
     - metadata CSV sidecars
     - perf CSVs
     - input/reference/kernel/diff PGM sets for visual inspection
   - The main validation loop is: fixture → userspace reference → kernel output → diff.

10. **Validation and CI boundary**
   - `make` builds both kernel and userspace; `sudo make check` runs the validation suite.
   - The suite covers unit tests, vivid enumeration, module load, metadata sanity, capture baselines, and full Tiny CV fixture checks.
   - GitHub Actions and the `.ci/` scripts enforce formatting, security, newline, and cppcheck gates.

11. **External runtime dependencies that matter architecturally**
   - The repo depends on Linux V4L2, `vivid` for virtual camera input, `dma-heap` for deterministic buffer allocation, and `perf` for measurement.
   - The documented reference environment is an Ubuntu lima guest, which matters because some kernel features and PMU counters are absent there.
   - Phase 5 explicit-sync validation is deliberately gated on V4L2 UAPI availability.

12. **How to draw the system**
   - Draw a userspace layer that prepares sources, drives V4L2 ioctls, drains metadata, and writes benchmark artifacts.
   - Draw a kernel layer with three distinct parts: mem2mem transport, tiny CV processing, and metadata ring buffer.
   - Show two distinct data planes: pixel buffers through OUTPUT→processing→CAPTURE, and metadata through `/dev/vpipe-meta`.
   - Highlight the explicit boundary between measurement/orchestration in userspace and deterministic frame handling in the kernel.

Main entry points: [`kmod/vpipe-m2m.c`](https://github.com/sysprog21/vpipe/blob/main/kmod/vpipe-m2m.c) (M2M device), [`user/vpipe-bench-fixture.c`](https://github.com/sysprog21/vpipe/blob/main/user/vpipe-bench-fixture.c) (Bench driver), [`scripts/bench_vpipe.sh`](https://github.com/sysprog21/vpipe/blob/main/scripts/bench_vpipe.sh) (Bench scripts), [`tests/validate-tinycv.sh`](https://github.com/sysprog21/vpipe/blob/main/tests/validate-tinycv.sh) (Validation suite)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_g_userspace["Userspace"]
  node_n_u_common["Shared API<br/>UAPI helpers<br/>[vpipe-common.h]"]
  node_n_u_capture["Capture modes<br/>capture tools"]
  node_n_u_fixture["Fixture feed<br/>fixture runner"]
  node_n_u_bench["Bench driver<br/>benchmark harness"]
  node_n_u_meta["Meta drain<br/>metadata tool<br/>[vpipe-meta-drain.c]"]
  node_n_u_ref["CV reference<br/>reference tool<br/>[vpipe-cv-ref.c]"]
  node_n_u_diff["PGM diff<br/>comparison tool<br/>[vpipe-pgm-diff.c]"]
end

subgraph group_g_kernel["Kernel"]
  node_n_k_m2m["M2M device<br/>v4l2 mem2mem<br/>[vpipe-m2m.c]"]
  node_n_k_cv["Tiny CV<br/>kernel transform<br/>[vpipe-cv.c]"]
  node_n_k_meta["Meta ring<br/>miscdevice<br/>[vpipe-meta.c]"]
  node_n_k_uapi["Kernel UAPI<br/>contract header<br/>[vpipe.h]"]
  node_n_k_internal["Internal state<br/>kernel internals<br/>[vpipe-internal.h]"]
end

subgraph group_g_validation["Validation"]
  node_n_fixtures["PGM fixtures<br/>test inputs<br/>[edge-block.pgm]"]
  node_n_scripts_bench["Bench scripts<br/>automation<br/>[bench_vpipe.sh]"]
  node_n_scripts_trace["Trace tools<br/>automation<br/>[trace_v4l2.sh]"]
  node_n_scripts_sum["Summarizer<br/>reporting"]
  node_n_tests["Validation suite<br/>test harness<br/>[validate-tinycv.sh]"]
  node_n_docs["Design docs<br/>[design.md]"]
  node_n_ci["CI gates<br/>ci checks<br/>[main.yml]"]
end

node_n_u_capture -->|"V4L2 ioctls"| node_n_k_m2m
node_n_u_fixture -->|"DMABUF input"| node_n_k_m2m
node_n_u_bench -->|"runs"| node_n_u_fixture
node_n_u_bench -->|"drains"| node_n_u_meta
node_n_u_ref -->|"compares via"| node_n_u_diff
node_n_u_common -->|"shared contract"| node_n_k_uapi
node_n_k_m2m -->|"device_run"| node_n_k_cv
node_n_k_m2m -->|"frame metadata"| node_n_k_meta
node_n_k_m2m -->|"controls"| node_n_k_uapi
node_n_k_cv -->|"uses"| node_n_k_internal
node_n_k_meta -->|"ring state"| node_n_k_internal
node_n_u_meta -->|"reads /dev/vpipe-meta"| node_n_k_meta
node_n_fixtures -->|"source data"| node_n_u_fixture
node_n_scripts_bench -->|"orchestrates"| node_n_u_capture
node_n_scripts_bench -->|"orchestrates"| node_n_u_bench
node_n_scripts_trace -->|"traces"| node_n_k_m2m
node_n_scripts_sum -->|"summarizes"| node_n_scripts_bench
node_n_tests -->|"validates"| node_n_u_ref
node_n_tests -->|"loads module"| node_n_k_m2m
node_n_docs -->|"describes"| node_n_k_m2m
node_n_ci -->|"gates"| node_n_tests

click node_n_u_common "https://github.com/sysprog21/vpipe/blob/main/user/vpipe-common.h"
click node_n_u_capture "https://github.com/sysprog21/vpipe/blob/main/user/vpipe-capture-read.c"
click node_n_u_fixture "https://github.com/sysprog21/vpipe/blob/main/user/vpipe-fixture-feed.c"
click node_n_u_bench "https://github.com/sysprog21/vpipe/blob/main/user/vpipe-bench-fixture.c"
click node_n_u_meta "https://github.com/sysprog21/vpipe/blob/main/user/vpipe-meta-drain.c"
click node_n_u_ref "https://github.com/sysprog21/vpipe/blob/main/user/vpipe-cv-ref.c"
click node_n_u_diff "https://github.com/sysprog21/vpipe/blob/main/user/vpipe-pgm-diff.c"
click node_n_k_m2m "https://github.com/sysprog21/vpipe/blob/main/kmod/vpipe-m2m.c"
click node_n_k_cv "https://github.com/sysprog21/vpipe/blob/main/kmod/vpipe-cv.c"
click node_n_k_meta "https://github.com/sysprog21/vpipe/blob/main/kmod/vpipe-meta.c"
click node_n_k_uapi "https://github.com/sysprog21/vpipe/blob/main/kmod/vpipe.h"
click node_n_k_internal "https://github.com/sysprog21/vpipe/blob/main/kmod/vpipe-internal.h"
click node_n_fixtures "https://github.com/sysprog21/vpipe/blob/main/tests/fixtures/edge-block.pgm"
click node_n_scripts_bench "https://github.com/sysprog21/vpipe/blob/main/scripts/bench_vpipe.sh"
click node_n_scripts_trace "https://github.com/sysprog21/vpipe/blob/main/scripts/trace_v4l2.sh"
click node_n_scripts_sum "https://github.com/sysprog21/vpipe/blob/main/scripts/summarize-benchmark.py"
click node_n_tests "https://github.com/sysprog21/vpipe/blob/main/tests/validate-tinycv.sh"
click node_n_docs "https://github.com/sysprog21/vpipe/blob/main/docs/design.md"
click node_n_ci "https://github.com/sysprog21/vpipe/blob/main/.github/workflows/main.yml"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_n_u_common,node_n_u_capture,node_n_u_fixture,node_n_u_bench,node_n_u_meta,node_n_u_ref,node_n_u_diff toneBlue
class node_n_k_m2m,node_n_k_cv,node_n_k_meta,node_n_k_uapi,node_n_k_internal toneAmber
class node_n_fixtures,node_n_scripts_bench,node_n_scripts_trace,node_n_scripts_sum,node_n_tests,node_n_docs,node_n_ci toneMint
```

## Components

### Userspace

Programs that orchestrate V4L2 flows, reference comparison, and artifact capture.

- [Shared API](https://github.com/sysprog21/vpipe/blob/main/user/vpipe-common.h): UAPI helpers · Userspace-side common types, ioctls, and buffer/control helpers. · `user/vpipe-common.h`
- [Capture modes](https://github.com/sysprog21/vpipe/blob/main/user/vpipe-capture-read.c): capture tools · Baseline capture programs for read, mmap, dmabuf, userptr, and m2m paths. · `user/vpipe-capture-read.c`
- [Fixture feed](https://github.com/sysprog21/vpipe/blob/main/user/vpipe-fixture-feed.c): fixture runner · Loads deterministic PGM bytes into dma-heap memory and feeds vpipe via DMABUF. · `user/vpipe-fixture-feed.c`
- [Bench driver](https://github.com/sysprog21/vpipe/blob/main/user/vpipe-bench-fixture.c): benchmark harness · Runs deterministic transport scenarios and records per-frame results. · `user/vpipe-bench-fixture.c`
- [Meta drain](https://github.com/sysprog21/vpipe/blob/main/user/vpipe-meta-drain.c): metadata tool · Reads /dev/vpipe-meta and writes per-frame metadata CSV. · `user/vpipe-meta-drain.c`
- [CV reference](https://github.com/sysprog21/vpipe/blob/main/user/vpipe-cv-ref.c): reference tool · Reference Tiny CV implementation used to compare kernel output. · `user/vpipe-cv-ref.c`
- [PGM diff](https://github.com/sysprog21/vpipe/blob/main/user/vpipe-pgm-diff.c): comparison tool · Compares input, reference, kernel, and diff images for validation. · `user/vpipe-pgm-diff.c`

### Kernel

V4L2 mem2mem device, Tiny CV transform, and metadata sideband.

- [M2M device](https://github.com/sysprog21/vpipe/blob/main/kmod/vpipe-m2m.c): v4l2 mem2mem · Kernel mem2mem transport with OUTPUT and CAPTURE queues. · `kmod/vpipe-m2m.c`
- [Tiny CV](https://github.com/sysprog21/vpipe/blob/main/kmod/vpipe-cv.c): kernel transform · Deterministic thresholding over a clamped ROI with no floating point. · `kmod/vpipe-cv.c`
- [Meta ring](https://github.com/sysprog21/vpipe/blob/main/kmod/vpipe-meta.c): miscdevice · Per-frame metadata sideband exposed as /dev/vpipe-meta with reader cursors. · `kmod/vpipe-meta.c`
- [Kernel UAPI](https://github.com/sysprog21/vpipe/blob/main/kmod/vpipe.h): contract header · Shared ioctl, control, and metadata layout definitions for user and kernel. · `kmod/vpipe.h`
- [Internal state](https://github.com/sysprog21/vpipe/blob/main/kmod/vpipe-internal.h): kernel internals · Private kernel definitions shared by the transport, CV, and metadata code. · `kmod/vpipe-internal.h`

### Validation

Fixtures, scripts, docs, and CI gates that exercise and measure the system.

- [PGM fixtures](https://github.com/sysprog21/vpipe/blob/main/tests/fixtures/edge-block.pgm): test inputs · Deterministic grayscale source images for transport and Tiny CV validation. · `tests/fixtures/edge-block.pgm`
- [Bench scripts](https://github.com/sysprog21/vpipe/blob/main/scripts/bench_vpipe.sh): automation · Shell scripts that run capture loops, perf tracing, and benchmark collection. · `scripts/bench_vpipe.sh`
- [Trace tools](https://github.com/sysprog21/vpipe/blob/main/scripts/trace_v4l2.sh): automation · Helpers for V4L2 tracing, worst-case tracing, and vivid setup. · `scripts/trace_v4l2.sh`
- [Summarizer](https://github.com/sysprog21/vpipe/blob/main/scripts/summarize-benchmark.py): reporting · Post-processes benchmark outputs into summary artifacts. · `scripts/summarize-benchmark.py`
- [Validation suite](https://github.com/sysprog21/vpipe/blob/main/tests/validate-tinycv.sh): test harness · Tiny CV and integration checks that verify module load, metadata, and capture behavior. · `tests/validate-tinycv.sh`
- [Design docs](https://github.com/sysprog21/vpipe/blob/main/docs/design.md): docs · Architecture and benchmark notes explaining the transport and measurement model. · `docs/design.md`
- [CI gates](https://github.com/sysprog21/vpipe/blob/main/.github/workflows/main.yml): ci checks · Formatting, security, newline, and cppcheck enforcement in CI and local scripts. · `.github/workflows/main.yml`

## Connections

- Capture modes → M2M device: V4L2 ioctls · Userspace capture tools drive the mem2mem device through V4L2.
- Fixture feed → M2M device: DMABUF input · Fixture bytes are loaded into dma-heap memory and queued to OUTPUT.
- Bench driver → Fixture feed: runs · Benchmark harness invokes deterministic fixture transport scenarios.
- Bench driver → Meta drain: drains · Bench runs collect metadata sideband output as CSV.
- CV reference → PGM diff: compares via · Reference output feeds the image diff tool for validation.
- Shared API → Kernel UAPI: shared contract · Userspace and kernel share ioctl and control definitions.
- M2M device → Tiny CV: device_run · Queued frames are processed by the Tiny CV kernel path.
- M2M device → Meta ring: frame metadata · Per-frame state is emitted to the metadata sideband.
- M2M device → Kernel UAPI: controls · The transport honors snapshotted per-buffer controls at QBUF time.
- Tiny CV → Internal state: uses · The transform uses private kernel state and helpers.
- Meta ring → Internal state: ring state · Metadata storage uses internal ring-buffer bookkeeping.
- Meta drain → Meta ring: reads /dev/vpipe-meta · Userspace drains metadata from the miscdevice.
- PGM fixtures → Fixture feed: source data · Fixture images seed deterministic transport runs.
- Bench scripts → Capture modes: orchestrates · Shell automation runs capture modes and records artifacts.
- Bench scripts → Bench driver: orchestrates · Benchmark scripts drive fixture-based runs.
- Trace tools → M2M device: traces · Tracing scripts observe V4L2 queueing and transport behavior.
- Summarizer → Bench scripts: summarizes · Benchmark summaries are derived from collected outputs.
- Validation suite → CV reference: validates · Validation compares kernel output with the reference path.
- Validation suite → M2M device: loads module · Integration checks exercise module load and capture paths.
- Design docs → M2M device: describes · Docs explain the transport boundary and measurement goals.
- CI gates → Validation suite: gates · CI enforces the repository validation suite and style checks.