# jishnujayakumar/mv-tractus architecture

> Architecture diagram of the GitHub repository jishnujayakumar/mv-tractus, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-04-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jishnujayakumar/mv-tractus
- Repository: https://github.com/jishnujayakumar/mv-tractus

## Overview

1. **What this repo is**
   - MV-Tractus is a local video-processing toolchain for extracting motion vectors from H.264/MPEG streams.
   - The system has two main faces: native CLI binaries and a Python package wrapper.

2. **Primary runtime boundary**
   - The core work happens in native code under `src/c`, which interfaces with FFmpeg APIs.
   - The repository does not define a backend service or networked architecture; it is a file-in / file-out desktop-style pipeline.

3. **Native extraction subsystem**
   - `src/c/extract_mvs.c` is the main motion-vector extractor.
   - `src/c/extract_mvs_with_frames.c` extends extraction to also save decoded frames.
   - `src/c/e_json.c` suggests a small serialization layer for writing motion vectors as JSON.
   - These programs produce artifacts in the output directories rather than returning in-memory results to another service.

4. **FFmpeg dependency layer**
   - The large `include/` tree is a vendored FFmpeg source/header snapshot, not just a thin header include directory.
   - This is the main third-party dependency boundary and likely the build target for the native binaries.
   - The repo’s own code sits on top of FFmpeg’s decode/filter APIs to access motion vectors and frame data.

5. **CLI build and execution path**
   - `compile` and `compile_command.txt` indicate a scripted native build workflow.
   - The compiled tools live in `bin/`, including `extract_mvs`, `extract_mvs_with_frames`, and `efmvs`.
   - `first_run.sh`, `install_ffmpeg.sh`, and `ffmpeg.conf` are setup/support assets for locating or building FFmpeg dependencies.

6. **Output artifacts and data flow**
   - Input: a video file, typically H.264/MPEG.
   - Processing: FFmpeg decodes the stream and exposes motion vectors to the extractor.
   - Output: JSON motion-vector files under `output/mv`, and optionally frame images under `output/frames`.
   - The default paths imply a simple local pipeline with convention-based storage.

7. **Python package subsystem**
   - `mvtpy/mv_tractus` is the packaged Python API.
   - `wrapper.py` exposes classes such as `MVTractus` and `MVTractusWithFrames`, mirroring the CLI capabilities.
   - This package is the higher-level integration boundary for users who want programmatic access instead of shell commands.

8. **Overlay/visualization path**
   - `mvtpy/mv_tractus/overlay.py` adds a visualization stage that draws motion vectors on saved frames.
   - The README shows this uses Pillow-style image handling and operates on two prior outputs: frame images and JSON motion-vector files.
   - That makes overlay a post-processing consumer of extractor outputs, not part of the decode stage.

9. **Python package build and distribution**
   - `mvtpy/pyproject.toml`, `setup.py`, and `setup.cfg` define the Python packaging boundary.
   - The repo also contains built distributions in `mvtpy/dist`, showing the package is intended for installable reuse.

10. **Experimental and prototype area**
   - `experiments/`, `original/`, and `python-boost/` contain alternate implementations and backups.
   - These include C, C++, and Python experiments, plus OpenCV/Boost.Python conversion prototypes.
   - For an architecture diagram, these should be treated as non-production R&D branches, not part of the main runtime flow.

11. **Technology profile that matters**
   - Native layer: C/C++ with FFmpeg.
   - Python layer: package wrapper for orchestration and visualization.
   - Image overlay and conversion experiments reference OpenCV, Boost.Python, and ZMQ headers, but the README emphasizes FFmpeg-based extraction as the core path.

12. **Overall architectural shape**
   - The system is a layered offline processing pipeline:
     - video input → FFmpeg-based native extractor → JSON motion vectors and optional frames → Python overlay/analysis tools.
   - The key diagram boundaries are: external video file, FFmpeg/native extraction engine, generated artifacts on disk, and Python consumer/visualization tooling.

Main entry points: [`src/c/extract_mvs_with_frames.c`](https://github.com/jishnujayakumar/mv-tractus/blob/master/src/c/extract_mvs_with_frames.c) (Extract + frames), [`src/c/extract_mvs.c`](https://github.com/jishnujayakumar/mv-tractus/blob/master/src/c/extract_mvs.c) (Extract MVs), [`mvtpy/mv_tractus/overlay.py`](https://github.com/jishnujayakumar/mv-tractus/blob/master/mvtpy/mv_tractus/overlay.py) (Overlay tool), [`mvtpy/pyproject.toml`](https://github.com/jishnujayakumar/mv-tractus/blob/master/mvtpy/pyproject.toml) (Package files)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_native["Native pipeline"]
  node_extract_mvs["Extract MVs<br/>native CLI<br/>[extract_mvs.c]"]
  node_extract_mvs_frames["Extract + frames<br/>native CLI"]
  node_e_json["JSON writer<br/>serialization<br/>[e_json.c]"]
  node_build_script["Build script<br/>build workflow"]
  node_compile_cmd["Compile args<br/>build config"]
  node_bin_tools["CLI tools<br/>built binaries"]
  node_setup_assets["Setup assets<br/>bootstrap<br/>[first_run.sh]"]
  node_ffmpeg_conf["FFmpeg config<br/>env config<br/>[ffmpeg.conf]"]
end

subgraph group_python["Python package"]
  node_mvt_wrapper["Python API<br/>package wrapper<br/>[wrapper.py]"]
  node_overlay["Overlay tool<br/>visualization<br/>[overlay.py]"]
  node_py_pkg["Package files<br/>packaging<br/>[pyproject.toml]"]
  node_dist_artifacts["Distributions<br/>release artifacts"]
end

subgraph group_thirdparty["FFmpeg snapshot"]
  node_ffmpeg_api[("FFmpeg APIs<br/>vendored media stack")]
end

subgraph group_rnd["Experiments"]
  node_experiments["Experiments<br/>prototype area"]
  node_original["Originals<br/>backup area"]
  node_python_boost["Boost prototypes<br/>prototype area"]
end

node_input_video["Video file<br/>input media<br/>[1.h264]"]
node_frames_out["Frame output<br/>artifact dir"]
node_mv_out["MV output<br/>artifact dir"]

node_input_video -->|"decode"| node_extract_mvs
node_input_video -->|"decode"| node_extract_mvs_frames
node_extract_mvs -->|"FFmpeg calls"| node_ffmpeg_api
node_extract_mvs_frames -->|"FFmpeg calls"| node_ffmpeg_api
node_extract_mvs -->|"serialize"| node_e_json
node_extract_mvs_frames -->|"serialize"| node_e_json
node_extract_mvs -->|"write"| node_mv_out
node_extract_mvs_frames -->|"write"| node_mv_out
node_extract_mvs_frames -->|"write"| node_frames_out
node_build_script -->|"drives"| node_compile_cmd
node_build_script -->|"produces"| node_bin_tools
node_setup_assets -->|"configures"| node_ffmpeg_conf
node_ffmpeg_conf -->|"targets"| node_ffmpeg_api
node_mvt_wrapper -->|"wraps"| node_bin_tools
node_overlay -->|"consumes"| node_frames_out
node_overlay -->|"consumes"| node_mv_out
node_py_pkg -->|"packages"| node_mvt_wrapper
node_py_pkg -->|"packages"| node_overlay
node_dist_artifacts -->|"built from"| node_py_pkg
node_experiments -.->|"prototype of"| node_extract_mvs
node_original -.->|"prototype of"| node_extract_mvs_frames
node_python_boost -.->|"prototype of"| node_overlay

click node_input_video "https://github.com/jishnujayakumar/mv-tractus/blob/master/input/1.h264"
click node_extract_mvs "https://github.com/jishnujayakumar/mv-tractus/blob/master/src/c/extract_mvs.c"
click node_extract_mvs_frames "https://github.com/jishnujayakumar/mv-tractus/blob/master/src/c/extract_mvs_with_frames.c"
click node_e_json "https://github.com/jishnujayakumar/mv-tractus/blob/master/src/c/e_json.c"
click node_build_script "https://github.com/jishnujayakumar/mv-tractus/tree/master/compile"
click node_compile_cmd "https://github.com/jishnujayakumar/mv-tractus/blob/master/compile_command.txt"
click node_bin_tools "https://github.com/jishnujayakumar/mv-tractus/tree/master/bin"
click node_setup_assets "https://github.com/jishnujayakumar/mv-tractus/blob/master/first_run.sh"
click node_ffmpeg_conf "https://github.com/jishnujayakumar/mv-tractus/blob/master/ffmpeg.conf"
click node_ffmpeg_api "https://github.com/jishnujayakumar/mv-tractus/tree/master/include"
click node_mvt_wrapper "https://github.com/jishnujayakumar/mv-tractus/blob/master/mvtpy/mv_tractus/wrapper.py"
click node_overlay "https://github.com/jishnujayakumar/mv-tractus/blob/master/mvtpy/mv_tractus/overlay.py"
click node_py_pkg "https://github.com/jishnujayakumar/mv-tractus/blob/master/mvtpy/pyproject.toml"
click node_dist_artifacts "https://github.com/jishnujayakumar/mv-tractus/tree/master/mvtpy/dist"
click node_experiments "https://github.com/jishnujayakumar/mv-tractus/tree/master/experiments"
click node_original "https://github.com/jishnujayakumar/mv-tractus/tree/master/original"
click node_python_boost "https://github.com/jishnujayakumar/mv-tractus/tree/master/python-boost"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_extract_mvs,node_extract_mvs_frames,node_e_json,node_build_script,node_compile_cmd,node_bin_tools,node_setup_assets,node_ffmpeg_conf toneBlue
class node_mvt_wrapper,node_overlay,node_py_pkg,node_dist_artifacts toneAmber
class node_ffmpeg_api toneMint
class node_experiments,node_original,node_python_boost toneRose
class node_input_video,node_frames_out,node_mv_out toneNeutral
```

## Components

### Native pipeline

C/C++ extraction binaries and their build/bootstrap flow.

- [Extract MVs](https://github.com/jishnujayakumar/mv-tractus/blob/master/src/c/extract_mvs.c): native CLI · Main motion-vector extraction binary. · `src/c/extract_mvs.c`
- [Extract + frames](https://github.com/jishnujayakumar/mv-tractus/blob/master/src/c/extract_mvs_with_frames.c): native CLI · Extractor variant that also saves decoded frames. · `src/c/extract_mvs_with_frames.c`
- [JSON writer](https://github.com/jishnujayakumar/mv-tractus/blob/master/src/c/e_json.c): serialization · Motion-vector serialization helper used by native extraction. · `src/c/e_json.c`
- [Build script](https://github.com/jishnujayakumar/mv-tractus/tree/master/compile): build workflow · Scripted native build entrypoint for the CLI binaries. · `compile`
- [Compile args](https://github.com/jishnujayakumar/mv-tractus/blob/master/compile_command.txt): build config · Saved compile command used to reproduce the native build. · `compile_command.txt`
- [CLI tools](https://github.com/jishnujayakumar/mv-tractus/tree/master/bin): built binaries · Compiled executables for extraction and related processing. · `bin`
- [Setup assets](https://github.com/jishnujayakumar/mv-tractus/blob/master/first_run.sh): bootstrap · FFmpeg setup and first-run support files. · `first_run.sh`
- [FFmpeg config](https://github.com/jishnujayakumar/mv-tractus/blob/master/ffmpeg.conf): env config · Local configuration used to locate or build FFmpeg dependencies. · `ffmpeg.conf`

### Python package

Packaged wrapper and visualization layer for programmatic use.

- [Python API](https://github.com/jishnujayakumar/mv-tractus/blob/master/mvtpy/mv_tractus/wrapper.py): package wrapper · Packaged Python interface mirroring the native extraction workflows. · `mvtpy/mv_tractus/wrapper.py`
- [Overlay tool](https://github.com/jishnujayakumar/mv-tractus/blob/master/mvtpy/mv_tractus/overlay.py): visualization · Post-processing stage that draws motion vectors onto saved frames. · `mvtpy/mv_tractus/overlay.py`
- [Package files](https://github.com/jishnujayakumar/mv-tractus/blob/master/mvtpy/pyproject.toml): packaging · Python packaging boundary for installable distribution. · `mvtpy/pyproject.toml`
- [Distributions](https://github.com/jishnujayakumar/mv-tractus/tree/master/mvtpy/dist): release artifacts · Built wheel and source distributions for the Python package. · `mvtpy/dist`

### FFmpeg snapshot

Vendored FFmpeg source tree and APIs used by the native extractor.

- [FFmpeg APIs](https://github.com/jishnujayakumar/mv-tractus/tree/master/include): vendored media stack · Vendored FFmpeg headers and sources that provide decode and filter capabilities. · `include`

### Experiments

Backups and prototype implementations outside the main runtime path.

- [Experiments](https://github.com/jishnujayakumar/mv-tractus/tree/master/experiments): prototype area · Alternative implementations, backups, and conversion experiments. · `experiments`
- [Originals](https://github.com/jishnujayakumar/mv-tractus/tree/master/original): backup area · Earlier native extractor versions and historical variants. · `original`
- [Boost prototypes](https://github.com/jishnujayakumar/mv-tractus/tree/master/python-boost): prototype area · OpenCV/Boost.Python conversion experiments and alternative wrappers. · `python-boost`

### Other components

- [Video file](https://github.com/jishnujayakumar/mv-tractus/blob/master/input/1.h264): input media · Local H.264/MPEG input consumed by the extractor. · `input/1.h264`
- **Frame output**: artifact dir · Saved decoded frame images produced by the native pipeline.
- **MV output**: artifact dir · Generated motion-vector JSON output directory.

## Connections

- Video file → Extract MVs: decode · Video input is decoded by the native motion-vector extractor.
- Video file → Extract + frames: decode · Video input is decoded by the extractor variant that also saves frames.
- Extract MVs → FFmpeg APIs: FFmpeg calls · Extractor uses FFmpeg decode APIs and motion-vector side data.
- Extract + frames → FFmpeg APIs: FFmpeg calls · Frames+vectors variant relies on FFmpeg decoding and frame access.
- Extract MVs → JSON writer: serialize · Extractor writes motion-vector results through the JSON layer.
- Extract + frames → JSON writer: serialize · Frames+vectors variant writes motion-vector JSON output.
- Extract MVs → MV output: write · Motion-vector JSON is emitted to the motion-vector output location.
- Extract + frames → MV output: write · Motion-vector JSON is emitted to the motion-vector output location.
- Extract + frames → Frame output: write · Decoded frames are saved as output artifacts.
- Build script → Compile args: drives · Build script uses the recorded compile command/workflow.
- Build script → CLI tools: produces · Native build produces CLI binaries in bin/.
- Setup assets → FFmpeg config: configures · Bootstrap scripts help locate or prepare FFmpeg dependencies.
- FFmpeg config → FFmpeg APIs: targets · Configuration points the native build/runtime at vendored FFmpeg.
- Python API → CLI tools: wraps · Python API mirrors the compiled command-line tools.
- Overlay tool → Frame output: consumes · Overlay operates on previously saved frame images.
- Overlay tool → MV output: consumes · Overlay consumes motion-vector JSON files.
- Package files → Python API: packages · Packaging boundary exposes the Python wrapper module.
- Package files → Overlay tool: packages · Packaging boundary includes visualization utilities.
- Distributions → Package files: built from · Distribution artifacts are produced from the Python packaging setup.
- Experiments → Extract MVs: prototype of · Experimental code explores alternate extractor implementations.
- Originals → Extract + frames: prototype of · Historical sources capture earlier extractor variants.
- Boost prototypes → Overlay tool: prototype of · Boost/OpenCV experiments explore Python-side visualization and conversion.
