# yohanchatelain/pytracer architecture

> Architecture diagram of the GitHub repository yohanchatelain/pytracer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/yohanchatelain/pytracer
- Repository: https://github.com/yohanchatelain/pytracer

## Overview

##### Project type and purpose
- Pytracer is an open-source Python developer tool (CLI application with a web GUI) for instrumenting Python code, collecting numerical trace data, computing statistics, and visualizing results via a Plotly/Dash dashboard.

##### File structure highlights
- Top level: CI configs (.github/), Docker definitions (docker/), CLI entry points (pytracer/__main__.py), tests, requirements/setup.
- pytracer/core: core tracing, I/O (reader/writer/exporter), statistics, wrapper/filter layers.
- pytracer/callgraph: call graph data structures, layout and styling, assets for graph visualization.
- pytracer/gui: Dash/Flask server, HTML/CSS/JS assets, callbacks, layout definitions.
- pytracer/module: instrumentation hooks, parser for Python bytecode modules.
- pytracer/data/config: JSON and exclusion filter files for tracer configuration.
- Scripts and tests directories for utilities and test coverage.

##### System components and interactions
a. CLI Front End
- Delegates to corresponding module (pytracer.trace, pytracer.parse, etc.)
b. Trace Engine
- Loads configuration (config.json)
- Instruments target Python functions via wrapper/filter modules
- Executes target application under stochastic arithmetic (via external “fuzzy” runtime)
- Emits raw traces to cache directory as Pickle files
c. Parse Engine
- Scans cache directory for Pickle traces
- Reads them through core/inout/reader
- Aggregates numerical differences using statistics module (cnh or general)
- Writes aggregated data and generated callgraph to HDF5 and Pickle via core/inout/exporter
d. Callgraph Generator
- Builds a call graph structure from traces (pytracer/callgraph/core)
- Lays out nodes/edges (layout, layout_style)
- Exposes serialized graph to visualization layer
e. Visualization Server
- Flask/Dash application (pytracer/gui/app)
- Loads aggregated HDF5 data and callgraph
- Provides interactive dashboard (HTML/CSS/JS assets, callbacks)
- Serves on specified host/port for browser interaction
f. Auxiliary services
- Configuration Manager reads JSON and exclusion lists
- Logging and caching subsystems (pytracer/cache, pytracer/utils/log)
- Testing harness (conftest.py, pytest.ini) for CI

##### Architectural patterns and principles
- Modular “command” pattern separating CLI subcommands
- Layered I/O abstraction (reader → core processing → exporter)
- Pluggable instrumentation via wrapper/filter
- MVC-like separation in GUI: model (HDF5 + callgraph), view (Dash layouts/assets), controller (callbacks)
- Config-driven behavior for inclusion/exclusion

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% CLI Front End
    subgraph "CLI Front End" 
        direction TB
        CLI["CLI Front End"]:::ui
        TraceCmd["trace (pytracer/module/tracer_init.py)"]:::ui
        ParseCmd["parse (pytracer/module/parser_init.py)"]:::ui
        VisualizeCmd["visualize (pytracer/scripts/pytracer)"]:::ui
    end

    %% Trace Engine
    subgraph "Trace Engine" 
        direction TB
        ConfigLoader["Configuration Loader"]:::core
        Wrapper["Instrumentation Hooks & Filters"]:::core
        Invocation["Invocation & I/O"]:::core
        ExternalRuntime["External Runtime"]:::ext
    end

    %% Persistence - Cache
    PickleStore(("Pickle Cache")):::store

    %% Parse Engine
    subgraph "Parse Engine"
        direction TB
        Reader["Trace Reader"]:::core
        Stats["Statistics Aggregator"]:::core
        Exporter["Data Exporter"]:::core
    end

    %% Callgraph Generator
    subgraph "Callgraph Generator"
        direction TB
        CGCore["Core Data Structure"]:::core
        Layout["Layout & Styling"]:::core
    end

    %% Persistence - HDF5 & Callgraph Store
    HDF5Store(("HDF5 Store")):::store
    CGStore(("Callgraph Store")):::store

    %% Visualization Server
    subgraph "Visualization Server"
        direction TB
        WebApp["Dash/Flask App"]:::ui
        LayoutDefs["Layout Definitions"]:::ui
        Callbacks["GUI Callbacks"]:::ui
        Assets["Static Assets"]:::ui
    end

    %% Config Manager
    ConfigJSON["Config Manager\n(JSON & Exclusions)"]:::ext

    %% Logging & Caching
    Logging["Logging Utility"]:::ext
    CacheUtil["Caching Subsystem"]:::ext

    %% Testing Harness
    subgraph "Testing Harness"
        direction TB
        PytestIni["pytest.ini"]:::ext
        Conftest["conftest.py"]:::ext
        Tests["Unit & Integration Tests"]:::ext
    end

    %% Connections
    CLI -->|trace command| TraceCmd
    CLI -->|parse command| ParseCmd
    CLI -->|visualize command| VisualizeCmd

    TraceCmd -->|invoke| TraceEngineTrace[/"Trace Engine"/] 
    TraceEngineTrace --> ConfigLoader
    TraceEngineTrace --> Wrapper
    TraceEngineTrace --> Invocation
    Invocation --> ExternalRuntime
    Invocation --> PickleStore

    ParseCmd -->|invoke| ParseEngineParse[/"Parse Engine"/]
    ParseEngineParse --> Reader
    ParseEngineParse --> Stats
    ParseEngineParse --> Exporter
    Reader --> PickleStore
    Exporter --> HDF5Store
    Exporter --> CGCore

    CGCore --> Layout
    Layout --> CGStore

    VisualizeCmd --> WebApp
    WebApp --> LayoutDefs
    WebApp --> Callbacks
    WebApp --> Assets
    WebApp --> HDF5Store
    WebApp --> CGStore

    ConfigJSON --> ConfigLoader
    ConfigJSON --> Wrapper

    Logging --> TraceEngineTrace
    CacheUtil --> Invocation

    PytestIni --> Tests
    Conftest --> Tests

    %% Click Events
    click CLI "https://github.com/yohanchatelain/pytracer/blob/master/pytracer/__main__.py"
    click TraceCmd "https://github.com/yohanchatelain/pytracer/blob/master/pytracer/module/tracer_init.py"
    click ParseCmd "https://github.com/yohanchatelain/pytracer/blob/master/pytracer/module/parser_init.py"
    click VisualizeCmd "https://github.com/yohanchatelain/pytracer/tree/master/pytracer/scripts/pytracer"
    click ConfigLoader "https://github.com/yohanchatelain/pytracer/blob/master/pytracer/core/config.py"
    click Wrapper "https://github.com/yohanchatelain/pytracer/blob/master/pytracer/core/wrapper/wrapper.py"
    click Invocation "https://github.com/yohanchatelain/pytracer/blob/master/pytracer/module/tracer.py"
    click CacheUtil "https://github.com/yohanchatelain/pytracer/blob/master/pytracer/cache.py"
    click ExternalRuntime "https://github.com/yohanchatelain/pytracer/blob/master/docker/Dockerfile.fuzzy"
    click Reader "https://github.com/yohanchatelain/pytracer/blob/master/pytracer/core/inout/reader/_pickle.py"
    click Stats "https://github.com/yohanchatelain/pytracer/blob/master/pytracer/core/stats/stats.py"
    click Exporter "https://github.com/yohanchatelain/pytracer/blob/master/pytracer/core/inout/exporter/_hdf5.py"
    click CGCore "https://github.com/yohanchatelain/pytracer/blob/master/pytracer/callgraph/core.py"
    click Layout "https://github.com/yohanchatelain/pytracer/blob/master/pytracer/callgraph/layout.py"
    click WebApp "https://github.com/yohanchatelain/pytracer/blob/master/pytracer/gui/app.py"
    click LayoutDefs "https://github.com/yohanchatelain/pytracer/blob/master/pytracer/gui/layout.py"
    click Callbacks "https://github.com/yohanchatelain/pytracer/blob/master/pytracer/gui/callbacks.py"
    click Assets "https://github.com/yohanchatelain/pytracer/tree/master/pytracer/gui/assets"
    click ConfigJSON "https://github.com/yohanchatelain/pytracer/blob/master/pytracer/data/config/config.json"
    click Logging "https://github.com/yohanchatelain/pytracer/blob/master/pytracer/utils/log.py"
    click PytestIni "https://github.com/yohanchatelain/pytracer/blob/master/pytest.ini"
    click Conftest "https://github.com/yohanchatelain/pytracer/blob/master/conftest.py"
    click Tests "https://github.com/yohanchatelain/pytracer/tree/master/pytracer/test"

    %% Styles
    classDef core fill:#BBDEFB,stroke:#0D47A1,color:#0D47A1
    classDef store fill:#C8E6C9,stroke:#1B5E20,color:#1B5E20
    classDef ui fill:#FFE0B2,stroke:#E65100,color:#E65100
    classDef ext fill:#E0E0E0,stroke:#424242,color:#424242
```
