# gaogaotiantian/viztracer architecture

> Architecture diagram of the GitHub repository gaogaotiantian/viztracer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/gaogaotiantian/viztracer
- Repository: https://github.com/gaogaotiantian/viztracer

## Overview

##### Main Components

a. Core Tracing Engine:
- Core VizTracer class components
- Monitoring/profiling hooks integration

b. Data Collection Components:
- Event capturing system
- Thread/Process management
- Custom event handlers
- PyTorch integration module

c. Output Processing:
- Data compression (vcompressor)
- Report builder
- JSON serialization layer

d. Visualization Layer:
- Perfetto-based web viewer
- Flamegraph generation
- Interactive UI components

##### Component Relationships

**[Python Application]**
- ↓
**[VizTracer Core] → [Monitoring System]**
- ↙     ↓     ↘
**[Thread] [Process] [Async]**
- ↘     ↓     ↙
**[Event Collection System]**
- ↓
**[Data Processing Layer]**
- ↓
**[Storage Layer (JSON)]**
- ↓
**[Web Visualization]**

##### Important Elements to Highlight

a. Input Sources:
- Python code execution
- Threading/multiprocessing
- Async operations
- PyTorch operations
- Custom events

b. Processing Pipeline:
- Event capture
- Data compression
- JSON serialization
- Report generation

c. Output Interfaces:
- JSON file output
- Web viewer
- VS Code extension integration

##### Integration Points

- Python runtime hooks
- PyTorch integration
- External tool connections
- VS Code extension interface

##### Technical Details

- sys.monitoring integration (Python 3.12+)
- sys.setprofile hooks (older versions)
- Native threading module integration
- Multiprocessing support
- Async/await handling
- PyTorch profiler integration

Remember to include:
- Version-specific components (e.g., Python 3.12+ features)
- Component interaction boundaries
- Performance-critical paths highlighted

This architecture should emphasize VizTracer's low-overhead design while showing its comprehensive tracing capabilities across different Python execution contexts.

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph Legend
        L1[Core Components]:::core
        L2[Data Collection]:::data
        L3[Processing]:::processing
        L4[Visualization]:::viz
        L5[External Integration]:::external
    end

    subgraph "Python Application Layer"
        PyApp[Python Application]:::external
        PyTorch[PyTorch Integration]:::external
    end

    subgraph "Core Tracing Layer"
        VizCore[VizTracer Core]:::core
        SnapTrace[SnapTrace C Module]:::core
        QuickTime[QuickTime Module]:::core
        EventNode[Event Node]:::core
    end

    subgraph "Data Collection Layer"
        EventBase[Event Base System]:::data
        VizEvent[Event Implementation]:::data
        CustomEvents[Custom Events]:::data
        ProcessAttach[Process Attachment]:::data
        VizLogging[Logging Integration]:::data
        VizPlugin[Plugin System]:::data
    end

    subgraph "Processing Layer"
        VCompressor[Compression Engine]:::processing
        ReportBuilder[Report Builder]:::processing
        FuncTree[Function Tree]:::processing
    end

    subgraph "Visualization Layer"
        WebViewer[Web Viewer]:::viz
        FlameGraph[Flamegraph]:::viz
        TraceViewer[Trace Viewer]:::viz
        ViewerImpl[Viewer Implementation]:::viz
    end

    PyApp --> VizCore
    PyTorch --> VizCore
    VizCore --> SnapTrace
    VizCore --> QuickTime
    VizCore --> EventNode
    
    SnapTrace --> EventBase
    EventBase --> VizEvent
    EventBase --> CustomEvents
    EventBase --> ProcessAttach
    EventBase --> VizLogging
    EventBase --> VizPlugin
    
    VizEvent --> VCompressor
    CustomEvents --> VCompressor
    VCompressor --> ReportBuilder
    ReportBuilder --> FuncTree
    
    FuncTree --> WebViewer
    FuncTree --> FlameGraph
    FuncTree --> TraceViewer
    WebViewer --> ViewerImpl
    FlameGraph --> ViewerImpl
    TraceViewer --> ViewerImpl

    %% Component Mappings
    click SnapTrace "https://github.com/gaogaotiantian/viztracer/blob/master/src/viztracer/modules/snaptrace.c"
    click VizCore "https://github.com/gaogaotiantian/viztracer/blob/master/src/viztracer/viztracer.py"
    click EventBase "https://github.com/gaogaotiantian/viztracer/blob/master/src/viztracer/event_base.py"
    click VizEvent "https://github.com/gaogaotiantian/viztracer/blob/master/src/viztracer/vizevent.py"
    click CustomEvents "https://github.com/gaogaotiantian/viztracer/blob/master/src/viztracer/vizobject.py"
    click ProcessAttach "https://github.com/gaogaotiantian/viztracer/blob/master/src/viztracer/attach.py"
    click VizLogging "https://github.com/gaogaotiantian/viztracer/blob/master/src/viztracer/vizlogging.py"
    click VCompressor "https://github.com/gaogaotiantian/viztracer/blob/master/src/viztracer/modules/vcompressor/vcompressor.c"
    click ReportBuilder "https://github.com/gaogaotiantian/viztracer/blob/master/src/viztracer/report_builder.py"
    click FuncTree "https://github.com/gaogaotiantian/viztracer/blob/master/src/viztracer/functree.py"
    click WebViewer "https://github.com/gaogaotiantian/viztracer/tree/master/src/viztracer/web_dist/"
    click FlameGraph "https://github.com/gaogaotiantian/viztracer/blob/master/src/viztracer/html/flamegraph.html"
    click TraceViewer "https://github.com/gaogaotiantian/viztracer/blob/master/src/viztracer/html/trace_viewer_full.html"
    click ViewerImpl "https://github.com/gaogaotiantian/viztracer/blob/master/src/viztracer/viewer.py"
    click VizPlugin "https://github.com/gaogaotiantian/viztracer/blob/master/src/viztracer/vizplugin.py"
    click QuickTime "https://github.com/gaogaotiantian/viztracer/blob/master/src/viztracer/modules/quicktime.c"
    click EventNode "https://github.com/gaogaotiantian/viztracer/blob/master/src/viztracer/modules/eventnode.c"

    classDef core fill:#2196F3,stroke:#1565C0,color:white
    classDef data fill:#4CAF50,stroke:#2E7D32,color:white
    classDef processing fill:#FF9800,stroke:#EF6C00,color:white
    classDef viz fill:#9C27B0,stroke:#6A1B9A,color:white
    classDef external fill:#9E9E9E,stroke:#424242,color:white
```
