How pytorch/kineto works
PROJECT OVERVIEW & COMPONENT IDENTIFICATION
• Project Purpose: Kineto is all about performance observability, diagnostics, and actionable recommendations for PyTorch-based machine learning workloads. Two major subprojects are:
• Libkineto: An in-process, low-overhead profiling library for GPU timeline tracing and deeper system analysis.
• tb_plugin: (Previously PyTorch TensorBoard Profiler) a tool for visualizing and analyzing profiling results.
• Additional support components include benchmarking scripts, sample programs, and stress tests that help validate performance and integration.
KEY COMPONENTS
a. Profiling Core (Libkineto)
• Libkineto source: Contains headers, CMake build scripts, and source files implementing multiple profiling interfaces (e.g., Cupti, Roctracer, and Xpupti plugins).
• Interfaces and APIs: Represent the core abstractions (ActivityProfiler, ActivityTrace, Logging APIs) used to capture and handle GPU events.
• Plugins and extensions: Modules under “src/plugin” that integrate with different runtime toolkits (e.g. Xpupti for SYCL toolkit support).
b. Visualization & Integration (tb_plugin)
• Frontend components: The “tb_plugin/fe” directory shows a modern web UI (React/TypeScript based) with components, charts, and API modules that convert raw profiling data into readable visualizations.
• Backend/Processing: The “tb_plugin/torch_tb_profiler” directory comprises modules that process trace data, perform event parsing, and generate insights for visualization.
• API layer: API endpoints and configuration files (openapi.yaml, generated API code) that support communication between the frontend and the back-end profiler.
c. Benchmarking & Testing Framework
• Benchmarks: The “benchmarks/perfetto” folder contains test scripts, installation instructions, and sample configurations to simulate and measure performance.
• Stress Tests & Unit Tests: Located in “libkineto/stress_test” and “libkineto/test”, these verify the reliability and performance under varying loads.
d. External Integrations and CI/CD
• GitHub workflows: Located in “.github/workflows”, these YAML files define CI/CD pipelines for building and testing the different components (e.g., libkineto builds, tb_plugin CI).
• Integration Points: Indicate external systems such as PyTorch (mentioned in the README) and third-party profiling tools.
• Three primary layers:
1. Data Capture / Instrumentation Layer:
• This is where Libkineto lives. It integrates directly with GPU APIs (Cupti, Roctracer, etc.) and captures low-level performance metrics.
2. Analysis & Processing Layer:
• Includes components for trace processing (within libkineto and within tb_plugin/profiler). Highlight parsing modules, recommendation engines, and overall trace analysis (e.g., HTA – Holistic Trace Analysis).
3. Visualization & User Interface Layer:
• Represented by tb_plugin’s frontend and supporting APIs. It converts processed metrics into insights and visualizations (charts, tables, interactive modules).
• Data Flow:
• That there are bidirectional interactions between the profiling back-end and the UI (e.g., user-triggered queries and real-time updates).
TECHNICAL STACK & DESIGN PATTERNS
• Technologies & Tools:
• C/C++ for Libkineto’s performance-critical components.
• CMake for build configurations.
• Python scripts in benchmarking and testing.
• TypeScript/React in the tb_plugin frontend.
• Integration with PyTorch and TensorBoard.
• Architectural Patterns:
• Separation of Concerns: Clearly separate low-level profiling, trace analysis, and front-end visualization.
• Plugin Architecture: The integration of multiple backends (Cupti, Roctracer, Xpupti) indicates a plug-and-play design.
• Modular and extensible design, as inferred from the distributed directory structure and decoupled modules in tb_plugin and libkineto.
FINAL CHECKLIST FOR THE PRINCIPAL SOFTWARE ENGINEER
• Identified layers and components (Libkineto core, tb_plugin frontend/back-end, benchmarking/testing modules, external integration and CI workflows).
• Profiling components (low-level, high-performance C/C++ modules).
• Processing and analysis modules (data parsers, recommendation engines).
• Visualization elements (frontend, UI components).
• Remain detailed but not overly cluttered; focus on capturing essential architectural interactions for maintenance and extension.