How facebookresearch/holistictraceanalysis works
Project’s Nature and Goals
• The project is a performance analysis tool aimed at analyzing trace data to identify performance bottlenecks in distributed GPU workloads.
• It provides a variety of analytical features (temporal breakdown, critical path analysis, kernel breakdown, memory bandwidth tracking, etc.) via its main APIs.
• It offers both interactive usage (via Jupyter notebooks or command-line scripts) and programmatic API access (through modules like TraceAnalysis and TraceDiff).
File Structure for Core Components
• The main source code lives in the “hta” directory:
• The “analyzers” submodule contains the core algorithms that perform various analyses – each analyzer is responsible for aspects like timeline, critical path, CUDA kernel analysis, etc.
• The “common” submodule includes shared functionalities like trace parsing, call stack construction, trace filtering, and building various intermediate representations.
• The “configs” directory holds configuration files (e.g., logging settings, parser configuration) that dictate runtime behavior.
• The “utils” submodule provides helper functions for testing, validation, and other general-purpose operations.
• The “docs”, “examples”, and “tests” directories suggest comprehensive support for documentation, demonstration notebooks, and unit testing.
• Auxiliary files (CI/CD workflows, Makefiles, etc.) indicate how the system is built, maintained, and deployed.
Main Components and Their Interactions
a. Input Layer:
• External Trace Files: These are JSON (or gzip-compressed JSON) trace data produced by PyTorch’s profiler. Mark these as the system’s primary external data.
• Data Preparation: A conceptual component where the traces are gathered in a unique folder.
b. API/Entry Points:
• TraceAnalysis API (found in hta/trace_analysis.py): This is the cornerstone for initiating various performance analyses.
• TraceDiff API (found in hta/trace_diff.py): Handles comparisons between traces.
c. Core Analysis Components:
• Analyzers: Separate modules (e.g., critical path analysis, kernel analysis, timeline analysis) that process the input traces.
• Common Utilities: Parsing modules, trace file handlers, call stack and graph generators – these are essential for preparing data for analysis.
d. Configuration and Environment Management:
• Configs folder: Manages default parameters, logging configurations, and environment settings.
e. Output & Reporting:
• Processed Data Frames/Visualizations which may be used in Jupyter notebooks or exported to external files.
• Integration with documentation (e.g., via the docs folder) to explain results and usage.
f. Testing and Continuous Integration:
• Unit and integration tests ensure reliability.
• CI/CD pipelines (via GitHub Actions/CircleCI workflows) automatically verify system health.
Architectural Patterns and Design Principles
• Modular Design: Separation of core analyzers, common utilities, and configuration files.
• API-Driven: Clearly defined entry points for accessing trace analysis functionalities.
• Extensibility: Organized into distinct modules making it easier to add or extend analysis routines.
• Testability: Presence of extensive tests indicates a layered design that is well-isolated for unit testing.
• Integration with External Tools: Clearly designed to work with external trace collection utilities (e.g., PyTorch Profiler), notebooks for interactive usage, and CI/CD for continuous development.
Elements
• Clear labels for each component:
• External Trace Data Sources (Input Folder/Data Store)
• API Endpoints (TraceAnalysis, TraceDiff)
• Analysis Modules (Critical Path Analyzer, Timeline Analyzer, Kernel Analyzer, etc.)
• Common Utility Layer (Trace Parsing, Call Stack Management, etc.)
• Configuration Management (logging.config, parser_config, etc.)
• Output Layer (Visualizations, DataFrames, exported files)
• Core service components (API, analyzers)
• Configuration/utility layers (configs and utils)
• Supporting modules (documentation, CI/CD, tests)
• External systems (trace producers and output consumers)
• Input Layer (External Trace Files)
• API Layer (TraceAnalysis & TraceDiff)
• Common & Utility Modules that assist the analyzers
• Configuration Management that affects the entire analysis pipeline
• Output/Reporting Mechanism that interfaces with visualization tools or notebooks.