How zartbot/shallowsim works
Project Type and Purpose
• ShallowSim is a simulation tool designed to analyze the inference performance of the DeepSeek-V3/R1 model. It benchmarks different model architectures on various GPU architectures and studies interconnect bus requirements in scale-up and scale-out scenarios.
• The README confirms its dual focus: simulation of model inference performance and analysis of architectural design, while the file tree shows a mix of Python scripts and Jupyter notebooks that drive analysis and visualization.
Main Components to Highlight
a. Simulation Engine (Core Module)
• Module File: “shallowsim.py”
b. Configuration & Model Parameters
• Files/Objects: ModelArgs, Config (as initialized in the notebooks)
• Role: Defines the simulation parameters and model-specific settings required to run performance tests.
c. GPU Information and Device Data Handler
• File: “device/gpu_info.csv”
• Role: Acts as an input data source. A component responsible for reading GPU details and providing this information to the simulation engine.
d. Data Analysis and Charting Components
• Files: Jupyter notebooks (e.g., decode.ipynb, prefill.ipynb, model_arch.ipynb) and produced CSV files (e.g., perf_vs_seq_len.csv, perf_vs_n_layers.csv, perf_vs_n_active_experts.csv)
• Role: Notebooks orchestrate simulation runs, data aggregation, result extraction, and plotting of performance metrics (with figures stored in the “figures” directory).
e. Experiment Management
• Files: The various notebooks serve as orchestrators for different simulation aspects (e.g., sequence length analysis, GPU-specific stats).
• Role: They calibrate the simulation by setting parameters (like EP counts, BatchSize, seq_len) and scheduling performance evaluations.
Relationships and Interactions
• The simulation process starts by reading the GPU details (from “device/gpu_info.csv”) via a dedicated function (get_gpu_info), which ensures that the simulation uses accurate hardware profiles.
• The ModelArgs and Config classes (or objects) prepare the simulation settings which are then passed into the simulation engine function (decode_time_with_ep_list).
• The final outputs include CSV files for further analysis and figures (performance plots stored under “figures”) that help evaluate the impact of model configurations and GPU interconnect characteristics.
Architectural Patterns & Design Principles
• Separation of Concerns: Different components handle configuration (ModelArgs/Config), simulation (shallowsim.py functions), data ingestion (gpu_info.csv), and analysis/visualization (Jupyter notebooks and plotting functions).
• Pipeline/Data Flow: The architecture represents a pipeline—from initial configuration and data ingestion to simulation, result aggregation, and visualization.
• Reusability & Extensibility: With distinct modules doing specific jobs, the design encourages future extension (e.g., adding additional simulation parameters or new visualization techniques) without overhauling the entire system.