# abhibambhaniya/genz-llm-analyzer architecture

> Architecture diagram of the GitHub repository abhibambhaniya/genz-llm-analyzer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/abhibambhaniya/genz-llm-analyzer
- Repository: https://github.com/abhibambhaniya/genz-llm-analyzer

## Overview

#### Purpose and Entry Points
GenZ is a Python LLM-inference performance and memory estimator. It combines model architecture, serving workload, hardware characteristics, datatype, and TP/PP parallelism to estimate latency, memory, communication, and power. Primary programmatic inference entry points are in `GenZ/LLM_inference/llm_prefill.py`, `GenZ/LLM_inference/llm_decode.py`, and `GenZ/LLM_inference/llm_chunked.py`. An interactive Streamlit frontend is rooted at `Website/Home.py`.

#### Model Construction
Model definitions encode transformer, MoE, and Mamba-style structures and named model families. `GenZ/Models/get_language_model.py` resolves a requested model into an operator-level representation; `GenZ/Models/default_models.py` and family catalogs such as `GenZ/Models/Model_sets/meta.py` supply architecture parameters. Layer/operator implementations reside in `GenZ/Models/attention.py`, `GenZ/Models/ffn.py`, and `GenZ/Models/mamba.py`.

#### Inference Workload Stages
Prefill and token-by-token decode are modeled separately because their sequence lengths, batching behavior, memory pressure, and bottlenecks differ. `GenZ/LLM_inference/llm_prefill.py` models prompt processing, while `GenZ/LLM_inference/llm_decode.py` models generation; `GenZ/LLM_inference/llm_spec_decode.py` adds speculative decoding. `GenZ/LLM_inference/llm_chunked.py` composes chunked-prefill behavior across these stages.

#### Operator Analysis and Cost Model
The model graph is lowered to compute/memory operators, then evaluated against system limits. `GenZ/operators.py` and `GenZ/operator_base.py` define operator abstractions; `GenZ/analyse_model.py` aggregates their execution and memory analysis. `GenZ/db.py`, `GenZ/unit.py`, and `GenZ/power.py` provide performance data handling, unit conversion, and power-related calculations. This boundary separates model/workload semantics from hardware cost estimation.

#### Hardware and Parallel Execution
Hardware platforms are represented by system definitions and can be partitioned across tensor and pipeline parallel dimensions. `GenZ/system.py` defines system-level characteristics, `Systems/system_configs.py` provides platform configurations, and `GenZ/parallelism.py` applies device allocation and parallel execution logic. `GenZ/collective_times.py` models TP ring-allreduce and PP point-to-point communication, making interconnect behavior part of latency estimation.

#### Optimization and Planning
The estimator accepts serving-level choices such as datatype, batching, beam count, and parallelism. `GenZ/LLM_inference/best_parallelization.py` searches/selects favorable parallelization arrangements, while `GenZ/LLM_inference/platform_size.py` estimates required platform scale. `GenZ/LLM_inference/utils.py` acts as shared orchestration support across inference modes.

#### External Simulation and Delivery Boundaries
Analytic estimation can be complemented by simulator adapters: `GenZ/Scale_Sim/get_scale_sim_time.py` targets Scale-Sim, and `GenZ/Astra_sim/get_astra_sim_time.py` plus `GenZ/Astra_sim/run.sh` invoke Astra-sim using network/memory/system inputs. Astra-sim depends on the `GenZ/aiconfigurator` submodule. The packaged library is configured through `pyproject.toml`; the Streamlit UI (`Website/Home.py` and `Website/pages/`) consumes the same analysis capabilities rather than defining a separate backend.

Main entry points: [`GenZ/LLM_inference/llm_decode.py`](https://github.com/abhibambhaniya/genz-llm-analyzer/blob/main/GenZ/LLM_inference/llm_decode.py) (Decode estimator), [`GenZ/analyse_model.py`](https://github.com/abhibambhaniya/genz-llm-analyzer/blob/main/GenZ/analyse_model.py) (Execution and memory analysis), [`GenZ/operators.py`](https://github.com/abhibambhaniya/genz-llm-analyzer/blob/main/GenZ/operators.py) (Operator abstractions), [`GenZ/LLM_inference/llm_prefill.py`](https://github.com/abhibambhaniya/genz-llm-analyzer/blob/main/GenZ/LLM_inference/llm_prefill.py) (Prefill estimator)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_model["Model construction"]
  node_model_catalogs["Model family catalogs<br/>architecture data<br/>[default_models.py]"]
  node_model_resolver["Model resolver<br/>model builder"]
  node_layer_operators["Transformer and Mamba layers<br/>layer implementations<br/>[attention.py]"]
end

subgraph group_inference["Inference planning"]
  node_prefill["Prefill estimator<br/>inference entry point<br/>[llm_prefill.py]"]
  node_decode["Decode estimator<br/>inference entry point<br/>[llm_decode.py]"]
  node_spec_decode["Speculative decode<br/>inference mode<br/>[llm_spec_decode.py]"]
  node_chunked["Chunked prefill<br/>inference composer<br/>[llm_chunked.py]"]
  node_serving_planner["Serving planner<br/>optimization tools"]
end

subgraph group_analysis["Cost analysis"]
  node_operator_model["Operator abstractions<br/>operator model<br/>[operators.py]"]
  node_model_analysis["Execution and memory analysis<br/>cost aggregator<br/>[analyse_model.py]"]
  node_cost_support["Performance, units and power<br/>cost support<br/>[db.py]"]
end

subgraph group_hardware["Hardware execution"]
  node_system_model["System model<br/>hardware abstraction<br/>[system.py]"]
  node_platform_configs["Platform configurations<br/>hardware catalog<br/>[system_configs.py]"]
  node_parallel_runtime["Parallel execution<br/>TP/PP planner<br/>[parallelism.py]"]
  node_collectives["Collective communication<br/>interconnect model"]
end

subgraph group_delivery["Delivery and simulation"]
  node_streamlit_ui["Streamlit UI<br/>interactive frontend<br/>[Home.py]"]
  node_scale_sim["Scale-Sim adapter<br/>simulator adapter"]
  node_astra_sim["Astra-sim adapter<br/>simulator adapter"]
  node_aiconfigurator["AIConfigurator submodule<br/>simulation dependency"]
end

node_model_catalogs -->|"architecture parameters"| node_model_resolver
node_model_resolver -->|"builds from"| node_layer_operators
node_layer_operators -->|"lowers to"| node_operator_model
node_model_resolver -->|"model representation"| node_prefill
node_model_resolver -->|"model representation"| node_decode
node_prefill -->|"prompt stage"| node_chunked
node_decode -->|"generation stage"| node_chunked
node_decode -->|"base generation model"| node_spec_decode
node_prefill -->|"workload operators"| node_operator_model
node_decode -->|"workload operators"| node_operator_model
node_spec_decode -->|"workload operators"| node_operator_model
node_chunked -->|"workload operators"| node_operator_model
node_operator_model -->|"execution and memory costs"| node_model_analysis
node_cost_support -->|"performance and power data"| node_model_analysis
node_platform_configs -->|"platform definitions"| node_system_model
node_system_model -->|"hardware limits"| node_parallel_runtime
node_parallel_runtime -->|"TP/PP communication"| node_collectives
node_parallel_runtime -->|"parallel execution constraints"| node_model_analysis
node_collectives -->|"communication latency"| node_model_analysis
node_serving_planner -->|"searches arrangements"| node_parallel_runtime
node_prefill -->|"serving choices"| node_serving_planner
node_decode -->|"serving choices"| node_serving_planner
node_streamlit_ui -->|"invokes"| node_prefill
node_streamlit_ui -->|"invokes"| node_decode
node_model_analysis -.->|"optional validation"| node_scale_sim
node_model_analysis -.->|"optional validation"| node_astra_sim
node_aiconfigurator -->|"dependency"| node_astra_sim

click node_model_catalogs "https://github.com/abhibambhaniya/genz-llm-analyzer/blob/main/GenZ/Models/default_models.py"
click node_model_resolver "https://github.com/abhibambhaniya/genz-llm-analyzer/blob/main/GenZ/Models/get_language_model.py"
click node_layer_operators "https://github.com/abhibambhaniya/genz-llm-analyzer/blob/main/GenZ/Models/attention.py"
click node_prefill "https://github.com/abhibambhaniya/genz-llm-analyzer/blob/main/GenZ/LLM_inference/llm_prefill.py"
click node_decode "https://github.com/abhibambhaniya/genz-llm-analyzer/blob/main/GenZ/LLM_inference/llm_decode.py"
click node_spec_decode "https://github.com/abhibambhaniya/genz-llm-analyzer/blob/main/GenZ/LLM_inference/llm_spec_decode.py"
click node_chunked "https://github.com/abhibambhaniya/genz-llm-analyzer/blob/main/GenZ/LLM_inference/llm_chunked.py"
click node_serving_planner "https://github.com/abhibambhaniya/genz-llm-analyzer/blob/main/GenZ/LLM_inference/best_parallelization.py"
click node_operator_model "https://github.com/abhibambhaniya/genz-llm-analyzer/blob/main/GenZ/operators.py"
click node_model_analysis "https://github.com/abhibambhaniya/genz-llm-analyzer/blob/main/GenZ/analyse_model.py"
click node_cost_support "https://github.com/abhibambhaniya/genz-llm-analyzer/blob/main/GenZ/db.py"
click node_system_model "https://github.com/abhibambhaniya/genz-llm-analyzer/blob/main/GenZ/system.py"
click node_platform_configs "https://github.com/abhibambhaniya/genz-llm-analyzer/blob/main/Systems/system_configs.py"
click node_parallel_runtime "https://github.com/abhibambhaniya/genz-llm-analyzer/blob/main/GenZ/parallelism.py"
click node_collectives "https://github.com/abhibambhaniya/genz-llm-analyzer/blob/main/GenZ/collective_times.py"
click node_streamlit_ui "https://github.com/abhibambhaniya/genz-llm-analyzer/blob/main/Website/Home.py"
click node_scale_sim "https://github.com/abhibambhaniya/genz-llm-analyzer/blob/main/GenZ/Scale_Sim/get_scale_sim_time.py"
click node_astra_sim "https://github.com/abhibambhaniya/genz-llm-analyzer/blob/main/GenZ/Astra_sim/get_astra_sim_time.py"
click node_aiconfigurator "https://github.com/abhibambhaniya/genz-llm-analyzer/tree/main/GenZ/aiconfigurator"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_model_catalogs,node_model_resolver,node_layer_operators toneBlue
class node_prefill,node_decode,node_spec_decode,node_chunked,node_serving_planner toneAmber
class node_operator_model,node_model_analysis,node_cost_support toneMint
class node_system_model,node_platform_configs,node_parallel_runtime,node_collectives toneRose
class node_streamlit_ui,node_scale_sim,node_astra_sim,node_aiconfigurator toneIndigo
```

## Components

### Model construction

- [Model family catalogs](https://github.com/abhibambhaniya/genz-llm-analyzer/blob/main/GenZ/Models/default_models.py): architecture data · Named families and default architecture parameters. · `GenZ/Models/default_models.py`
- [Model resolver](https://github.com/abhibambhaniya/genz-llm-analyzer/blob/main/GenZ/Models/get_language_model.py): model builder · Resolves requested models into operator-level representations. · `GenZ/Models/get_language_model.py`
- [Transformer and Mamba layers](https://github.com/abhibambhaniya/genz-llm-analyzer/blob/main/GenZ/Models/attention.py): layer implementations · `GenZ/Models/attention.py`

### Inference planning

- [Prefill estimator](https://github.com/abhibambhaniya/genz-llm-analyzer/blob/main/GenZ/LLM_inference/llm_prefill.py): inference entry point · Models prompt processing. · `GenZ/LLM_inference/llm_prefill.py`
- [Decode estimator](https://github.com/abhibambhaniya/genz-llm-analyzer/blob/main/GenZ/LLM_inference/llm_decode.py): inference entry point · Models token-by-token generation. · `GenZ/LLM_inference/llm_decode.py`
- [Speculative decode](https://github.com/abhibambhaniya/genz-llm-analyzer/blob/main/GenZ/LLM_inference/llm_spec_decode.py): inference mode · `GenZ/LLM_inference/llm_spec_decode.py`
- [Chunked prefill](https://github.com/abhibambhaniya/genz-llm-analyzer/blob/main/GenZ/LLM_inference/llm_chunked.py): inference composer · Composes chunked-prefill behavior across stages. · `GenZ/LLM_inference/llm_chunked.py`
- [Serving planner](https://github.com/abhibambhaniya/genz-llm-analyzer/blob/main/GenZ/LLM_inference/best_parallelization.py): optimization tools · Selects parallelism and estimates platform scale. · `GenZ/LLM_inference/best_parallelization.py`

### Cost analysis

- [Operator abstractions](https://github.com/abhibambhaniya/genz-llm-analyzer/blob/main/GenZ/operators.py): operator model · Compute and memory operator definitions. · `GenZ/operators.py`
- [Execution and memory analysis](https://github.com/abhibambhaniya/genz-llm-analyzer/blob/main/GenZ/analyse_model.py): cost aggregator · Aggregates operator execution and memory estimates. · `GenZ/analyse_model.py`
- [Performance, units and power](https://github.com/abhibambhaniya/genz-llm-analyzer/blob/main/GenZ/db.py): cost support · `GenZ/db.py`

### Hardware execution

- [System model](https://github.com/abhibambhaniya/genz-llm-analyzer/blob/main/GenZ/system.py): hardware abstraction · System-level hardware characteristics. · `GenZ/system.py`
- [Platform configurations](https://github.com/abhibambhaniya/genz-llm-analyzer/blob/main/Systems/system_configs.py): hardware catalog · `Systems/system_configs.py`
- [Parallel execution](https://github.com/abhibambhaniya/genz-llm-analyzer/blob/main/GenZ/parallelism.py): TP/PP planner · Applies device allocation and tensor/pipeline parallelism. · `GenZ/parallelism.py`
- [Collective communication](https://github.com/abhibambhaniya/genz-llm-analyzer/blob/main/GenZ/collective_times.py): interconnect model · Models TP ring-allreduce and PP point-to-point time. · `GenZ/collective_times.py`

### Delivery and simulation

- [Streamlit UI](https://github.com/abhibambhaniya/genz-llm-analyzer/blob/main/Website/Home.py): interactive frontend · Consumes the shared analysis capabilities. · `Website/Home.py`
- [Scale-Sim adapter](https://github.com/abhibambhaniya/genz-llm-analyzer/blob/main/GenZ/Scale_Sim/get_scale_sim_time.py): simulator adapter · `GenZ/Scale_Sim/get_scale_sim_time.py`
- [Astra-sim adapter](https://github.com/abhibambhaniya/genz-llm-analyzer/blob/main/GenZ/Astra_sim/get_astra_sim_time.py): simulator adapter · Invokes Astra-sim with network, memory, and system inputs. · `GenZ/Astra_sim/get_astra_sim_time.py`
- [AIConfigurator submodule](https://github.com/abhibambhaniya/genz-llm-analyzer/tree/main/GenZ/aiconfigurator): simulation dependency · `GenZ/aiconfigurator`

## Connections

- Model family catalogs → Model resolver: architecture parameters
- Model resolver → Transformer and Mamba layers: builds from
- Transformer and Mamba layers → Operator abstractions: lowers to
- Model resolver → Prefill estimator: model representation
- Model resolver → Decode estimator: model representation
- Prefill estimator → Chunked prefill: prompt stage
- Decode estimator → Chunked prefill: generation stage
- Decode estimator → Speculative decode: base generation model
- Prefill estimator → Operator abstractions: workload operators
- Decode estimator → Operator abstractions: workload operators
- Speculative decode → Operator abstractions: workload operators
- Chunked prefill → Operator abstractions: workload operators
- Operator abstractions → Execution and memory analysis: execution and memory costs
- Performance, units and power → Execution and memory analysis: performance and power data
- Platform configurations → System model: platform definitions
- System model → Parallel execution: hardware limits
- Parallel execution → Collective communication: TP/PP communication
- Parallel execution → Execution and memory analysis: parallel execution constraints
- Collective communication → Execution and memory analysis: communication latency
- Serving planner → Parallel execution: searches arrangements
- Prefill estimator → Serving planner: serving choices
- Decode estimator → Serving planner: serving choices
- Streamlit UI → Prefill estimator: invokes
- Streamlit UI → Decode estimator: invokes
- Execution and memory analysis → Scale-Sim adapter: optional validation
- Execution and memory analysis → Astra-sim adapter: optional validation
- AIConfigurator submodule → Astra-sim adapter: dependency
