# kuleuven-micas/stream architecture

> Architecture diagram of the GitHub repository kuleuven-micas/stream, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kuleuven-micas/stream
- Repository: https://github.com/kuleuven-micas/stream

## Overview

##### Project Type and Purpose
- Stream is a design space exploration (DSE) framework focused on hardware architecture-mapping for multi-core deep learning accelerators. It allows exploration of mappings at various granularities (from layer-by-layer to fine-grained, layer-fused processing), and it builds upon an existing framework (ZigZag).
- The README emphasizes installation, running a tutorial, and extensive documentation, which indicates that while it is a research-oriented DSE tool, it includes structured components catering to academic and practical evaluation of accelerator mapping strategies.

##### File Structure
- The top-level directories and files include configuration directories (".devcontainer", ".github"), documentation ("docs"), main scripts ("main_stream_co.py", "main_stream_ga.py"), and the main module folder ("stream").
- Inside the stream folder, the structure is split into several clear components:
- cost_model: Contains components for modeling communication, memory, scheduling, and group allocation. These likely implement the evaluation metrics for different mapping strategies.
- hardware: Defines the hardware architecture. This is split into submodules like “accelerator”, “core”, and “noc” (network-on-chip from "communication_link.py") along with common utilities.
- inputs: Contains examples and testing files for hardware configurations, mappings, and workloads (ONNX files and YAML definitions). It acts as the ingestion layer for design inputs.
- opt: Contains optimization routines, further divided for allocation (including genetic algorithm and constraint optimization), partitioning, and scheduling. These modules probably refine the design exploration process.
- parser: Contains submodules to build and validate representations of hardware, mappings, and ONNX models. This phase converts input definitions into internal representations.
- stages: Represents the sequential phases of processing such as parsing, allocation, estimation, and generation. It orchestrates the overall DSE workflow.
- workload: Consists of the internal representation and computation models for neural networks (including dependency propagation for operations) and ONNX workload parsing.
- Additional utility and node_tensor support help glue the system together.
- There are also configuration files (e.g., pyproject.toml, .pre-commit-config.yaml, GitHub workflows) which indicate adherence to modern development and CI/CD practices.

- Frontend/Interface:
- Main entry points: “main_stream_co.py” and “main_stream_ga.py” (possibly corresponding to different exploration strategies – one might be a conventional/coarse method and the other genetic algorithm based).
- The “inputs” directory acts as the ingestion layer for hardware definitions, runtime workloads (ONNX models), and mapping configurations.
- Parsing Layer:
- “parser” module: Converts the raw inputs into validated internal models (hardware, mapping, workload). It interacts directly with the “inputs” files.
- Core Processing Stages:
- “stages”: Orchestrates the sequential workflow. Specific stages include:
- Parsing Stage: Handled by components in “stages/parsing”.
- Allocation/Optimization Stage: Uses the “opt” components such as “allocation” (with genetic algorithm and constraint optimization) and “scheduling” (for layer stack arrangement).
- Estimation Stage: Located in “stages/estimation” with evaluation models and cost estimation routines.
- Generation Stage: Involves scheduling order and tiling generation.
- The “cost_model” and “hardware” submodules are deeply intertwined with the optimization and estimation processes.
- Supporting Components:
- Visualization: Provides graphical and data-driven feedback (found in “visualization”).
- Workload: Structures the computational graph and dependency propagation, being central to mapping decisions.
- External Dependencies:
- The project builds on top of ZigZag, so include an element representing its dependence or a bridge component to that framework.

##### Architectural Patterns and Design Principles
- Modularity: Each major function is encapsulated in its own directory (e.g., input parsing, optimization, visualization), supporting separation of concerns.
- Multi-stage processing: The “stages” directory implies a pipeline-like design typical in DSE and compilation-type frameworks.
- Extensibility: With its clear structure for input examples and testing, the architecture supports extension for new hardware configurations or mapping algorithms.
- Hybrid Optimization: The presence of both genetic algorithms and constraint optimization modules indicates a flexible approach to solving allocation problems.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Entry Points
    subgraph "Entry Points"
        A["main_stream_co.py"]:::frontend
        B["main_stream_ga.py"]:::frontend
    end

    %% Input Layer
    subgraph "Input Layer"
        C["Inputs (Hardware, Mapping, Workload)"]:::ingestion
    end

    %% Parsing Layer
    subgraph "Parsing Layer"
        D["Parser"]:::parsing
    end

    %% Processing/Optimization Pipeline
    subgraph "Processing/Optimization Pipeline"
        E["Stages (Orchestration)"]:::processing
        F["Constraint Optimization"]:::processing
        G["Genetic Algorithm Optimization"]:::processing
        H["Partitioning"]:::processing
        I["Scheduling"]:::processing
    end

    %% Hardware Abstraction & Cost Modeling
    subgraph "Hardware Abstraction & Cost Modeling"
        J1["Hardware"]:::hardware
        J2["Cost Model"]:::hardware
    end

    %% Workload Modeling
    subgraph "Workload Modeling"
        K["Workload Modeling"]:::workload
    end

    %% Visualization & Reporting
    subgraph "Visualization & Reporting"
        L["Visualization"]:::visualization
    end

    %% External Dependency
    subgraph "External Dependency"
        X["ZigZag Framework"]:::external
    end

    %% Connections
    A --> C
    B --> C
    C --> D
    D --> E
    E --> F
    E --> G
    E --> H
    E --> I
    E --> J1
    E --> J2
    E --> K
    F --> L
    G --> L
    H --> L
    I --> L
    J1 --> L
    J2 --> L
    K --> L
    X -->|"depends"| J2

    %% Click Events
    click A "https://github.com/kuleuven-micas/stream/blob/master/main_stream_co.py"
    click B "https://github.com/kuleuven-micas/stream/blob/master/main_stream_ga.py"
    click C "https://github.com/kuleuven-micas/stream/tree/master/stream/inputs"
    click D "https://github.com/kuleuven-micas/stream/tree/master/stream/parser"
    click E "https://github.com/kuleuven-micas/stream/tree/master/stream/stages"
    click F "https://github.com/kuleuven-micas/stream/tree/master/stream/opt/allocation/constraint_optimization"
    click G "https://github.com/kuleuven-micas/stream/tree/master/stream/opt/allocation/genetic_algorithm"
    click H "https://github.com/kuleuven-micas/stream/tree/master/stream/opt/partitioning"
    click I "https://github.com/kuleuven-micas/stream/tree/master/stream/opt/scheduling"
    click J1 "https://github.com/kuleuven-micas/stream/tree/master/stream/hardware"
    click J2 "https://github.com/kuleuven-micas/stream/tree/master/stream/cost_model"
    click K "https://github.com/kuleuven-micas/stream/tree/master/stream/workload"
    click L "https://github.com/kuleuven-micas/stream/tree/master/stream/visualization"

    %% Styles
    classDef frontend fill:#ccf,stroke:#006,stroke-width:2px;
    classDef ingestion fill:#cfc,stroke:#060,stroke-width:2px;
    classDef parsing fill:#fcf,stroke:#606,stroke-width:2px;
    classDef processing fill:#ffc,stroke:#660,stroke-width:2px;
    classDef hardware fill:#cff,stroke:#066,stroke-width:2px;
    classDef workload fill:#f9f,stroke:#909,stroke-width:2px;
    classDef visualization fill:#fdd,stroke:#900,stroke-width:2px;
    classDef external fill:#eef,stroke:#445,stroke-width:2px;
```
