# infobellit-solutions-pvt-ltd/echoswift architecture

> Architecture diagram of the GitHub repository infobellit-solutions-pvt-ltd/echoswift, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/infobellit-solutions-pvt-ltd/echoswift
- Repository: https://github.com/infobellit-solutions-pvt-ltd/echoswift

## Overview

##### Project Type & Purpose
- EchoSwift is an LLM Inference Benchmarking Tool aimed at measuring performance metrics (latency, throughput, TTFT) for various LLM inference endpoints. It uses a CLI interface and handles dataset preparation, benchmark execution, and result visualization.
- The README confirms the primary use case (benchmarking LLM inference performance across various inference servers) and gives detailed instructions on commands (dataprep, start, plot).

##### File Structure
- Top-Level Files:
- README.md and LICENSE give context about the project.
- config.json holds user-specified parameters for benchmarking.
- setup.py and requirements.txt indicate the project’s nature as an installable Python package.
- Main Module Directory (echoswift):
- __init__.py for package initialization.
- cli.py: Implements the command-line interface.
- dataset.py: Handles dataset preparation (e.g., downloading and filtering data as described in the README).
- llm_inference_benchmark.py and llm_inference_master.py: Likely contain the benchmark orchestration code, managing how inference tasks are run and how results are aggregated.
- utils/ contains helper modules including:
- avg_locust_results.py: Likely for aggregating results from Locust (load testing) runs.
- plot_results.py: For generating visual plots of benchmark outcomes.
- Other Directories:
- .github/workflows contains CI/CD pipeline configurations, suggesting automated testing or deployment (e.g., publishing the package).
- tests/ contains unit tests verifying CLI functionality.

##### Main Components & Their Relationships
a. Key Components:
- CLI Module (cli.py): Entry point, parses commands (dataprep, start, plot).
- Dataset Management (dataset.py): Downloads and filters data.
- Benchmarking Engine:
- llm_inference_benchmark.py: Executes benchmarks based on input parameters.
- llm_inference_master.py: Possibly coordinates the benchmarks over multiple concurrent users or test configurations.
- Result Processing:
- utils/avg_locust_results.py: Processes and averages the raw benchmark data.
- utils/plot_results.py: Visualizes the metrics (latency, throughput, TTFT).
- Configuration Management: config.json provides benchmark parameters.
- Testing & CI/CD: tests directory and GitHub workflows ensure code quality and deployment integrity.

b. Relationships & Data Flow:
- The CLI parses user commands and routes them to the appropriate module.
- For “dataprep”, the CLI calls dataset.py to download and preprocess the data from a source (Huggingface).
- For “start”, the CLI reads config.json, then triggers the benchmarking process via llm_inference_benchmark.py and llm_inference_master.py, which in turn communicate with external LLM inference endpoints (TGI, vLLM, etc.).
- The benchmark process logs results (probably as CSV files) into a specified output directory.

##### Architectural Patterns & Design Principles
- Modular design: Separation of CLI, benchmark logic, dataset management, and result processing.
- Configurability: Use of an external config.json to allow users to easily modify benchmark parameters.
- Extensibility: The codebase supports multiple inference servers and can be extended to include new benchmarking metrics.
- Standard practices in Python package development: Setup files, tests, and CI/CD pipelines.

##### Elements
- Components (e.g., CLI, Data Preparation, Benchmark Engine, Visualization, Testing/CI)
- Dataset download/preprocessing → Benchmark execution → Results logging and aggregation → Visualization.
- Interface components (e.g., CLI)
- Core service modules (benchmark engine, dataset module)
- Helper utilities (result processors, plotting)
- External systems (LLM inference servers, CI/CD pipelines)

## Diagram (Mermaid)

```mermaid
flowchart TD
    User["User Command"]:::interface

    subgraph "Interface Layer"
        CLI["CLI Module"]:::interface
    end

    subgraph "Application Core"
        DP["Dataset Management"]:::core
        BE["Benchmarking Engine (Execution)"]:::core
        BM["Benchmarking Engine (Orchestration)"]:::core
        RA["Result Processing (Aggregation)"]:::utils
        RV["Result Processing (Visualization)"]:::utils
        Config["Configuration Management"]:::config
    end

    subgraph "External & QA"
        External["LLM Inference Endpoints"]:::external
        CI["CI/CD Pipeline"]:::qa
        Tests["Tests"]:::qa
    end

    %% Data flows
    User -->|"triggers"| CLI
    CLI -->|"dataprep"| DP
    CLI -->|"start"| Config
    Config -->|"inputs"| BE
    Config -->|"inputs"| BM
    BM -->|"coordinates"| BE
    BE -->|"calls"| External
    BE -->|"logsResults"| RA
    RA -->|"aggregates"| RV
    CLI -->|"plot"| RV

    CI --- Tests

    %% Styles
    classDef interface fill:#f9f,stroke:#333,stroke-width:2px;
    classDef core fill:#bbf,stroke:#333,stroke-width:2px;
    classDef utils fill:#bfb,stroke:#333,stroke-width:2px;
    classDef config fill:#ff9,stroke:#333,stroke-width:2px;
    classDef external fill:#f99,stroke:#333,stroke-width:2px;
    classDef qa fill:#9cf,stroke:#333,stroke-width:2px;

    %% Click Events
    click CLI "https://github.com/infobellit-solutions-pvt-ltd/echoswift/blob/main/echoswift/cli.py"
    click DP "https://github.com/infobellit-solutions-pvt-ltd/echoswift/blob/main/echoswift/dataset.py"
    click BE "https://github.com/infobellit-solutions-pvt-ltd/echoswift/blob/main/echoswift/llm_inference_benchmark.py"
    click BM "https://github.com/infobellit-solutions-pvt-ltd/echoswift/blob/main/echoswift/llm_inference_master.py"
    click RA "https://github.com/infobellit-solutions-pvt-ltd/echoswift/blob/main/echoswift/utils/avg_locust_results.py"
    click RV "https://github.com/infobellit-solutions-pvt-ltd/echoswift/blob/main/echoswift/utils/plot_results.py"
    click Config "https://github.com/infobellit-solutions-pvt-ltd/echoswift/blob/main/config.json"
    click CI "https://github.com/infobellit-solutions-pvt-ltd/echoswift/tree/main/.github/workflows"
    click Tests "https://github.com/infobellit-solutions-pvt-ltd/echoswift/tree/main/tests"
```
