# fissionai/flotorch architecture

> Architecture diagram of the GitHub repository fissionai/flotorch, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/fissionai/flotorch
- Repository: https://github.com/fissionai/flotorch

## Overview

##### Project Type and Purpose
- FloTorch is designed to simplify and automate the evaluation of LLM setups. Its purpose is to reduce the trial-and-error process by supporting parallel experiments, providing deterministic evaluations, and integrating cloud services for scalability and robustness.
- The README emphasizes a “well-architected framework” with pillars such as security, performance, and cost optimization. The product is positioned as an innovative generative AI solution that automates complex experimentation workflows.

##### File Structure for Architectural Clues
- The project is clearly split into several top-level sections:
- The “app” directory contains the primary backend service (its main.py, routing logic, and configuration) along with nginx configurations for request routing and load balancing.
- The “ui” folder represents the frontend built with modern JavaScript frameworks (likely Nuxt/Vue) and includes configuration files, assets, and server configurations.
- Directories like “core”, “lambda_handlers”, “evaluation”, “indexing”, and “retriever” indicate specialized microservices or modules that handle specific tasks (like embedding, inference, evaluation, indexing, and retrieval).
- The presence of CloudFormation templates in the “cfn” folder and deployment scripts (Dockerfiles in multiple locations, provision.sh, etc.) shows the emphasis on AWS cloud deployment. Specific templates hint at App Runner, ECS, ECR, Lambda, VPC setups, and even OpenSearch integrations.
- There is also an organized structure for configuration, constants, dependencies, and utility functions, suggesting a modular design that favors separation of concerns.

##### Key Components and Their Interactions
a. Core Components:
- Frontend (UI): The “ui” folder holds the Nuxt/Vue-based application components, pages, and plugins responsible for the user interface and client-side logic.
- Backend Application: Under “app” you have the primary service that initializes the application (with the entry point app/main.py), sets up routes (in app/routes), and integrates third-party services via dependencies (e.g., database and S3 in app/dependencies).
- Orchestration and Utility Modules: The “core” directory contains business logic for chunking, embeddings, evaluation, inference, and more – these are the heart of the experiment logic and decision-making processes.
- Microservices & Handlers: Modules like “evaluation”, “indexing”, “retriever”, and “lambda_handlers” allow for segregated functions that can be deployed independently. They are likely containerized (as seen by multiple Dockerfiles) and deployed as separate services on AWS.
- Infrastructure as Code: The “cfn” folder holds CloudFormation templates that provision AWS resources including ECS services, App Runner setups, Lambda functions, VPC endpoints, etc.
- Configuration & Deploy Tools: Files (Dockerfile, provision.sh, etc.) and directories like .github and .codecatalyst indicate CI/CD integration and automated deployment workflows.

b. Relationships and Data Flow:
- The UI interacts with the backend API endpoints provided in the “app/routes”.
- The backend orchestrates experiment configurations and delegates specialized tasks to dedicated services (indexing, evaluation, retrieval) as required.
- Microservices communicate with each other (or via AWS services) to perform specific tasks such as embedding generation (via “core/embedding”), inference processing (via “core/inference”), and evaluating results (via “core/eval”).
- Infrastructure components deployed via CloudFormation (in “cfn”) connect these services to the underlying AWS resources (databases, Lambda handlers, OpenSearch clusters, etc.).
- Utility functions in “util” and “constants” support standardized operations such as error handling, S3 and DynamoDB interactions.

##### Architectural Patterns and Design Principles
- Modular & Microservices: Separate services for indexing, retrieval, evaluation, and orchestration allow for independent scaling and maintenance.
- Layered Architecture: Clear separation between the frontend, API layer, business logic (core), and infrastructure (deployment templates, handlers).
- DevOps & CI/CD: The inclusion of GitHub and CodeCatalyst workflow files, multiple Dockerfiles, and CloudFormation (cfn) templates highlight automated build, test, and deployment pipelines.
- Cloud-Native Deployment: Use of AWS services (as seen in lambda templates, ECS templates, etc.) ensuring scalability, reliability, and cost efficiency.
- External Dependencies Management: The “dependencies” and “requirements.txt” files articulate the use of external libraries, ensuring all components follow decoupled integration patterns.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Frontend Layer
    subgraph "Frontend Layer"
        UI["Frontend (Nuxt/Vue)"]:::ui
    end

    %% Backend Layer
    subgraph "Backend Layer"
        API["Backend API Service"]:::backend
    end

    %% Core Processing
    subgraph "Core & Processing"
        CORE["Core Processing Modules"]:::core
    end

    %% Microservices
    subgraph "Microservices"
        IDX["Indexing Microservice"]:::micro
        EVAL["Evaluation Microservice"]:::micro
        RETR["Retriever Microservice"]:::micro
        LAMB["Lambda Handlers"]:::micro
    end

    %% Infrastructure
    subgraph "Infrastructure"
        INF["Cloud Infrastructure (CloudFormation)"]:::infra
    end

    %% CI/CD & Deployment Tools
    subgraph "CI/CD & Deployment Tools"
        GITHUB["GitHub Actions (.github)"]:::cicd
        CODECAT["CodeCatalyst (.codecatalyst)"]:::cicd
        DOCKER1["Dockerfile (root)"]:::cicd
        DOCKER2["Dockerfile (app)"]:::cicd
        PROV["Provision Script (provision.sh)"]:::cicd
    end

    %% Connections
    UI -->|"requests"| API
    API -->|"orchestrates"| CORE
    CORE -->|"calls"| IDX
    CORE -->|"calls"| EVAL
    CORE -->|"calls"| RETR
    CORE -->|"calls"| LAMB

    API -->|"provisions"| INF
    CORE -->|"supported_by"| INF
    IDX -->|"managed_by"| INF
    EVAL -->|"managed_by"| INF
    RETR -->|"managed_by"| INF
    LAMB -->|"managed_by"| INF

    GITHUB -->|"deploys"| API
    CODECAT -->|"deploys"| API
    DOCKER1 -->|"containerizes"| API
    DOCKER2 -->|"containerizes"| API
    PROV -->|"provisions"| API

    %% Styles
    classDef ui fill:#AED6F1,stroke:#2E86C1,stroke-width:2px;
    classDef backend fill:#FAD7A0,stroke:#E67E22,stroke-width:2px;
    classDef core fill:#D5F5E3,stroke:#27AE60,stroke-width:2px;
    classDef micro fill:#F5B7B1,stroke:#C0392B,stroke-width:2px;
    classDef infra fill:#D7BDE2,stroke:#8E44AD,stroke-width:2px;
    classDef cicd fill:#F9E79F,stroke:#D4AC0D,stroke-width:2px;

    %% Click Events
    click UI "https://github.com/fissionai/flotorch/tree/main/ui/"
    click API "https://github.com/fissionai/flotorch/blob/main/app/main.py"
    click CORE "https://github.com/fissionai/flotorch/tree/main/core/"
    click IDX "https://github.com/fissionai/flotorch/tree/main/indexing/"
    click EVAL "https://github.com/fissionai/flotorch/tree/main/evaluation/"
    click RETR "https://github.com/fissionai/flotorch/tree/main/retriever/"
    click LAMB "https://github.com/fissionai/flotorch/tree/main/lambda_handlers/"
    click INF "https://github.com/fissionai/flotorch/tree/main/cfn/"
    click GITHUB "https://github.com/fissionai/flotorch/tree/main/.github/"
    click CODECAT "https://github.com/fissionai/flotorch/tree/main/.codecatalyst/"
    click DOCKER1 "https://github.com/fissionai/flotorch/tree/main/Dockerfile"
    click DOCKER2 "https://github.com/fissionai/flotorch/tree/main/app/Dockerfile"
    click PROV "https://github.com/fissionai/flotorch/blob/main/provision.sh"
```
