# sloweyyy/experimenthub architecture

> Architecture diagram of the GitHub repository sloweyyy/experimenthub, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sloweyyy/experimenthub
- Repository: https://github.com/sloweyyy/experimenthub

## Overview

##### Main System Components
- Frontend:
- Located in the experimenthub directory.
- Built with Next.js (with React and Tailwind CSS) and uses a component-based design.
- Handles user interactions such as managing experiments, displaying interactive visualizations, and job management pages.
- Backend:
- Located in the backend directory.
- Uses FastAPI to serve a REST API (and potentially WebSocket connectivity for real-time training monitoring).
- Houses the main application logic for experiment management, model training routines (e.g., MNIST model training found in backend/models), and database interactions.
- Data and Persistence:
- There is a SQLite database file (experiment_db.sqlite) used for storage.
- The backend/data directory contains datasets (e.g., MNIST raw training and test files) that support training operations.
- Machine Learning Components:
- Model definitions and training routines reside in backend/models.
- The application includes configurable hyperparameters for different model types (MLP, CNN, RNN).
- Job Queue & Real-time Monitoring:
- The README mentions a job queue to manage multiple training jobs.
- Data flow might include WebSocket connections to update clients with training progress.
- Docker & Deployment:
- Dockerfiles for both frontend and backend along with docker-compose configuration indicate containerized deployment.

##### Interactions Between Components
- User requests (HTTP/WebSocket) from the Frontend communicate to the Backend (API Gateway).
- The Frontend consumes the REST API endpoints as well as push/pull real-time data.
- The Backend interacts with the SQLite database for experiment management.
- The ML training modules in the Backend process incoming experiment requests, access the MNIST dataset, and execute training, updating the database and triggering progress via WebSockets.
- Docker Compose orchestrates the running containers (separate images for frontend and backend).

##### Architectural Patterns and Design Principles
- Separation of Concerns: Clear division into frontend and backend.
- Microservices/Modular Design: Even though there’s a single backend service, different directories indicate modular code separation for API endpoints, data management, and ML training.
- Containerization: Dockerfiles and docker-compose indicate that each component runs in its own container.
- Real-Time Communication: Use of WebSockets for monitoring job progress, which is a key interactive design aspect.

##### Elements
- Components:
- Frontend (Next.js App with pages, components, and static assets from experimenthub)
- Backend (FastAPI server, data processing modules, ML training services)
- Database (SQLite file represented as the persistent data store)
- Data Storage (Directory holding MNIST dataset and any experiment logging data)
- Docker & Deployment (Containers for both the frontend and backend, orchestrated with docker-compose)
- Relationships and Data Flow:
- Flow between the ML training modules and the Database for updating experiment status.
- Visual Aids:
- Grouping related components using containers (for instance, group all frontend files together and all backend services together).

##### Detailed Steps for the Principal Software Engineer
2. In the Frontend block, detail that it's a Next.js application handling routes such as experiments and jobs with UI components.
3. In the Backend block, illustrate:
- FastAPI as the API gateway.
- Sub-components like:
- API Handlers (in app/main.py),
- ML Training Modules (mnist_model.py, trainer.py),
- Database access (database.py).
- Data processing pipeline for the MNIST dataset.
5. A Docker layer outside these components, indicating that both the Frontend and Backend are containerized and orchestrated with docker-compose.
6. Detail the interactivity between user inputs (via the UI) and the underlying model training jobs managed by the backend’s job queue.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Containerization Layer
    subgraph "Containerization"
        DC["Docker Compose Orchestrator"]:::docker
        DFE["Dockerfile Frontend"]:::docker
        DFB["Dockerfile Backend"]:::docker
    end

    %% Frontend Container
    subgraph "Frontend Container"
        FE["Next.js Application (UI)"]:::frontend
        FEJobs["Job Management UI"]:::frontend
    end

    %% Backend Container
    subgraph "Backend Container"
        API["FastAPI Server (API Handlers)"]:::backend
        ML["ML Model Training Modules"]:::ml
    end

    %% Data and Persistence
    subgraph "Data & Persistence"
        DB["SQLite Database"]:::database
        Dataset["MNIST Dataset"]:::database
    end

    %% Docker to Containers
    DC --> FE
    DC --> API
    DFE --> FE
    DFB --> API

    %% Interaction between Frontend and Backend
    FE -->|"RESTAPI/WebSocket"| API
    API -->|"TriggersTraining"| ML
    ML -->|"ReadsData"| Dataset
    ML -->|"UpdatesStatus"| DB
    API -->|"SendsProgress"| FE
    FEJobs -->|"RequestsJobStatus"| API

    %% Styles
    classDef frontend fill:#DDEEFF,stroke:#333,stroke-width:2px;
    classDef backend fill:#FFEEDD,stroke:#333,stroke-width:2px;
    classDef database fill:#ddffdd,stroke:#333,stroke-width:2px;
    classDef docker fill:#ffddff,stroke:#333,stroke-width:2px;
    classDef ml fill:#ffffcc,stroke:#333,stroke-width:2px;

    %% Click Events
    click FE "https://github.com/sloweyyy/experimenthub/tree/main/experimenthub"
    click FEJobs "https://github.com/sloweyyy/experimenthub/tree/main/experimenthub/app/jobs"
    click API "https://github.com/sloweyyy/experimenthub/tree/main/backend/app"
    click ML "https://github.com/sloweyyy/experimenthub/tree/main/backend/models"
    click DB "https://github.com/sloweyyy/experimenthub/blob/main/backend/experiment_db.sqlite"
    click Dataset "https://github.com/sloweyyy/experimenthub/tree/main/backend/data/MNIST"
    click DC "https://github.com/sloweyyy/experimenthub/blob/main/docker-compose.yml"
    click DFE "https://github.com/sloweyyy/experimenthub/blob/main/Dockerfile.frontend"
    click DFB "https://github.com/sloweyyy/experimenthub/blob/main/Dockerfile.backend"
```
