# isaac-art/cvec_web architecture

> Architecture diagram of the GitHub repository isaac-art/cvec_web, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2024-12-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/isaac-art/cvec_web
- Repository: https://github.com/isaac-art/cvec_web

## Overview

##### Web Server Layer
- Main entry point (main.py) should be highlighted
- Static file serving for HTML files (static/*.html)

##### Core Processing Components
- Repeng Library Integration (repeng folder)
- Control module (control.py)
- Extraction module (extract.py)
- SAES module (saes.py)
- Testing component (tests.py)

##### Worker System
- Separate worker process (run_worker.py)
- Training queue management
- Vector processing pipeline

##### Data Storage
- SQLite database (app.db)
- Model cache (HF_HOME reference)

##### External Dependencies
- Vector storage system

##### Use a layered architecture approach
- [Client Layer (Browser)]
- ↕
- [Web Server Layer (FastAPI)]
- ↕
- [Application Core]
- Vector Processing
- Control Logic
- Extraction System
- ↕
- [Worker Process]
- ↕
- [Storage Layer]

##### Component Relationships
- How the web interface communicates with the core processing
- Queue system between main server and worker
- Demonstrate data persistence flows

##### Repeng library integration
- Modular structure of the repeng package
- It to both the main server and worker process

##### Worker pipeline
- Queue intake
- Training process
- Vector storage

##### Database Interactions
- Connections to app.db
- Vector storage locations
- Cache management for HF models

Remember to:
- Use consistent symbols throughout
- Version information
- Document any important configuration dependencies (like HF_HOME)

## Diagram (Mermaid)

```mermaid
graph TB
    %% Client Layer
    subgraph Client
        Browser["Web Browser"]
    end

    %% Web Server Layer
    subgraph WebServer
        FastAPI["FastAPI Server"]:::server
        click FastAPI "https://github.com/isaac-art/cvec_web/blob/main/main.py"
        
        subgraph WebInterface["Static Web Interface"]
            Index["Index Page"]:::ui
            Axis["Axis Page"]:::ui
            Moon["Moon Page"]:::ui
            click Index "https://github.com/isaac-art/cvec_web/blob/main/static/index.html"
            click Axis "https://github.com/isaac-art/cvec_web/blob/main/static/axis.html"
            click Moon "https://github.com/isaac-art/cvec_web/blob/main/static/moon.html"
        end
    end

    %% Core Processing Layer
    subgraph CoreProcessing["Application Core"]
        subgraph Repeng["Repeng Library"]
            Control["Control Module"]:::core
            Extract["Extraction Module"]:::core
            SAES["SAES Module"]:::core
            Tests["Testing Component"]:::core
            click Control "https://github.com/isaac-art/cvec_web/blob/main/repeng/control.py"
            click Extract "https://github.com/isaac-art/cvec_web/blob/main/repeng/extract.py"
            click SAES "https://github.com/isaac-art/cvec_web/blob/main/repeng/saes.py"
            click Tests "https://github.com/isaac-art/cvec_web/blob/main/repeng/tests.py"
            click Repeng "https://github.com/isaac-art/cvec_web/tree/main/repeng/"
        end
        Settings["Settings Config"]:::config
        click Settings "https://github.com/isaac-art/cvec_web/blob/main/settings.py"
    end

    %% Worker System
    subgraph WorkerSystem
        Worker["Worker Process"]:::worker
        Queue["Training Queue"]:::queue
        click Worker "https://github.com/isaac-art/cvec_web/blob/main/run_worker.py"
    end

    %% Storage Layer
    subgraph Storage
        DB[(SQLite Database)]:::db
        ModelCache[("HF Model Cache")]:::cache
        VectorStore[("Vector Storage")]:::storage
        click DB "https://github.com/isaac-art/cvec_web/blob/main/app.db"
    end

    %% External Dependencies
    subgraph External
        HF["Hugging Face\nMeta-Llama-3-8B"]:::external
    end

    %% Relationships
    Browser <--> FastAPI
    FastAPI --> WebInterface
    FastAPI <--> Repeng
    FastAPI --> Queue
    Worker --> Queue
    Worker <--> Repeng
    Repeng <--> DB
    Worker <--> DB
    Worker <--> ModelCache
    Worker <--> VectorStore
    ModelCache <--> HF
    Settings -.- FastAPI
    Settings -.- Worker

    %% Styles
    classDef server fill:#3498db,stroke:#2980b9,color:white
    classDef ui fill:#95a5a6,stroke:#7f8c8d,color:white
    classDef core fill:#2ecc71,stroke:#27ae60,color:white
    classDef worker fill:#f1c40f,stroke:#f39c12,color:black
    classDef queue fill:#e67e22,stroke:#d35400,color:white
    classDef db fill:#95a5a6,stroke:#7f8c8d,color:white
    classDef cache fill:#95a5a6,stroke:#7f8c8d,color:white
    classDef storage fill:#95a5a6,stroke:#7f8c8d,color:white
    classDef external fill:#e74c3c,stroke:#c0392b,color:white
    classDef config fill:#9b59b6,stroke:#8e44ad,color:white

    %% Legend
    subgraph Legend
        L1["Web Components"]:::server
        L2["UI Elements"]:::ui
        L3["Core Processing"]:::core
        L4["Worker System"]:::worker
        L5["Storage Systems"]:::db
        L6["External Services"]:::external
        L7["Configuration"]:::config
    end
```
