# shayanjm/vec2text architecture

> Architecture diagram of the GitHub repository shayanjm/vec2text, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2024-12-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/shayanjm/vec2text
- Repository: https://github.com/shayanjm/vec2text

## Overview

##### Main Components

a. Core Architecture (Top Level):
- Input Processing Layer
- Embedding Inversion Layer
- Output Generation Layer

b. Model Components (Second Level):
- Create two parallel branches:
- Hypothesizer Path (Zero-step model)
- Corrector Path (Iterative refinement)

##### Data Flow and Processing

a. Input Flow:
- Text Input → Embedding Generation
- Direct Embedding Input
- A "Tokenization" subprocess connected to text input

b. Model Processing:
- Two-phase process:
- Initial Hypothesis Generation
- Iterative Correction Loop

##### Key Architectural Elements

a. Training Components:
- Dataset Processors
- Training Pipeline
- Model Checkpointing

b. Inference Components:
- Parallel paths for:
- Single-step Inference
- Multi-step Inference with Beam Search

##### Implementation Details

a. Model Architecture:
- InversionModel
- CorrectorEncoderModel
- Trainers (Base, Corrector, Inversion)

b. Support Systems:
- Peripheral components:
- Embedding Cache
- Evaluation Metrics
- Model Hub Integration

##### Specific Implementation Notes

- Annotations for:
- Supported embedding types (text-embedding-ada-002, gtr-base)
- Beam search parameters
- Model configuration options

##### Technical Details

- Input/output dimensions
- Key hyperparameters
- Memory requirements for beam search

##### Integration Points

- External connections:
- HuggingFace Hub
- OpenAI API
- PyTorch ecosystem

- The dual-path architecture (hypothesizer and corrector)
- The iterative refinement process
- The flexibility in handling both text and embedding inputs
- The training pipeline's complexity
- The modular nature of the system

This architecture reflects the project's focus on both research and practical application, showing how it handles both training new models and using pre-trained ones for inference.

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph Input
        TI[Text Input]
        EI[Embedding Input]
        TK[Tokenization]:::data
    end

    subgraph Core["Core Processing"]
        direction TB
        IPL[Input Processing Layer]
        EIL[Embedding Inversion Layer]
        OGL[Output Generation Layer]
    end

    subgraph ModelArch["Model Architecture"]
        direction TB
        HP[Hypothesizer]:::model
        CR[Corrector]:::model
        MC[Model Config]:::config
        MU[Model Utils]:::util
    end

    subgraph Training["Training Components"]
        direction TB
        BT[Base Trainer]:::trainer
        CT[Corrector Trainer]:::trainer
        IT[Inversion Trainer]:::trainer
        DP[Dataset Processing]:::data
        CO[Collation]:::data
    end

    subgraph Inference["Inference System"]
        direction TB
        API[API Interface]:::api
        RC[Run Config]:::config
        MR[Main Runner]:::runner
    end

    subgraph Evaluation["Evaluation System"]
        direction TB
        MT[Metrics]:::eval
        AU[Analysis Utils]:::eval
    end

    subgraph Baseline["Baseline Models"]
        direction TB
        FI[Fewshot Inversion]:::baseline
        DI[Decode Inversion]:::baseline
    end

    %% Connections
    TI --> TK
    TK --> IPL
    EI --> IPL
    IPL --> EIL
    EIL --> OGL

    HP --> EIL
    CR --> EIL
    MC --> ModelArch
    MU --> ModelArch

    DP --> Training
    CO --> Training
    BT --> Training
    CT --> Training
    IT --> Training

    API --> Inference
    RC --> Inference
    MR --> Inference

    MT --> Evaluation
    AU --> Evaluation

    %% Click Events
    click HP "https://github.com/shayanjm/vec2text/blob/master/vec2text/models/inversion.py"
    click CR "https://github.com/shayanjm/vec2text/blob/master/vec2text/models/corrector_encoder.py"
    click BT "https://github.com/shayanjm/vec2text/blob/master/vec2text/trainers/base.py"
    click CT "https://github.com/shayanjm/vec2text/blob/master/vec2text/trainers/corrector.py"
    click IT "https://github.com/shayanjm/vec2text/blob/master/vec2text/trainers/inversion.py"
    click DP "https://github.com/shayanjm/vec2text/blob/master/vec2text/data_helpers.py"
    click TK "https://github.com/shayanjm/vec2text/blob/master/vec2text/tokenize_data.py"
    click CO "https://github.com/shayanjm/vec2text/blob/master/vec2text/collator.py"
    click MC "https://github.com/shayanjm/vec2text/blob/master/vec2text/models/config.py"
    click MU "https://github.com/shayanjm/vec2text/blob/master/vec2text/models/model_utils.py"
    click MT "https://github.com/shayanjm/vec2text/blob/master/vec2text/metrics.py"
    click AU "https://github.com/shayanjm/vec2text/blob/master/vec2text/analyze_utils.py"
    click API "https://github.com/shayanjm/vec2text/blob/master/vec2text/api.py"
    click RC "https://github.com/shayanjm/vec2text/blob/master/vec2text/run_args.py"
    click MR "https://github.com/shayanjm/vec2text/blob/master/vec2text/run.py"
    click FI "https://github.com/shayanjm/vec2text/blob/master/vec2text/trainers_baseline/fewshot_inversion_trainer.py"
    click DI "https://github.com/shayanjm/vec2text/blob/master/vec2text/trainers_baseline/decode_inversion_trainer.py"

    %% Styling
    classDef model fill:#3498db,stroke:#2980b9,color:white
    classDef trainer fill:#2ecc71,stroke:#27ae60,color:white
    classDef data fill:#e74c3c,stroke:#c0392b,color:white
    classDef config fill:#9b59b6,stroke:#8e44ad,color:white
    classDef util fill:#f1c40f,stroke:#f39c12,color:black
    classDef api fill:#1abc9c,stroke:#16a085,color:white
    classDef runner fill:#e67e22,stroke:#d35400,color:white
    classDef eval fill:#34495e,stroke:#2c3e50,color:white
    classDef baseline fill:#95a5a6,stroke:#7f8c8d,color:white
```
