# maxkrnl/prompt-tester architecture

> Architecture diagram of the GitHub repository maxkrnl/prompt-tester, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/maxkrnl/prompt-tester
- Repository: https://github.com/maxkrnl/prompt-tester

## Overview

##### Project Type & Purpose
- This is a **Python CLI demo** illustrating a two-phase RAG pipeline:
a. **Index Building** (one-time precompute of FAISS index)
b. **Interactive Generation** (load index + quantized Llama model to produce tweets or replies)
- No web frontend or database server—pure offline, file-based storage and local model inference.

##### File Structure
Top-level scripts and directories:
- `company_data/` – raw `.txt` knowledge files
- `build_rag_index.py` – ingestion, chunking, embedding, FAISS index creation
- `rag_utils_demo.py` – loading FAISS index + retrieval logic
- `llm_utils_demo.py` – instantiating 8-bit quantized Llama, prompt building, output cleaning
- `demo.py` – CLI interface orchestrating RAG retrieval + model inference
- Artifacts: `faiss_index.bin`, `doc_texts.pkl` (index + chunk store)

##### README Insights
- **Precompute**: run `build_rag_index.py` once after updating `company_data/`.
- **Dependencies**: Python 3.8+, FAISS, Huggingface Transformers, quantization libs.
- **Runtime Flow**:
1. CLI menu → choice: new tweet or reply
2. Query passes into `rag_utils_demo` → retrieve top-k chunks
3. Retrieved context + user prompt passed to `llm_utils_demo` → text generation

##### Defining the Main Components
a. Ingestion & Indexing Module
- Input: raw `.txt` files in `company_data/`
- Processes: chunking → embedding (e.g., via Huggingface embeddings) → FAISS index creation
- Output: `faiss_index.bin`, `doc_texts.pkl`
b. Retrieval Module
- Input: user query, FAISS index on disk
- Process: similarity search → retrieve top-k document chunks
- Output: context snippets
c. Model Inference Module
- Input: context snippets + formatted prompt
- Process: load quantized Llama (8-bit) via `transformers` → generate text
- Output: raw Llama output → cleaned reply/tweet text
d. CLI Orchestrator
- Presents menu, reads user input, calls Retrieval → Model Inference → displays output
e. External Dependencies
- Huggingface Hub (model download & login)
- Local filesystem for index and artifacts

##### Relationships & Data Flows
- From `company_data/` → Indexing Module → `faiss_index.bin` + `doc_texts.pkl`
- CLI Orchestrator → Retrieval Module → Model Inference Module → CLI Output
- External: Model weights fetched from Huggingface Hub before first inference
- Data store boundaries: separate build-time vs. runtime artifacts

##### Architectural Patterns & Principles
- Modular separation: indexing, retrieval, inference, and orchestration each in its own script/module
- File-based persistence: FAISS index + pickled chunks ensure fast startup
- Single responsibility: each module does one thing well

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph "Build Phase"
        raw["company_data/"]:::datastore
        build["build_rag_index.py"]:::module
        faiss["faiss_index.bin"]:::datastore
        pickle["doc_texts.pkl"]:::datastore
    end

    subgraph "Runtime Phase"
        cli["demo.py"]:::module
        retrieval["rag_utils_demo.py"]:::module
        inference["llm_utils_demo.py"]:::module
        hf["Huggingface Hub"]:::external
    end

    raw -->|"chunk & embed"| build
    build --> faiss
    build --> pickle

    faiss -->|"load index & chunks"| retrieval
    pickle --> retrieval
    cli -->|"invoke retrieval"| retrieval
    retrieval -->|"top-k chunks"| inference
    hf -->|"model download"| inference
    inference -->|"cleaned reply/tweet"| cli

    click raw "https://github.com/maxkrnl/prompt-tester/tree/main/company_data/"
    click build "https://github.com/maxkrnl/prompt-tester/blob/main/build_rag_index.py"
    click faiss "https://github.com/maxkrnl/prompt-tester/blob/main/faiss_index.bin"
    click pickle "https://github.com/maxkrnl/prompt-tester/blob/main/doc_texts.pkl"
    click retrieval "https://github.com/maxkrnl/prompt-tester/blob/main/rag_utils_demo.py"
    click inference "https://github.com/maxkrnl/prompt-tester/blob/main/llm_utils_demo.py"
    click cli "https://github.com/maxkrnl/prompt-tester/blob/main/demo.py"

    classDef module fill:#ADD8E6,stroke:#333,stroke-width:1px;
    classDef datastore fill:#90EE90,stroke:#333,stroke-width:1px,shape:cylinder;
    classDef external fill:#FFA500,stroke:#333,stroke-width:1px,shape:cloud
```
