# oneal2000/mind architecture

> Architecture diagram of the GitHub repository oneal2000/mind, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/oneal2000/mind
- Repository: https://github.com/oneal2000/mind

## Overview

##### Project Type and Purpose
- This is a research‐oriented machine-learning pipeline (“MIND”) for unsupervised hallucination detection in transformer‐based LLMs, coupled with the HELM benchmark for evaluation.
- The repository is organized around a data‐processing/training/evaluation workflow rather than a classic client–server or UI-driven application.

##### File Structure Analysis
- Top‐level scripts:
- generate_data.py  → auto-annotates Wikipedia sentences with hallucinations
- generate_hd.py    → extracts hidden-state features for model training
- detection_score.py → evaluates trained classifier on the HELM benchmark
- auto-labeled/
- train/            → dataset.py (data loader) + train.py (classifier training)
- wiki/             → raw Wikipedia splits (train/valid/test)
- helm/                 → HELM benchmark data: model outputs, prompts, hidden states, activations
- utils/                → shared code (gen.py, model.py) for invoking LLMs, feature routines
- requirements.txt      → Python libraries (torch, transformers, scikit-learn, spacy, etc.)

##### README Insights
- Four-step pipeline:
1. generate_data.py
2. generate_hd.py
3. auto-labeled/train/train.py
4. detection_score.py
- External dependencies: any Transformer-based LLM (via HuggingFace Transformers) plus Spacy for entity extraction.
- HELM benchmark data organized per model, with prompt→output→hidden states→activation values.

1. Wikipedia Data Loader (utils + auto-labeled/wiki)
2. Auto-Annotation Module (generate_data.py + Spacy + Transformers)
3. Feature Extraction Module (generate_hd.py + Transformers hooks + utils)
4. Classifier Training Module (auto-labeled/train/dataset.py & train.py + scikit-learn or PyTorch MLP)
5. HELM Benchmark Data Store (helm/data, helm/hd, helm/hd_act)
6. Evaluation Module (detection_score.py + output Excel reports)
7. Shared Utilities Layer (utils/gen.py, utils/model.py)

- Wikipedia Data Loader → Auto-Annotation Module
- Auto-Annotation Module → auto-labeled output JSON
- auto-labeled JSON → Feature Extraction Module → hidden-state JSON
- hidden-state JSON → Classifier Training Module → trained model artifact
- HELM Benchmark Data Store → Evaluation Module + trained model → performance reports
- Shared Utilities Layer ← all scripts (generate_data.py, generate_hd.py, train.py, detection_score.py)
- External LLMs ↔ Auto-Annotation Module & Feature Extraction Module

c. Architectural Patterns and Principles:
- Pipeline pattern: each step produces artifacts consumed by the next
- Modularization: clear separation between data annotation, feature extraction, training, evaluation
- Reusability: utils/ contains common code for invoking models and processing hidden states
- Configuration by script parameters and JSON outputs for reproducibility

d. Key Technologies & Frameworks:
- Python 3.9
- Spacy (entity extraction)
- HuggingFace Transformers (LLM inference + hidden states)
- PyTorch (ML model, MLP classifier)
- scikit-learn (metrics, train/test splits)
- pandas/numpy (data handling)
- Excel reports (result_*.xlsx)

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Data Sources"
        WikiLoader["Wikipedia Data Loader"]:::data
        HELMStore["HELM Benchmark Data Store"]:::data
    end

    subgraph "External Services"
        ExtLLM["External LLMs"]:::external
    end

    subgraph "Processing Pipeline"
        Annotation["Auto-Annotation Module (Step 1)"]:::process
        AutoJSON["Auto-labeled JSON"]:::data
        FeatureExtract["Feature Extraction Module (Step 2)\n(mean pooling)"]:::process
        HiddenJSON["Hidden-State JSON"]:::data
        ClassifierTrain["Classifier Training Module (Step 3)"]:::train
        ModelArtifact["Trained MLP Model Artifact"]:::data
        Evaluation["Evaluation Module (Step 4)"]:::train
    end

    Utils["Shared Utilities"]:::util

    %% Data Flow
    WikiLoader -->|"raw Wikipedia splits"| Annotation
    Annotation -->|"hallucination labels (JSON)"| AutoJSON
    AutoJSON --> FeatureExtract
    FeatureExtract -->|"hidden-state features (JSON)"| HiddenJSON
    HiddenJSON --> ClassifierTrain
    ClassifierTrain -->|"trained MLP model"| ModelArtifact
    ModelArtifact --> Evaluation
    HELMStore -->|"HELM data (prompts, outputs, hidden states)"| Evaluation

    %% External interactions
    ExtLLM <--> Annotation
    ExtLLM <--> FeatureExtract

    %% Shared utilities usage
    Annotation -.-> Utils
    FeatureExtract -.-> Utils
    ClassifierTrain -.-> Utils
    Evaluation -.-> Utils

    %% Click Events
    click WikiLoader "https://github.com/oneal2000/mind/tree/main/auto-labeled/wiki/"
    click Annotation "https://github.com/oneal2000/mind/blob/main/generate_data.py"
    click FeatureExtract "https://github.com/oneal2000/mind/blob/main/generate_hd.py"
    click FeatureExtract "https://github.com/oneal2000/mind/blob/main/generate_hd_for_helm.py"
    click ClassifierTrain "https://github.com/oneal2000/mind/tree/main/auto-labeled/train/"
    click HELMStore "https://github.com/oneal2000/mind/tree/main/helm/data/"
    click Evaluation "https://github.com/oneal2000/mind/blob/main/detection_score.py"
    click Utils "https://github.com/oneal2000/mind/tree/main/utils/"

    %% Styles
    classDef data fill:#cce5ff,stroke:#0366d6,color:#0366d6
    classDef process fill:#d4edda,stroke:#28a745,color:#155724
    classDef train fill:#fff3cd,stroke:#856404,color:#856404
    classDef util fill:#e2e3e5,stroke:#6c757d,color:#343a40
    classDef external fill:#e2e3e5,stroke:#6c757d,color:#343a40
```
