# d2i-ai/eigenscore architecture

> Architecture diagram of the GitHub repository d2i-ai/eigenscore, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/d2i-ai/eigenscore
- Repository: https://github.com/d2i-ai/eigenscore

## Overview

##### Project type and purpose
- This is a research‐code repository implementing the INSIDE hallucination‐detection method for large language models (LLMs).
- It provides data processing, model‐interface layers, evaluation metrics, and plotting utilities to run experiments on question‐answer datasets (SQuAD, CoQA, NaturalQuestions, etc.).

##### File structure
- Top‐level scripts and configs:
- run.sh – orchestrates end‐to‐end execution.
- _settings.py – global configuration (paths, hyperparameters).
- Data storage:
- data/datasets/*.json, fig1.pdf – raw datasets and reference figures.
- Data loading & evaluation:
- dataeval/ – modules for loading each dataset, abstraction for evaluation tasks.
- Core functions:
- func/ – evaluation functions (evalFunc.py), metric computation (metric.py), plotting (plot.py).
- Modeling layer:
- models/ – model‐loading logic (_load_model.py), interfaces to NLI and OpenAI models (nli.py, openai_models.py).
- Pipeline orchestration:
- pipeline/generate.py – main script to generate LLM outputs and internal states for eigenscore.
- Utilities:
- utils/parallel.py – helper for multiprocessing and parallel data processing.

##### README for additional insights
- The README confirms it is a code release for a paper at ICLR 2024 on hallucination detection.

##### Elements
- Clear labels: “Dataset Storage,” “Data Loader,” “Model Interface,” “Inference Engine,” “Evaluation,” “Plotting,” “Utilities,” “Orchestrator.”
- Datasets → Data Loader
- Data Loader → Inference Engine
- Inference Engine → Evaluation
- Evaluation → Plotting
- Orchestrator → all modules
- Rectangles (green) for processing modules
- Parallelograms (orange) for scripts/orchestrator

##### Detailed guidance
- Run.sh/_settings.py at the top or side to show centralized orchestration.
- Modules from left to right or top to bottom in data‐flow order.

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph "Orchestrator"
        Run[/run.sh/]:::orchestrator
        Settings[/_settings.py/]:::orchestrator
    end

    DataStorage[(Datasets)]:::storage
    DataLoader["Data Loader & Preprocessor"]:::module
    ModelInterface["Model Interface Layer"]:::module
    Inference["Inference & Feature Extraction"]:::module
    subgraph "Evaluation Module"
        EvalFunc["evalFunc.py"]:::module
        Metric["metric.py"]:::module
    end
    Plot["Visualization Module"]:::module
    Utils[/Utilities/parallel.py/]:::utility

    Run -->|execute pipeline| Inference
    Settings -->|configuration| Inference

    DataStorage -->|load JSON| DataLoader
    DataLoader -->|parsed examples| Inference
    Inference -->|return logits & states| EvalFunc
    EvalFunc -->|compute metrics| Metric
    Metric -->|results JSON| Plot

    Utils -.->|parallelize loading| DataLoader
    Utils -.->|parallelize inference| Inference

    click DataStorage "https://github.com/d2i-ai/eigenscore/tree/main/data/datasets/"
    click DataLoader "https://github.com/d2i-ai/eigenscore/tree/main/dataeval/"
    click ModelInterface "https://github.com/d2i-ai/eigenscore/tree/main/models/"
    click Inference "https://github.com/d2i-ai/eigenscore/blob/main/pipeline/generate.py"
    click EvalFunc "https://github.com/d2i-ai/eigenscore/blob/main/func/evalFunc.py"
    click Metric "https://github.com/d2i-ai/eigenscore/blob/main/func/metric.py"
    click Plot "https://github.com/d2i-ai/eigenscore/blob/main/func/plot.py"
    click Utils "https://github.com/d2i-ai/eigenscore/blob/main/utils/parallel.py"
    click Run "https://github.com/d2i-ai/eigenscore/blob/main/run.sh"
    click Settings "https://github.com/d2i-ai/eigenscore/blob/main/_settings.py"

    classDef storage fill:#88ccff,stroke:#000,color:#000;
    classDef module fill:#ccffcc,stroke:#000,color:#000;
    classDef orchestrator fill:#ffdb99,stroke:#000,color:#000;
    classDef utility stroke-dasharray: 5 5,fill:#f9f9f9,stroke:#888,color:#000;
```
