# dt-service-consulting/unbiasai architecture

> Architecture diagram of the GitHub repository dt-service-consulting/unbiasai, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dt-service-consulting/unbiasai
- Repository: https://github.com/dt-service-consulting/unbiasai

## Overview

##### Project Type and Purpose
- This is an open-source Python package (“UnBiasAI”) designed to analyze, detect, and mitigate bias in AI-generated text.
- It provides a controlled Retrieval-Augmented Generation (RAG) pipeline, utilities for embedding and language detection, modules for different bias detectors, and integration with multiple LLM APIs (OpenAI GPT-4, Claude, Cohere, Mistral, DeepSeek).

##### File Structure Analysis
- Top-level directories and files:
- data/ – contains static datasets (e.g., retrieval_recency.csv)
- notebooks/ – Jupyter notebooks demonstrating usage and analysis
- src/
– queries/ – scripts to run specific data retrieval experiments
– unbiasai/
· biases/ – modules implementing bias detection logic
· config.py – environment and API key configuration
· utils.py – helper functions (embedding, LLM initialization, language detection)
- tests/ – unit tests for core utilities
- pyproject.toml & requirements.txt – dependency definitions
- .env or ENVFILE – environment variables (API keys, Supabase credentials)

##### Key Components and Their Relationships
a. Data Layer
– Static CSV datasets (data/)
– Supabase database for document retrieval (via create_client in utils/config)
b. Core Pipeline
– Retrieval module (src/queries and retrieval_bias code) fetches context from Supabase or CSV
– Embedding utility (get_embedding in utils.py) calls OpenAI embedding endpoint
– Language detection utility (detect_language in utils.py) identifies text language
– Bias detectors (modules in src/unbiasai/biases):
· retrieval_recency.py
· retrieval_language.py
· other bias types can be added here
– RAG orchestrator (can be in user code or notebooks) that:
1. Retrieves context
2. Generates or augments prompts via an LLM
3. Passes output to bias detection modules
c. External Services
– OpenAI API (embeddings, completions)
– Other LLM APIs (Claude, Cohere, Mistral, DeepSeek) via initialize_llm
– Supabase (vector DB or document store)
d. User Interface / Entry Points
– Python functions and classes in src/unbiasai/utils.py and src/unbiasai/biases
– Example scripts in src/queries/
– Jupyter notebooks demonstrating end-to-end analysis

##### Architectural Patterns and Design Principles
- Modular design: separate packages for utilities, biases, and query scripts
- Dependency inversion: utils.py abstracts LLM initialization and embeddings so bias modules are decoupled from specific LLM APIs
- Extensibility: new bias detectors can be added under src/unbiasai/biases; new retrieval strategies under src/queries
- Testability: core utilities have unit tests; business logic in pure Python modules

##### Elements
- Clear labels for each component (e.g., “Retrieval Module”, “Embedding Service”, “OpenAI API”, “Bias Detector: Retrieval Recency”)

– “Retrieval Module” → “Supabase (External DB)” and/or “CSV Data”
– “Embedding Service” → “OpenAI Embeddings API”
7. Surround everything with a boundary labeled “UnBiasAI Package (src/unbiasai).”

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% User Interface
    User["User / Notebook"]:::client

    %% Configuration
    Config["Configuration Loader\n(.env / ENVFILE)"]:::internal
    click Config "https://github.com/dt-service-consulting/unbiasai/blob/main/src/unbiasai/config.py"

    %% Pipeline Orchestrator
    Orchestrator["Pipeline Orchestrator"]:::internal

    %% Data Layer
    subgraph "Data Layer"
        style DataLayer fill:#e8f5e9,stroke:#2e7d32
        CSV["Static CSV Dataset"]:::data
        click CSV "https://github.com/dt-service-consulting/unbiasai/blob/main/data/retrieval_recency.csv"
        DataPkg["data/__init__.py"]:::data
        click DataPkg "https://github.com/dt-service-consulting/unbiasai/blob/main/data/__init__.py"
        Supabase["Supabase (Vector DB)"]:::external
    end

    %% Query Scripts
    subgraph "Query Scripts (src/queries)" 
        style Queries fill:#e3f2fd,stroke:#1976d2
        RetrievalQuery["Retrieval Module:\nRecency Query"]:::internal
        click RetrievalQuery "https://github.com/dt-service-consulting/unbiasai/blob/main/src/queries/retrieval_recency.py"
    end

    %% UnBiasAI Package
    subgraph "UnBiasAI Package\n(src/unbiasai)" 
        style Package fill:#e3f2fd,stroke:#1565c0
        Utils["Utilities (get_embedding,\n detect_language,\n initialize_llm)"]:::internal
        click Utils "https://github.com/dt-service-consulting/unbiasai/blob/main/src/unbiasai/utils.py"
        subgraph "Bias Detection Modules"
            style Biases fill:#fff3e0,stroke:#ef6c00
            BiasRecency["Bias Detector:\nRetrieval Recency"]:::internal
            click BiasRecency "https://github.com/dt-service-consulting/unbiasai/blob/main/src/unbiasai/biases/retrieval_recency.py"
            BiasLanguage["Bias Detector:\nRetrieval Language"]:::internal
            click BiasLanguage "https://github.com/dt-service-consulting/unbiasai/blob/main/src/unbiasai/biases/retrieval_language.py"
            AddBias["+ Add new bias module"]:::extension
        end
    end

    %% External Services
    subgraph "External LLM APIs"
        style ExternalLLM fill:#ffebee,stroke:#c62828
        OpenAI["OpenAI API"]:::external
        Claude["Claude API"]:::external
        Cohere["Cohere API"]:::external
        Mistral["Mistral API"]:::external
        DeepSeek["DeepSeek API"]:::external
    end

    %% Results
    Results["Results Aggregator"]:::internal

    %% Tests
    Tests["Unit Tests (test_utils.py)"]:::test
    click Tests "https://github.com/dt-service-consulting/unbiasai/blob/main/tests/test_utils.py"

    %% Flows
    User -->|triggers| Orchestrator
    Config -->|provides API keys| Utils
    Orchestrator -->|calls| RetrievalQuery
    RetrievalQuery -->|reads| CSV
    RetrievalQuery -->|queries| Supabase
    Orchestrator -->|requests embedding| Utils
    Utils -->|calls| OpenAI
    Orchestrator -->|initializes LLM| Utils
    Utils -->|calls| OpenAI
    Utils -->|calls| Claude
    Utils -->|calls| Cohere
    Utils -->|calls| Mistral
    Utils -->|calls| DeepSeek
    Orchestrator -->|passes text to| BiasRecency
    Orchestrator -->|passes text to| BiasLanguage
    BiasRecency -->|sends result| Results
    BiasLanguage -->|sends result| Results
    Results -->|returns| User

    %% Legend
    classDef client fill:#e8eaf6,stroke:#3949ab,color:#000
    classDef internal fill:#bbdefb,stroke:#0d47a1,color:#000
    classDef data fill:#c8e6c9,stroke:#2e7d32,color:#000
    classDef external fill:#ffcdd2,stroke:#b71c1c,color:#000
    classDef extension stroke-dasharray: 5 5,fill:none,stroke:#ef6c00,color:#000
    classDef test fill:#f3e5f5,stroke:#6a1b9a,color:#000
```
