# codelion/adaptive-classifier architecture

> Architecture diagram of the GitHub repository codelion/adaptive-classifier, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/codelion/adaptive-classifier
- Repository: https://github.com/codelion/adaptive-classifier

## Overview

##### Project type and purpose
- This is a Python library (adaptive-classifier) designed to provide a flexible, continuously-learning text classification system built on top of HuggingFace transformers.
- It exposes a simple API (AdaptiveClassifier) that lets users add examples, make predictions, persist state, push to/pull from the HF Hub, and supports specialized modes (hallucination detection, LLM routing, config optimization).

##### File structure
- Top-level:
- src/adaptive_classifier: core library code
– classifier.py: main API surface
– models.py: model‐loading & embedding logic
– memory.py: prototype memory management
– ewc.py: elastic weight consolidation (avoiding catastrophic forgetting)
- scripts/: standalone tools & evaluations (router, hallucination detector, etc.)
- examples/: user-facing example scripts
- docs/: API.md and supplementary docs
- tests/: unit tests for classifier and memory
- config files: pyproject.toml, setup.py, requirements.txt, GitHub Actions workflow

##### README for additional insights
- The library fuses three core components:
a) Transformer Embeddings (HuggingFace)
b) Prototype Memory (faiss or in-memory prototypes)
c) Adaptive Neural Layer (continual learning / EWC)
- There are multiple “personalities” of the classifier (hallucination detector, router, config optimizer) driven by the same underlying architecture.

a. Main Components
1) Client / User Code (examples, API consumers)
2) AdaptiveClassifier API (classifier.py)
3) Model Loader & Embedding Module (models.py + transformers)
4) Prototype Memory Store (memory.py + faiss-cpu/gpu)
5) Adaptive Neural Layer (in-process training, ewc.py)
6) Persistence Layer (save/load via safetensors, HF Hub integration)
7) Scripts & Evaluation Tools (scripts/…)

b. Relationships & Interactions
- User code calls AdaptiveClassifier methods (add_examples, predict, save, load).
- AdaptiveClassifier orchestrates:
- retrieving embeddings from the transformer model
- querying/inserting prototypes in memory
- running adaptive training steps via the neural layer (EWC to prevent forgetting)
- serializing/deserializing state to disk or HF Hub
- Scripts import the same AdaptiveClassifier and run specialized evaluation pipelines (router eval, hallucination detector eval).
- CI pipeline (GitHub Actions) runs tests and publishes package.

c. Architectural Patterns & Principles
- Modular separation: embedding, memory, adaptation, persistence
- Dependency inversion: core library depends on abstracted model & memory back-ends
- Continuous learning loop: data flows from add_examples → memory update → neural adaptation → updated classifier
- Extensibility: same core pipeline powers multiple “flavors” (classifier, router, detector)

d. Key Technologies, Frameworks & Libraries
- Python ≥3.8
- PyTorch ≥2.0 & HuggingFace transformers
- safetensors (fast, secure weight storage)
- faiss-cpu/gpu for efficient prototype retrieval
- GitHub Actions for CI/CD
- Optional HF Hub for model distribution

##### Final Advice
- You capture both inference (predict) and training (add_examples + EWC) flows.
- Don’t forget the persistence layer (disk + HF Hub) and the role of GitHub Actions for testing/deployment.
- Keep it extensible: illustrate how new “personalities” (hallucination detector, router) reuse the same core pipeline.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Entry Points
    examples["Client: Basic & Advanced Usage"]:::entry
    click examples "https://github.com/codelion/adaptive-classifier/blob/main/examples/basic_usage.py"
    click examples "https://github.com/codelion/adaptive-classifier/blob/main/examples/advanced_usage.py"

    subgraph "Scripts & Evaluation Tools"
        direction TB
        eval_router_arena["Eval LLM Router Arena"]:::entry
        click eval_router_arena "https://github.com/codelion/adaptive-classifier/blob/main/scripts/eval_llmrouter_arena.py"
        eval_router_classifier["Eval LLM Router Classifier"]:::entry
        click eval_router_classifier "https://github.com/codelion/adaptive-classifier/blob/main/scripts/eval_llmrouter_classifier.py"
        config_optimizer["Gen LLM Config Optimizer"]:::entry
        click config_optimizer "https://github.com/codelion/adaptive-classifier/blob/main/scripts/gen_llm_config_optimizer.py"
        halluc_detector["Gen LLM Hallucination Detector"]:::entry
        click halluc_detector "https://github.com/codelion/adaptive-classifier/blob/main/scripts/gen_llm_hallucination_detector.py"
        ar_config["Router Config (JSON)"]:::entry
        click ar_config "https://github.com/codelion/adaptive-classifier/blob/main/scripts/adaptive_router/config.json"
        ar_tensors["Router Tensors (safetensors)"]:::entry
        click ar_tensors "https://github.com/codelion/adaptive-classifier/blob/main/scripts/adaptive_router/tensors.safetensors"
    end

    docs["Documentation: API Reference"]:::entry
    click docs "https://github.com/codelion/adaptive-classifier/blob/main/docs/API.md"

    subgraph "Unit Tests"
        direction TB
        test_classifier["Test Classifier"]:::entry
        click test_classifier "https://github.com/codelion/adaptive-classifier/blob/main/tests/test_classifier.py"
        test_memory["Test Memory"]:::entry
        click test_memory "https://github.com/codelion/adaptive-classifier/blob/main/tests/test_memory.py"
    end

    CI["GitHub Actions CI/CD"]:::ci
    click CI "https://github.com/codelion/adaptive-classifier/blob/main/.github/workflows/publish.yml"

    %% Core Library Modules
    subgraph "Core Library" 
        direction TB
        classifier["AdaptiveClassifier API\n(classifier.py)"]:::core
        click classifier "https://github.com/codelion/adaptive-classifier/blob/main/src/adaptive_classifier/classifier.py"
        models["Model Loader & Embedding\n(models.py)"]:::core
        click models "https://github.com/codelion/adaptive-classifier/blob/main/src/adaptive_classifier/models.py"
        memory["Prototype Memory Store\n(memory.py)"]:::core
        click memory "https://github.com/codelion/adaptive-classifier/blob/main/src/adaptive_classifier/memory.py"
        ewc["Adaptive Neural Layer\n(EWC & Training)\n(ewc.py)"]:::core
        click ewc "https://github.com/codelion/adaptive-classifier/blob/main/src/adaptive_classifier/ewc.py"
        persistence["Persistence Layer\n(save/load, HF Hub)"]:::core
        click persistence "https://github.com/codelion/adaptive-classifier/blob/main/src/adaptive_classifier/classifier.py"
    end

    %% External Systems
    transformers["HuggingFace Transformers"]:::external
    faiss["FAISS (CPU/GPU)"]:::external
    safetensors["safetensors"]:::external
    HFHub["HuggingFace Hub"]:::external

    %% Data Flows
    examples -->|"calls API methods"| classifier
    eval_router_arena -->|imports & runs| classifier
    eval_router_classifier -->|imports & runs| classifier
    config_optimizer -->|imports & runs| classifier
    halluc_detector -->|imports & runs| classifier

    classifier -->|"get_embeddings(text)"| models
    models -->|"load_model() & embed(text)"| transformers
    classifier -->|"query/insert prototypes"| memory
    memory -->|"index & search"| faiss
    classifier -->|"train_step(data)"| ewc
    ewc -->|"update weights"| classifier

    classifier -->|"save()/load()"| persistence
    persistence -->|"serialize weights"| safetensors
    persistence -->|"push/pull model"| HFHub

    %% Supporting Processes
    CI -->|runs tests & publishes| classifier
    CI -->|runs tests| test_classifier
    CI -->|runs tests| test_memory

    docs -->|references API| classifier

    %% Styles
    classDef core fill:#B0E0E6,stroke:#333,stroke-width:1px
    classDef external fill:#98FB98,stroke:#333,stroke-width:1px
    classDef entry fill:#FFD580,stroke:#333,stroke-width:1px
    classDef ci fill:#D3D3D3,stroke:#333,stroke-width:1px
```
