# kmaurinjones/allmeans architecture

> Architecture diagram of the GitHub repository kmaurinjones/allmeans, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-11-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kmaurinjones/allmeans
- Repository: https://github.com/kmaurinjones/allmeans

## Overview

##### Project Type and Purpose
- AllMeans is an **open-source Python library** and CLI tool for **automatic topic modeling**.
- It exposes both a **Scikit-learn–style API** (`TopicModel`) and a **CLI interface** (`allmeans` command).

##### File Structure
- Top-level package: `AllMeans/`
- `cli.py` – CLI entrypoint
- `api.py` – Python API façade
- `core/` – Orchestrator and core components (preprocessing, pipeline, types)
- `algorithms/` – Clustering implementations (K-Means, NMF, LDA, HDBSCAN+UMAP)
- `evaluation/` – Metric calculators (coherence, diversity, etc.)
- `tests/`, `examples/`, CI config, packaging (`pyproject.toml`)

##### README Workflow
The published pipeline is:
1. **Preprocessing** (sentence splitting, lemmatization, POS filtering)
2. **Feature Extraction** (TF-IDF, BoW, SIF embeddings)
3. **Clustering** (various algorithms, optional auto-K selection)
4. **Keyword Extraction & Labeling**
5. **Evaluation** (Coherence, Silhouette, Diversity, etc.)
6. **Output** (TopicModelResults → API return or JSON via CLI)

##### Main Components
a. **Entrypoints**
- **CLI Layer** (console script → `cli.py`)
- **API Layer** (`api.py`, exposes `TopicModel` class)
b. **Core Orchestrator**
- **Pipeline** (`core/pipeline.py`) calls each stage
- **Types & Config** (`core/types.py`) carry parameters
c. **Processing Modules**
- **Preprocessing** (`core/features.py`, `core/labeling.py`)
- **Feature Extractors** (TF-IDF, BoW in core; SIF in optional embeddings)
- **Clustering Algorithms** (`algorithms/clustering.py`)
- **Auto-K Selector** (logic embedded in pipeline)
d. **Evaluation Modules**
- **Metrics** (`evaluation/metrics.py`)
e. **Data Models & Results**
- **TopicModelResults**, **Topic**, assignments, scores
f. **External Dependencies**

##### Relationships & Data Flows
- **Text Input** → **Preprocessing** → **Feature Matrix** → **Clustering** → **Cluster Assignments**
- **Cluster Assignments** → **Keyword Extraction & Labeling** → **Topic Labels & Keywords**
- **Cluster Assignments** + **Features** → **Evaluation Metrics**
- **Final Results** → returned by API or serialized by CLI

##### Architectural Patterns & Design Principles
- **Modular Plug-ins**: feature methods and clustering algorithms implement a common interface
- **Pipeline Pattern**: sequential stages, pluggable via configuration
- **Separation of Concerns**: clear layers for preprocessing, modeling, evaluation, and I/O

## Diagram (Mermaid)

```mermaid
graph TD
    %% Entrypoint Layer
    subgraph "Entrypoints"
        CLI["CLI Layer\n(cli.py)"]:::entry
        API["API Layer\n(api.py)"]:::entry
    end

    %% Core Orchestrator
    subgraph "Core Orchestrator"
        PIPELINE["Pipeline\n(core/pipeline.py)"]:::orchestrator
        TYPES["Config & Types\n(core/types.py)"]:::orchestrator
    end

    %% Processing Modules
    subgraph "Processing Modules"
        FEAT["Preprocessing & Feature Extraction\n(core/features.py)"]:::preprocess
        LABEL["Keyword Extraction & Labeling\n(core/labeling.py)"]:::preprocess
        CLUST["Clustering Algorithms\n(algorithms/clustering.py)"]:::algorithm
    end

    %% Evaluation
    subgraph "Evaluation"
        METRICS["Evaluation Metrics\n(evaluation/metrics.py)"]:::evaluation
    end

    %% Examples & CI/Packaging
    subgraph "Examples & CI/Packaging"
        BASIC["Basic Usage Example\n(examples/basic_usage.py)"]:::external
        WIKI["Wikipedia Example\n(examples/wikipedia_example.py)"]:::external
        CI["CI Pipeline\n(.github/workflows/ci.yml)"]:::external
        REL["Release Pipeline\n(.github/workflows/release.yml)"]:::external
        PKG["Packaging Config\n(pyproject.toml)"]:::external
        TEST["Tests\n(tests/test_api.py)"]:::external
    end

    %% External Dependencies
    subgraph "External Libraries"
        SKL["scikit-learn"]:::external
        NP["numpy"]:::external
        SP["scipy"]:::external
        SPACY["spacy"]:::external
        HDB["hdbscan"]:::external
        UMAP["umap-learn"]:::external
        ST["sentence-transformers"]:::external
        GENSIM["gensim"]:::external
    end

    %% Data Flow
    TextIn["Text Input"]:::external
    Results["TopicModelResults"]:::external

    %% Connections
    TextIn -->|"raw text"| FEAT
    FEAT -->|"feature matrix"| CLUST
    CLUST -->|"cluster assignments"| LABEL
    LABEL -->|"labeled topics"| METRICS
    CLUST -->|"assignments & features"| METRICS
    METRICS -->|"evaluation scores"| Results

    %% Orchestration
    CLI -->|"invoke"| PIPELINE
    API -->|"call TopicModel"| PIPELINE
    TYPES -->|"config"| PIPELINE
    PIPELINE --> FEAT
    PIPELINE --> CLUST
    PIPELINE --> LABEL
    PIPELINE --> METRICS
    PIPELINE --> Results

    %% Examples & CI/Packaging Flows
    BASIC -->|"demo CLI/API"| PIPELINE
    WIKI -->|"demo CLI/API"| PIPELINE
    PKG -->|"version/config"| PIPELINE
    CI -->|"tests & build"| TEST
    REL -->|"release"| PKG

    %% External Dependencies Links
    FEAT --> SPACY
    CLUST --> SKL
    CLUST --> HDB
    CLUST --> UMAP
    FEAT --> ST
    METRICS --> GENSIM
    FEAT --> NP
    CLUST --> NP
    METRICS --> NP
    PIPELINE --> NP
    FEAT --> SP
    CLUST --> SP

    %% Click Events
    click CLI "https://github.com/kmaurinjones/allmeans/blob/main/AllMeans/cli.py"
    click API "https://github.com/kmaurinjones/allmeans/blob/main/AllMeans/api.py"
    click PIPELINE "https://github.com/kmaurinjones/allmeans/blob/main/AllMeans/core/pipeline.py"
    click TYPES "https://github.com/kmaurinjones/allmeans/blob/main/AllMeans/core/types.py"
    click FEAT "https://github.com/kmaurinjones/allmeans/blob/main/AllMeans/core/features.py"
    click LABEL "https://github.com/kmaurinjones/allmeans/blob/main/AllMeans/core/labeling.py"
    click CLUST "https://github.com/kmaurinjones/allmeans/blob/main/AllMeans/algorithms/clustering.py"
    click METRICS "https://github.com/kmaurinjones/allmeans/blob/main/AllMeans/evaluation/metrics.py"
    click BASIC "https://github.com/kmaurinjones/allmeans/blob/main/examples/basic_usage.py"
    click WIKI "https://github.com/kmaurinjones/allmeans/blob/main/examples/wikipedia_example.py"
    click CI "https://github.com/kmaurinjones/allmeans/blob/main/.github/workflows/ci.yml"
    click REL "https://github.com/kmaurinjones/allmeans/blob/main/.github/workflows/release.yml"
    click PKG "https://github.com/kmaurinjones/allmeans/blob/main/pyproject.toml"
    click TEST "https://github.com/kmaurinjones/allmeans/blob/main/tests/test_api.py"

    %% Styles
    classDef entry fill:#d5f5e3,stroke:#27ae60;
    classDef orchestrator fill:#85c1e9,stroke:#21618c;
    classDef preprocess fill:#aed6f1,stroke:#2874a6;
    classDef algorithm fill:#f9e79f,stroke:#b7950b;
    classDef evaluation fill:#d7bde2,stroke:#6c3483;
    classDef external fill:#d5dbdb,stroke:#566573;
```
