# jordicarreraventura/clustexts architecture

> Architecture diagram of the GitHub repository jordicarreraventura/clustexts, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jordicarreraventura/clustexts
- Repository: https://github.com/jordicarreraventura/clustexts

## Overview

##### Project type and purpose
- This is a single‐module Python library (“Clustexts”) that performs automated k‑means clustering on text corpora.
- It exposes a simple class interface to vectorize text, optionally apply dimensionality reduction, run k‑means across a range of k values (elbow method), and return cluster labels.

##### File structure
- Top level: packaging files (.gitignore, LICENSE, README.md, pyproject.toml, requirements.txt).
- src/clustexts/clustexts.py: core implementation
- tests/: unit and integration tests, including a sample artifact for intents.csv.

##### README for additional insights
- Confirms dependencies on scikit‑learn, numpy, pandas, scipy, matplotlib/seaborn, tqdm.
- Describes the two main sub‑tasks: vectorization (TF‑IDF + optional SVD) and clustering (elbow‑method k‑means), plus optional reporting (plots, examples, verbosity).

a. Client / Application Layer
- The user code (e.g. a Jupyter notebook or downstream application) that instantiates Clustexts and feeds it an iterable of raw texts.
b. Clustexts Library Core
- Vectorization Module – wraps scikit‑learn’s TfidfVectorizer
- Optional Reducer Module – wraps scikit‑learn’s TruncatedSVD
- Clustering Module – runs scikit‑learn’s KMeans repeatedly over k∈[k_min, k_max], computes inertia, applies elbow criterion
- Reporting Module (optional) – generates plots via matplotlib/seaborn and example outputs
c. External Dependencies
- scikit-learn, numpy, pandas, scipy, matplotlib, seaborn, tqdm
d. Output
- Iterable of integer cluster labels (aligned with input sequence)
- Optional visual artifacts (plots)

##### Relationships and data flow
- Raw Texts → [Vectorization Module] → TF‑IDF feature matrix
- TF‑IDF Matrix → [Optional SVD Reducer] → Lower‑dim Dense Matrix (if configured)
- Resulting Matrix → [Clustering Module]
- Internally: loop over k, compute inertia, track best elbow point
- Final cluster assignments → returned to Client
- If reporting flags are set, side‑feeds to [Reporting Module] → plots to screen or file

##### Architectural patterns & design principles
- Single responsibility per submodule (vectorize, reduce, cluster, report)
- Pipeline pattern (data flows through ordered stages)
- Dependency inversion: wraps scikit‑learn classes behind a stable interface
- Configurable via constructor parameters for extensibility

##### Technologies, frameworks, libraries
- Python 3.x, pyproject.toml (PEP 517 packaging)
- scikit‑learn (TfidfVectorizer, TruncatedSVD, KMeans)
- numpy / pandas for data manipulation
- scipy for numeric routines
- matplotlib / seaborn for plotting
- tqdm for progress reporting

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Main Pipeline
    Client["Client / Application Layer"]:::external
    Vectorizer["Vectorization Module"]:::module
    Reducer["Reducer Module (Optional SVD)"]:::module
    Clusterer["Clustering Module (Elbow Method Loop)"]:::module
    Reporter["Reporting Module (Plots & Verbosity)"]:::module
    Output["Output\n(cluster labels + plots)"]:::output

    Client -->|"Iterable[str]"| Vectorizer
    Vectorizer -->|"ndarray (TF‑IDF matrix)"| Reducer
    Reducer -->|"ndarray (reduced matrix)"| Clusterer
    Clusterer -->|"cluster labels"| Output
    Reporter -->|"plots / examples"| Output
    Clusterer --> Reporter

    Output -->|"Iterable[int] + visuals"| Client

    %% External Dependencies
    subgraph "External Dependencies"
        sklearn["scikit‑learn"]:::external
        numpy["numpy"]:::external
        pandas["pandas"]:::external
        scipy["scipy"]:::external
        matplotlib["matplotlib"]:::external
        seaborn["seaborn"]:::external
        tqdm["tqdm"]:::external
    end

    sklearn --> Vectorizer
    sklearn --> Reducer
    sklearn --> Clusterer
    matplotlib --> Reporter
    seaborn --> Reporter
    tqdm --> Reporter
    numpy --> Vectorizer
    pandas --> Vectorizer
    scipy --> Clusterer

    %% Repository Files
    subgraph "Repository Files"
        Init["src/clustexts/__init__.py"]:::file
        Core["src/clustexts/clustexts.py"]:::file
        Pyproj["pyproject.toml"]:::file
        Req["requirements.txt"]:::file
        Readme["README.md"]:::file
        TestsDir["tests/"]:::dir
        IntTest["tests/integration.py"]:::file
        Artifact["tests/artifacts/intents.csv"]:::file
        License["LICENSE"]:::file
        Gitignore[".gitignore"]:::file
    end

    %% Click Events
    click Init "https://github.com/jordicarreraventura/clustexts/blob/main/src/clustexts/__init__.py"
    click Core "https://github.com/jordicarreraventura/clustexts/blob/main/src/clustexts/clustexts.py"
    click Pyproj "https://github.com/jordicarreraventura/clustexts/blob/main/pyproject.toml"
    click Req "https://github.com/jordicarreraventura/clustexts/blob/main/requirements.txt"
    click Readme "https://github.com/jordicarreraventura/clustexts/blob/main/README.md"
    click TestsDir "https://github.com/jordicarreraventura/clustexts/tree/main/tests/"
    click IntTest "https://github.com/jordicarreraventura/clustexts/blob/main/tests/integration.py"
    click Artifact "https://github.com/jordicarreraventura/clustexts/blob/main/tests/artifacts/intents.csv"
    click License "https://github.com/jordicarreraventura/clustexts/tree/main/LICENSE"
    click Gitignore "https://github.com/jordicarreraventura/clustexts/blob/main/.gitignore"

    %% Styles
    classDef module fill:#D0E6FF,stroke:#69A1FF,color:#000000
    classDef external fill:#E6FFE6,stroke:#5CB85C,color:#000000
    classDef output fill:#FFF5CC,stroke:#FFD700,color:#000000
    classDef file fill:#F0F0F0,stroke:#AAAAAA,color:#333333
    classDef dir fill:#F9F9F9,stroke:#888888,color:#333333
```
