# kaynup/binary-text-classification-with-sklearn-jax architecture

> Architecture diagram of the GitHub repository kaynup/binary-text-classification-with-sklearn-jax, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kaynup/binary-text-classification-with-sklearn-jax
- Repository: https://github.com/kaynup/binary-text-classification-with-sklearn-jax

## Overview

##### Project type and purpose
- This is an open-source sentiment classification toolkit with two parallel modeling branches:
- A scikit-learn baseline (Logistic Regression + TF-IDF)
- A JAX/Flax/Optax neural prototype (embedding + pooling, with plans for LSTM)
- It provides both training pipelines (notebooks + scripts) and local inference endpoints via FastAPI.

##### File structure
- data/
– raw: original CSV from Kaggle
– processed: per-configuration TF-IDF matrices, vectorizer pickles, label splits
- notebooks/: exploratory analysis, preprocessing, model training & saving
- src/tokenizers.py + utils.py: reusable code for text tokenization/preprocessing
- models/
– sklearn: saved .joblib logistic regression
– jax: saved Flax pipelines (.joblib)
- local-deploy/: two FastAPI apps (sklearn-app.py, jax-app.py), static index.html, logs
- logs/: benchmark and training history logs
- requirements.txt, README.md, LICENSE

##### README for additional insights
- Confirms the dual-branch modeling strategy
- Describes dataset origin, performance benchmarks, known limitations
- Details how to run each FastAPI endpoint and sample request/response
- Notes logging behavior and common errors

a. Main components
- Data Ingestion & Preprocessing
- Feature Extraction & Tokenization
- Model Training Pipelines (sklearn vs JAX)
- Model Artifact Storage (models/sklearn, models/jax)
- Inference API Layer (FastAPI apps)
- Logging & Monitoring
- Experimentation Notebooks
- Utility Modules
- External Dataset Source (Kaggle)
b. Relationships & interactions
1. Raw data → preprocessing scripts/notebooks → processed data
2. Processed data + tokenizers → feature matrices (TF-IDF or embeddings)
3. Feature matrices → training pipelines → trained model artifacts
4. Inference request (JSON) → FastAPI server → vectorizer/tokenizer → model.predict → JSON response
5. Each inference writes to local-deploy/inference.log
6. Training logs written to logs/training_logs and logs/initial-benchmark-logs
c. Architectural patterns & design principles
- Separation of concerns: data, training, inference, logging each in its own module/directory
- Two “micro-services” for inference (one per modeling branch)
- Reusable utility and tokenizer libraries share code between training & inference
- Versioned datasets and models allow reproducibility and experimentation
d. Key technologies, frameworks & libraries
- Core language: Python
- Modeling: scikit-learn, JAX, Flax, Optax
- Serialization: joblib, pickle, NPZ
- API: FastAPI, Uvicorn
- Data: NumPy, SciPy, TF-IDF vectorizer
- Logging: Python logging to text/CSV

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% External Data Source
    Kaggle["Kaggle Dataset CSV"]:::dataClass

    %% Data Processing
    subgraph "Data Processing"
        direction TB
        RawData["Raw Data CSV"]:::dataClass
        RawDataProc["Ingestion & Preprocessing Scripts"]:::procClass
        Proc1["dataset_001"]:::dataClass
        Proc2["dataset_002"]:::dataClass
        Proc3["dataset_003"]:::dataClass
    end

    Kaggle --> RawData
    RawData --> RawDataProc
    RawDataProc --> Proc1
    RawDataProc --> Proc2
    RawDataProc --> Proc3

    click RawData "https://github.com/kaynup/binary-text-classification-with-sklearn-jax/blob/main/data/raw/Juggernaut Sentiment Analysis - by kaggle user Adeoluwa Adeboye.csv"
    click Proc1 "https://github.com/kaynup/binary-text-classification-with-sklearn-jax/tree/main/data/processed/dataset_001"
    click Proc2 "https://github.com/kaynup/binary-text-classification-with-sklearn-jax/tree/main/data/processed/dataset_002"
    click Proc3 "https://github.com/kaynup/binary-text-classification-with-sklearn-jax/tree/main/data/processed/dataset_003"

    %% Utilities
    subgraph "Utilities"
        direction TB
        Tokenizer["Tokenizers (src/tokenizers.py)"]:::procClass
        Utils["General Utilities (utils.py)"]:::procClass
    end

    click Tokenizer "https://github.com/kaynup/binary-text-classification-with-sklearn-jax/blob/main/src/tokenizers.py"
    click Utils "https://github.com/kaynup/binary-text-classification-with-sklearn-jax/blob/main/utils.py"

    %% Feature Extraction
    subgraph "Feature Extraction"
        direction TB
        TFIDF["TF-IDF Vectorizer"]:::procClass
        Embedding["Embedding & Pooling"]:::procClass
    end

    Proc1 --> TFIDF
    Proc1 --> Embedding
    Tokenizer --> TFIDF
    Tokenizer --> Embedding
    Utils --> TFIDF
    Utils --> Embedding

    %% Training Pipelines & Artifacts
    subgraph "Training Pipelines"
        direction TB
        SKLearnTrain["Scikit-learn Training Pipeline"]:::procClass
        JAXTrain["JAX/Flax Training Pipeline"]:::procClass
        SKModel["LogReg Model Artifact"]:::dataClass
        JAXModel1["pooling_batch-200-hidden=256 Pipeline"]:::dataClass
        JAXModel2["pooling_batch-200_2-hidden=256-128 Pipeline"]:::dataClass
        JAXModel3["pooling_batch-300 Pipeline"]:::dataClass
    end

    TFIDF --> SKLearnTrain
    Embedding --> JAXTrain
    SKLearnTrain --> SKModel
    JAXTrain --> JAXModel1
    JAXTrain --> JAXModel2
    JAXTrain --> JAXModel3

    click SKModel "https://github.com/kaynup/binary-text-classification-with-sklearn-jax/blob/main/models/sklearn/logreg-80k.joblib"
    click JAXModel1 "https://github.com/kaynup/binary-text-classification-with-sklearn-jax/blob/main/models/jax/bag_of_words-with_pooling/pooling_batch-200-hidden=256_pipeline.joblib"
    click JAXModel2 "https://github.com/kaynup/binary-text-classification-with-sklearn-jax/blob/main/models/jax/bag_of_words-with_pooling/pooling_batch-200_2-hidden=256-128_pipeline.joblib"
    click JAXModel3 "https://github.com/kaynup/binary-text-classification-with-sklearn-jax/blob/main/models/jax/bag_of_words-with_pooling/pooling_batch-300_pipeline.joblib"

    %% Experimentation Notebooks
    subgraph "Experimentation Notebooks"
        direction TB
        NB1["data-preprocessing.ipynb"]:::expClass
        NB2["sklearn-lab.ipynb"]:::expClass
        NB3["Jax-workstation.ipynb"]:::expClass
        NB4["sklearn-model-save.ipynb"]:::expClass
    end

    RawDataProc --> NB1
    TFIDF --> NB2
    Embedding --> NB3
    SKLearnTrain --> NB4

    click NB1 "https://github.com/kaynup/binary-text-classification-with-sklearn-jax/blob/main/notebooks/data-preprocessing.ipynb"
    click NB2 "https://github.com/kaynup/binary-text-classification-with-sklearn-jax/blob/main/notebooks/sklearn-lab.ipynb"
    click NB3 "https://github.com/kaynup/binary-text-classification-with-sklearn-jax/blob/main/notebooks/Jax-workstation.ipynb"
    click NB4 "https://github.com/kaynup/binary-text-classification-with-sklearn-jax/blob/main/notebooks/sklearn-model-save.ipynb"

    %% Inference API Layer
    subgraph "Inference API Layer"
        direction TB
        SKApp["Sklearn FastAPI App"]:::infClass
        JAXApp["JAX FastAPI App"]:::infClass
        UI["Static UI (index.html)"]:::infClass
    end

    TFIDF --> SKApp
    SKModel --> SKApp
    Tokenizer --> SKApp

    Embedding --> JAXApp
    JAXModel1 --> JAXApp
    JAXModel2 --> JAXApp
    JAXModel3 --> JAXApp
    Tokenizer --> JAXApp

    UI --> SKApp
    UI --> JAXApp

    click SKApp "https://github.com/kaynup/binary-text-classification-with-sklearn-jax/blob/main/local-deploy/sklearn-app.py"
    click JAXApp "https://github.com/kaynup/binary-text-classification-with-sklearn-jax/blob/main/local-deploy/jax-app.py"
    click UI "https://github.com/kaynup/binary-text-classification-with-sklearn-jax/blob/main/local-deploy/index.html"

    %% Logging
    subgraph "Logging"
        direction TB
        TrainLog1["training_history_2.csv"]:::dataClass
        TrainLog2["training_history_3.csv"]:::dataClass
        BenchLog["log_reg-val-check.txt"]:::dataClass
        InferLog["inference.log"]:::dataClass
    end

    SKLearnTrain --> TrainLog1
    SKLearnTrain --> TrainLog2
    SKLearnTrain --> BenchLog
    SKApp --> InferLog
    JAXApp --> InferLog

    click TrainLog1 "https://github.com/kaynup/binary-text-classification-with-sklearn-jax/blob/main/logs/training_logs/training_history_2.csv"
    click TrainLog2 "https://github.com/kaynup/binary-text-classification-with-sklearn-jax/blob/main/logs/training_logs/training_history_3.csv"
    click BenchLog "https://github.com/kaynup/binary-text-classification-with-sklearn-jax/blob/main/logs/initial-benchmark-logs/log_reg-val-check.txt"
    click InferLog "https://github.com/kaynup/binary-text-classification-with-sklearn-jax/blob/main/local-deploy/inference.log"

    %% Project Metadata
    subgraph "Project Metadata"
        direction TB
        Req["requirements.txt"]:::dataClass
        Readme["README.md"]:::dataClass
        Lic["LICENSE"]:::dataClass
    end

    classDef dataClass fill:#D0E6FF,stroke:#67A3FF
    classDef procClass fill:#D0FFD0,stroke:#3EBF3E
    classDef infClass fill:#FFE6C0,stroke:#FFA33E
    classDef expClass fill:#EEEEEE,stroke:#888888
```
