How kaynup/binary-text-classification-with-sklearn-jax works
Project type and purpose
• This is an open-source sentiment classification toolkit with two parallel modeling branches:
• A scikit-learn baseline (Logistic Regression + TF-IDF)
• A JAX/Flax/Optax neural prototype (embedding + pooling, with plans for LSTM)
• It provides both training pipelines (notebooks + scripts) and local inference endpoints via FastAPI.
File structure
• data/
– raw: original CSV from Kaggle
– processed: per-configuration TF-IDF matrices, vectorizer pickles, label splits
• notebooks/: exploratory analysis, preprocessing, model training & saving
• src/tokenizers.py + utils.py: reusable code for text tokenization/preprocessing
• models/
– sklearn: saved .joblib logistic regression
– jax: saved Flax pipelines (.joblib)
• local-deploy/: two FastAPI apps (sklearn-app.py, jax-app.py), static index.html, logs
• logs/: benchmark and training history logs
• requirements.txt, README.md, LICENSE
README for additional insights
• Confirms the dual-branch modeling strategy
• Describes dataset origin, performance benchmarks, known limitations
• Details how to run each FastAPI endpoint and sample request/response
• Notes logging behavior and common errors
a. Main components
• Data Ingestion & Preprocessing
• Feature Extraction & Tokenization
• Model Training Pipelines (sklearn vs JAX)
• Model Artifact Storage (models/sklearn, models/jax)
• Inference API Layer (FastAPI apps)
• Logging & Monitoring
• Experimentation Notebooks
• Utility Modules
• External Dataset Source (Kaggle)
b. Relationships & interactions
1. Raw data → preprocessing scripts/notebooks → processed data
2. Processed data + tokenizers → feature matrices (TF-IDF or embeddings)
3. Feature matrices → training pipelines → trained model artifacts
4. Inference request (JSON) → FastAPI server → vectorizer/tokenizer → model.predict → JSON response
5. Each inference writes to local-deploy/inference.log
6. Training logs written to logs/training_logs and logs/initial-benchmark-logs
c. Architectural patterns & design principles
• Separation of concerns: data, training, inference, logging each in its own module/directory
• Two “micro-services” for inference (one per modeling branch)
• Reusable utility and tokenizer libraries share code between training & inference
• Versioned datasets and models allow reproducibility and experimentation
d. Key technologies, frameworks & libraries
• Core language: Python
• Modeling: scikit-learn, JAX, Flax, Optax
• Serialization: joblib, pickle, NPZ
• API: FastAPI, Uvicorn
• Data: NumPy, SciPy, TF-IDF vectorizer
• Logging: Python logging to text/CSV