How raaghavagarwal28/fashion-mnist works
Project Type and Purpose
• This is a machine-learning research/demo pipeline contained in a Jupyter notebook (with a companion Python script).
• Purpose: train and evaluate a CNN on Fashion-MNIST, produce Grad-CAM visualizations, quantitatively evaluate with a “Pointing Game,” and generate natural-language explanations via Google Gemini LLM.
File Structure
• Top level:
• Another_copy_of_LuvByte.ipynb (the orchestrating notebook)
• another_copy_of_luvbyte.py (script version of the same pipeline)
• README.md
• No traditional frontend/backend split or service directories—this is a single‐process ML pipeline.
Key Architectural Insights from the README
• Pipeline stages are implemented sequentially in code cells/functions.
• External dependency: Google Gemini API (for LLM explanations).
• Core technologies: TensorFlow/Keras, tf-keras-vis (Grad-CAM), OpenCV, scikit-learn, Matplotlib/Seaborn, Pillow, google-generativeai.
Main Components & Their Interactions
a. Data Ingestion & Preprocessing Module
• Load Fashion-MNIST from Keras datasets
• Normalize pixel values, reshape to (28,28,1)
b. Model Definition & Training Module
• CNN with Conv2D, BatchNorm, MaxPooling, GlobalAveragePooling2D, Dense, Dropout
• Compile and train with early stopping
c. Evaluation Module
• Compute test accuracy and macro-F1Score
• Generate confusion matrix plots
• Run “Pointing Game” quantitative evaluation on Grad-CAM localizations
d. Grad-CAM Visualization Module
• Use tf-keras-vis to overlay heatmaps on input images
• Produce visual outputs for each test/custom image
e. LLM Explanation Module
• Send Grad-CAM outputs (e.g., heatmap statistics, key pixel location) to Google Gemini API
• Receive and render natural-language explanation
f. Custom Image Prediction Pipeline
• Accept user image (via notebook file upload cell)
• Preprocess, run through trained CNN, Grad-CAM, and LLM modules
• Prediction, heatmap overlay, and textual explanation
Architectural Patterns & Design Principles
• Linear data-processing pipeline (ETL-style with distinct stages)
• Modular code cells/functions that can be extracted into standalone scripts
• Separation of concerns: model training vs. interpretability vs. LLM integration
• Beneath it, arrange modules left to right in execution order:
1. Data Ingestion & Preprocessing
2. Model Definition & Training
3. Evaluation (Accuracy, F1, Confusion Matrix, Pointing Game)
4. Grad-CAM Visualization
5. LLM Explanation (External Service: Google Gemini API)
6. Custom Image Prediction Pipeline
• Raw dataset → Preprocessing → Trained Model
• Trained Model + Test Images → Evaluation & Grad-CAM
• Grad-CAM Outputs → Pointing Game & LLM API → Explanations
• Custom Image Input → entire pipeline
Detail
• Every functional block from the notebook is represented.
• Both quantitative evaluation (Pointing Game) and qualitative explanation (Gemini LLM).
• Don’t forget the user-uploaded custom image flow.