How naveedqadir/smartedu_classifier works
Project Type & Purpose
• This is an end-to-end machine-learning pipeline for classifying student performance into three categories.
• Key stages: synthetic data generation → preprocessing → exploratory data analysis (EDA) → model training & evaluation → model selection → deployment via CLI.
File-Structure Analysis
• Top level:
• data/ – raw CSVs
• notebooks/ – Jupyter analysis & EDA
• src/ – Python scripts for dataset generation, training, prediction, utilities
• models/ – saved model artifacts (joblib)
• outputs/ – evaluation plots & reports
• Config/build files: requirements.txt, .gitignore
• No microservices or web/frontend—just a batch-/CLI-based ML pipeline written in Python.
README Insights
• Emphasizes CLI for inference and notebook for interactive EDA.
Main Components & Interactions
A. Data Layer
• Synthetic Dataset Generator (src/generate_dataset.py)
• data/students.csv
B. Preprocessing
• utils.py: SimpleImputer, StandardScaler, LabelEncoder
C. EDA & Visualization
• notebooks/student_performance.ipynb (inline plots)
• outputs/*.png
D. Model Training
• src/train.py
• Models: LogisticRegression, DecisionTree, RandomForest, KNN, SVC, optional XGBoost
• 5-fold CV + optional GridSearch (--tune)
E. Model Registry
• models/best_model.joblib
• models/preprocessor.joblib
F. Evaluation Outputs
• outputs/model_metrics.csv, confusion_matrix.png, roc_curves.png, etc.
G. Inference (CLI)
• src/predict.py → outputs/predictions.csv or console output
1. Generator → raw CSV
2. CSV → Preprocessor → cleaned data
3. Cleaned data → EDA & Model Training
4. Training → Model Registry + Evaluation Outputs
5. Registry & Preprocessor → CLI Inference → Predictions
Architectural Patterns & Principles
• Linear pipeline (ETL → train → deploy)
• Clear separation of concerns: data generation, preprocessing, training, inference
• Reusable utilities (utils.py)
• Pluggable model registry (joblib-based)
• Command-line interface built with argparse
• Notebook mirrors script logic for transparency