How jatin-mehra119/map-charting-student-math-misunderstandings-solution works
Project Type & Purpose
• This is an ML pipeline for a Kaggle competition (“MAP – Charting Student Math Misunderstandings”).
• Its goal is to preprocess labelled student-response data, train six LLM-based classifiers with diverse tuning methods (LoRA, QLoRA, full fine-tune), and ensemble their outputs to maximize MAP@3.
Main Architectural Components
a. Data Ingestion & Storage
• Source: Kaggle dataset (CSV files for train/test).
• Tools: pandas, scikit-learn LabelEncoder.
b. Data Preprocessing & Feature Engineering
• Text assembly: “Question,” “Answer,” “Is Correct,” “Student Explanation.”
• Feature engineering:
• Create “target” = Category + ":" + Misconception
• Encode “label” via LabelEncoder
• Correct answers per QuestionId and flag is_correct
• (Additional) Derive features such as answer length, token counts, POS tag ratios, or domain-specific flags (e.g., decimal–whole number confusion indicator).
c. Model Training Modules (Parallel)
• Hunyuan 7B (LoRA)
• Qwen 3 4B (LoRA)
• Qwen 2 8B (QLoRA)
• Qwen 3 14B (QLoRA)
• Qwen 3 8B (Full FT)
• DeepseekMath 7B (Full FT)
• Shared tech: HuggingFace Transformers, PEFT, BitsAndBytes, PyTorch, GPUs.
d. Prediction Extraction
• Each model generates logits → softmax → top-K probabilities.
• Intermediate storage: numpy arrays or serialized JSON.
e. Ensemble Aggregator
• Combines model outputs via weighted probability sum, agreement bonus, confidence bonus.
• Applies family-based filtering (True_/False_ families) and backfills to ensure 3 outputs.
f. Inference & Submission
• Notebook (inference notebook.ipynb) that loads ensemble scores, formats submission CSV for Kaggle.
g. Write-up & Documentation
• writeup.md for methodology; README.md for overview.
Relationships & Data Flows
• Each Model → Prediction Extraction → feeds into Ensemble Aggregator
• Ensemble → Inference Notebook → Submission
Architectural Patterns & Principles
• Pipeline pattern with clear separation of concerns.
• Parallel training modules to exploit model diversity.
• Ensemble design for robustness and improved generalization.
• Modular feature engineering to decouple raw data transforms from model code.
Key Technologies & Frameworks
• Python, pandas, scikit-learn LabelEncoder
• HuggingFace Transformers + PEFT (LoRA, QLoRA)
• BitsAndBytes for 4-bit quantization
• PyTorch on GPU clusters (L4 instances)
• NumPy for probability handling