How sumedh220122/fine-tunining-ibm-granite-to-interpret-heat-equation works
Project Type and Purpose
• This is a machine-learning pipeline project, specifically a fine-tuning workflow for IBM Granite LLM to solve the time-dependent heat equation.
• The two Jupyter notebooks partition responsibilities:
• dataset-creation.ipynb generates and formats the Q&A dataset
• llm-fine-tuning.ipynb orchestrates preprocessing, tokenization, PEFT fine-tuning, and evaluation
File Structure
• Top-level artifacts:
• README.md (project overview & methodology)
• dataset-creation.ipynb (data-prep module)
• llm-fine-tuning.ipynb (training & evaluation module)
• No separate “src” or “tests”—the notebooks themselves are the orchestration layer.
• External dependencies: Kaggle dataset, Hugging Face transformers & PEFT, cosine-similarity embedding service.
Main Components to Model
a. Data Ingestion & Creation
• Source: Kaggle heat-solution dataset
• Notebook: dataset-creation.ipynb
• Output: raw Q&A pairs in train/validation splits
b. Preprocessing Pipeline
• Steps: pad token definition, data collator, tokenization (batch map-reduce)
• Tools: Hugging Face Tokenizers, custom collator functions
c. Fine-Tuning Engine
• Framework: Hugging Face Transformers + PEFT
• Compute: GPU/TPU environment invoked inside llm-fine-tuning.ipynb
• Model: IBM Granite base model → partially fine-tuned
d. Evaluation Module
• Metric: cosine similarity between generated answers & references
• Components: embedding model (external), compute_metrics function
e. Artifacts & Storage
• Intermediate: tokenized datasets, model checkpoints
• Final: fine-tuned LLM model, evaluation reports
Relationships & Data Flow
raw dataset → data-creation notebook → processed dataset → preprocessing submodule → tokenized batches → fine-tuning engine → checkpoints → evaluation module → final metrics
• Kaggle dataset download
• Embedding service for cosine similarity
• (Optional) Hugging Face Hub for model storage
Architectural Patterns & Design Principles
• Pipeline pattern: sequential transformation stages
• Modular separation: data-prep vs. training vs. evaluation
• PEFT pattern: freezes most model parameters, tunes a small subset
• Notebook orchestration: each Jupyter file encapsulates one or more pipeline stages
Technology Stack Call-outs
• Python 3.x, Jupyter Notebooks
• Hugging Face Transformers, PEFT library
• Tokenizers library, custom DataCollator
• Cosine-similarity embedder (could be sentence-transformers)
• Kaggle dataset API