How karitselmuthu/hugging_face_fine_tuning works
This is a reusable text fine-tuning toolkit: task configurations and source datasets become prepared prompt/response splits, which are used to train full models or LoRA adapters, evaluate generations, and generate answers from saved runs. Data integrity checks and task-specific metrics support that workflow; audit and label-comparison commands provide additional analysis. The README also documents a separate, earlier horoscope-specific experiment suite, including MLX QLoRA examples. Connections for that suite are documented at the workflow level; its individual implementation paths were not sampled.
Main entry points: train_task.py Train model · evaluate_task.py Evaluate runs · prepare_task.py Prepare splits · infer_task.py Generate answers
Components 17
Reusable pipeline
- Task configurations
tasks - Prepare splits
src/prepare_task.py - Prepared splits
data - Task helpers
src/task_core.py - Train model
src/train_task.py - Saved runs
models - Generate answers
src/infer_task.py
Data safeguards
- Data integrity
src/data_integrity.py - Audit prepared data
src/audit_task_data.py
Evaluation and analysis
- Evaluate runs
src/evaluate_task.py - Task metrics
src/task_metrics.py - Evaluation results
results - Compare label runs
src/compare_label_evaluations.py
Horoscope experiments
- Horoscope examples
src/horoscope_experiments
Other components
- Task author
- Task datasets
- Hugging Face models
Connections 28
- Task author → Task configurations: configures
README.md - Task datasets → Prepare splits: loads data
src/prepare_task.py - Task configurations → Prepare splits: configures
src/prepare_task.py - Prepare splits → Task helpers: formats prompts
src/prepare_task.py - Prepare splits → Data integrity: records fingerprints
src/prepare_task.py - Prepare splits → Prepared splits: writes splits
README.md - Task author → Train model: starts training
README.md - Task configurations → Train model: configures
src/train_task.py - Train model → Prepared splits: reads splits
src/train_task.py - Train model → Task helpers: uses helpers
src/train_task.py - Train model → Data integrity: verifies data
src/train_task.py - Hugging Face models → Train model: provides starting model
README.md - Train model → Saved runs: saves run
README.md - Task author → Evaluate runs: starts evaluation
README.md - Evaluate runs → Task helpers: loads run helpers
src/evaluate_task.py - Evaluate runs → Prepared splits: reads split
src/evaluate_task.py - Evaluate runs → Data integrity: verifies data
src/evaluate_task.py - Evaluate runs → Task metrics: scores generations
src/evaluate_task.py - Evaluate runs → Evaluation results: writes evaluations
README.md - Task author → Generate answers: provides inputs
README.md - Generate answers → Task helpers: formats prompt
src/infer_task.py - Generate answers → Saved runs: loads saved run
README.md - Hugging Face models → Generate answers: backs saved run
- Audit prepared data → Data integrity: checks artifacts
src/audit_task_data.py - Audit prepared data → Prepared splits: audits splits
README.md - Compare label runs → Evaluation results: compares evaluations
- Task author → Horoscope examples: runs examples
README.md - Horoscope examples → Hugging Face models: uses models