How nuthan-hue/finetuning_llms works
Project Type and Purpose
• This is an open-source, AI-driven multi-agent “Kaggle competition slaying” framework.
• Purpose: Autonomously participate in any Kaggle competition; use LLMs (Google Gemini, OpenAI GPT, Anthropic Claude) to plan, train, submit, and iterate until top 20%.
File Structure
• Top-level
• Docker-related files (Dockerfile, docker-compose.yml, .dockerignore) for containerized deployment.
• Makefile for local build and orchestration shortcuts.
• .env.example for environment variables (AI keys, Kaggle keys, log levels).
• init-db.sql and monitoring/prometheus.yml for observability and state initialization.
• kaggle_agent.py, utility scripts under scripts/, notebooks under notebook/ for experiments.
• models/ for storing checkpoints and final trained models.
• submissions/ historic CSV submission artifacts.
• tests/ and test_problem_understanding.py for full test suite.
• Core Source (src/)
• src/agents/ subdivided into:
– llm_agents/ (ProblemUnderstandingAgent, DataAnalysisAgent, PlanningAgent, PreprocessingAgent, StrategyAgent) – the AI decision-makers.
– orchestrator/ (Orchestrator, phases.py) – the workflow coordination engine.
– data_collector/, model_trainer/, submission/, leaderboard/ – the worker modules that execute tasks.
• src/utils/ai_caller.py centralizes LLM provider integration.
• CLI (src/cli.py) and entrypoint (src/main.py).
README for Architectural Insights
• Clearly states a three‐tier architecture:
1. AI AGENTS (Decision Makers)
2. ORCHESTRATOR (Workflow Manager)
3. WORKERS (Task Executors)
• Lists external integrations: Kaggle API, LLM provider endpoints, Prometheus monitoring.
a. Main Components
• Layer 1: AI Agents (LLM Agents)
• Layer 2: Orchestrator
• Layer 3: Workers (Data Collector, Model Trainer, Submission Handler, Leaderboard Monitor)
• Data Stores: Raw & Processed Data (directories under data/), Model Artifacts (models/), Logs (logs/), Submissions (submissions/)
• External Services: Kaggle API, LLM Provider(s), Prometheus
• Infrastructure: Docker containers, Docker Compose network
b. Relationships and Interactions
• Orchestrator → Workers: “Tasks & Context”
• Workers → Data Store: read/write raw & processed files, models, logs
• Workers ↔ External Services:
• Data Collector ↔ Kaggle API
• Submission Handler ↔ Kaggle API
• Leaderboard Monitor → Kaggle API → Orchestrator/AI Agents (feedback loop)
• ai_caller module as a shared utility invoked by all LLM Agents.
c. Architectural Patterns and Principles
• Three-tier layered pattern (Presentation/Decision, Coordination, Execution)
• Plugin-style Agent pattern (BaseAgent inheritance) for extensibility
• Zero-hardcoded logic principle – all decisions flow from LLM Agents
• Iterative feedback loop (leaderboard → strategy refinement)
d. Relevant Technologies
• Python 3.8+
• LLM Providers: Google Gemini, OpenAI GPT, Anthropic Claude (src/utils/ai_caller.py)
• ML frameworks: LightGBM, XGBoost, PyTorch (MLP)
• Containerization: Docker, Docker Compose
• Monitoring: Prometheus (monitoring/prometheus.yml)
• Testing: pytest (tests/)
• Notebook support: Jupyter (notebooks/)
• CLI & Async: asyncio, custom cli.py / main.py