# nuthan-hue/finetuning_llms architecture

> Architecture diagram of the GitHub repository nuthan-hue/finetuning_llms, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-11-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nuthan-hue/finetuning_llms
- Repository: https://github.com/nuthan-hue/finetuning_llms

## Overview

##### Project Type and Purpose
- This is an open-source, AI-driven multi-agent “Kaggle competition slaying” framework.
- Purpose: Autonomously participate in any Kaggle competition; use LLMs (Google Gemini, OpenAI GPT, Anthropic Claude) to plan, train, submit, and iterate until top 20%.

##### File Structure
- **Top-level**
- Docker-related files (`Dockerfile`, `docker-compose.yml`, `.dockerignore`) for containerized deployment.
- `Makefile` for local build and orchestration shortcuts.
- `.env.example` for environment variables (AI keys, Kaggle keys, log levels).
- `init-db.sql` and `monitoring/prometheus.yml` for observability and state initialization.
- `kaggle_agent.py`, utility scripts under `scripts/`, notebooks under `notebook/` for experiments.
- `models/` for storing checkpoints and final trained models.
- `submissions/` historic CSV submission artifacts.
- `tests/` and `test_problem_understanding.py` for full test suite.

- **Core Source (`src/`)**
- `src/agents/` subdivided into:
– `llm_agents/` (ProblemUnderstandingAgent, DataAnalysisAgent, PlanningAgent, PreprocessingAgent, StrategyAgent) – the AI decision-makers.
– `orchestrator/` (Orchestrator, phases.py) – the workflow coordination engine.
– `data_collector/`, `model_trainer/`, `submission/`, `leaderboard/` – the worker modules that execute tasks.
- `src/utils/ai_caller.py` centralizes LLM provider integration.
- CLI (`src/cli.py`) and entrypoint (`src/main.py`).

##### README for Architectural Insights
- Clearly states a three‐tier architecture:
1. AI AGENTS (Decision Makers)
2. ORCHESTRATOR (Workflow Manager)
3. WORKERS (Task Executors)
- Lists external integrations: Kaggle API, LLM provider endpoints, Prometheus monitoring.

a. Main Components
- Layer 1: AI Agents (LLM Agents)
- Layer 2: Orchestrator
- Layer 3: Workers (Data Collector, Model Trainer, Submission Handler, Leaderboard Monitor)
- Data Stores: Raw & Processed Data (directories under `data/`), Model Artifacts (`models/`), Logs (`logs/`), Submissions (`submissions/`)
- External Services: Kaggle API, LLM Provider(s), Prometheus
- Infrastructure: Docker containers, Docker Compose network

b. Relationships and Interactions
- Orchestrator → Workers: “Tasks & Context”
- Workers → Data Store: read/write raw & processed files, models, logs
- Workers ↔ External Services:
- Data Collector ↔ Kaggle API
- Submission Handler ↔ Kaggle API
- Leaderboard Monitor → Kaggle API → Orchestrator/AI Agents (feedback loop)
- ai_caller module as a shared utility invoked by all LLM Agents.

c. Architectural Patterns and Principles
- Three-tier layered pattern (Presentation/Decision, Coordination, Execution)
- Plugin-style Agent pattern (BaseAgent inheritance) for extensibility
- Zero-hardcoded logic principle – all decisions flow from LLM Agents
- Iterative feedback loop (leaderboard → strategy refinement)

d. Relevant Technologies
- Python 3.8+
- LLM Providers: Google Gemini, OpenAI GPT, Anthropic Claude (`src/utils/ai_caller.py`)
- ML frameworks: LightGBM, XGBoost, PyTorch (MLP)
- Containerization: Docker, Docker Compose
- Monitoring: Prometheus (`monitoring/prometheus.yml`)
- Testing: pytest (tests/)
- Notebook support: Jupyter (notebooks/)
- CLI & Async: `asyncio`, custom `cli.py` / `main.py`

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% AI Agents Layer
    subgraph "AI Agents (Decision Layer)"
        direction TB
        BaseAgent["BaseAgent"]:::agent
        PUAgent["ProblemUnderstandingAgent"]:::agent
        DAAgent["DataAnalysisAgent"]:::agent
        PlanAgent["PlanningAgent"]:::agent
        PreprocAgent["PreprocessingAgent"]:::agent
        StratAgent["StrategyAgent"]:::agent
    end

    %% Orchestrator Layer
    subgraph "Orchestrator (Coordination Layer)"
        direction TB
        Orchestrator["Orchestrator"]:::orchestrator
        Phases["phases.py"]:::orchestrator
        OptAI["optimization_ai.py"]:::orchestrator
    end

    %% Workers Layer
    subgraph "Workers (Execution Layer)"
        direction TB
        subgraph "Data Collector"
            direction TB
            DC_Collector["collector.py"]:::worker
            DC_Analysis["analysis.py"]:::worker
            DC_External["external.py"]:::worker
            DC_Kaggle["kaggle_data.py"]:::worker
            DC_Utils["utils.py"]:::worker
        end
        subgraph "Model Trainer"
            direction TB
            MT_Trainer["trainer.py"]:::worker
            MT_Pipeline["data_pipeline.py"]:::worker
            MT_Preproc["preprocessing.py"]:::worker
            MT_Models["models.py"]:::worker
            MT_Detect["detection.py"]:::worker
        end
        subgraph "Submission Handler"
            direction TB
            SH_Submitter["submitter.py"]:::worker
            SH_Formatting["formatting.py"]:::worker
            SH_KaggleAPI["kaggle_api.py"]:::worker
            SH_Predictions["predictions.py"]:::worker
            SH_Preproc["preprocessing.py"]:::worker
            SH_Utils["utils.py"]:::worker
        end
        subgraph "Leaderboard Monitor"
            direction TB
            LB_Monitor["monitor.py"]:::worker
            LB_Fetcher["fetcher.py"]:::worker
            LB_Analysis["analysis.py"]:::worker
            LB_Utils["utils.py"]:::worker
        end
    end

    %% Shared Utility
    AICaller["ai_caller.py"]:::utility

    %% CLI & Entry
    CLI["cli.py"]:::cli
    Main["main.py"]:::cli
    Entry["kaggle_agent.py"]:::cli

    %% Data Stores
    subgraph "Data Stores"
        direction TB
        RawData["Raw Data\n(data/raw/)"]:::datastore
        ProcData["Processed Data\n(data/processed/)"]:::datastore
        ModelsStore["Model Artifacts\n(models/)"]:::datastore
        Logs["Logs\n(logs/)"]:::datastore
        SubsStore["Submissions\n(submissions/)"]:::datastore
    end

    %% External Services
    KaggleAPI["Kaggle API"]:::external
    LLMProviders["LLM Providers\n(Gemini, GPT, Claude)"]:::external
    Prometheus["Prometheus"]:::external

    %% Arrows: Agents to Orchestrator
    BaseAgent -->|loads| Orchestrator
    PUAgent -->|Decisions & Plans| Orchestrator
    DAAgent -->|Decisions & Plans| Orchestrator
    PlanAgent -->|Decisions & Plans| Orchestrator
    PreprocAgent -->|Decisions & Plans| Orchestrator
    StratAgent -->|Decisions & Plans| Orchestrator

    %% Orchestrator to Workers
    Orchestrator -->|Tasks & Context| DC_Collector
    Orchestrator -->|Tasks & Context| MT_Trainer
    Orchestrator -->|Tasks & Context| SH_Submitter
    Orchestrator -->|Tasks & Context| LB_Monitor

    %% Workers to Data Stores
    DC_Collector -->|read/write| RawData
    DC_Analysis -->|analyze| ProcData
    MT_Trainer -->|read| ProcData
    MT_Models -->|write| ModelsStore
    SH_Submitter -->|write| SubsStore
    LB_Monitor -->|read| SubsStore
    LB_Monitor -->|log| Logs

    %% Workers to External
    DC_Kaggle -->|"fetch data()"| KaggleAPI
    SH_KaggleAPI -->|"submit_predictions()"| KaggleAPI
    LB_Fetcher -->|"monitor_leaderboard()"| KaggleAPI

    %% Feedback Loop
    LB_Monitor -->|feedback| StratAgent
    StratAgent -->|refine plan| Orchestrator

    %% Shared Utility usage
    PUAgent -->|invoke| AICaller
    DAAgent -->|invoke| AICaller
    PlanAgent -->|invoke| AICaller
    PreprocAgent -->|invoke| AICaller
    StratAgent -->|invoke| AICaller

    %% CLI & Entrypoint
    Entry -->|calls| Main
    Main -->|invokes| CLI
    Main -->|starts| Orchestrator

    %% Monitoring
    Prometheus -->|scrape metrics| Logs
    Prometheus -->|scrape metrics| Orchestrator

    %% Click Events
    click BaseAgent "https://github.com/nuthan-hue/finetuning_llms/blob/master/src/agents/llm_agents/base_llm_agent.py"
    click PUAgent "https://github.com/nuthan-hue/finetuning_llms/blob/master/src/agents/llm_agents/problem_understanding_agent.py"
    click DAAgent "https://github.com/nuthan-hue/finetuning_llms/blob/master/src/agents/llm_agents/data_analysis_agent.py"
    click PlanAgent "https://github.com/nuthan-hue/finetuning_llms/blob/master/src/agents/llm_agents/planning_agent.py"
    click PreprocAgent "https://github.com/nuthan-hue/finetuning_llms/blob/master/src/agents/llm_agents/preprocessing_agent.py"
    click StratAgent "https://github.com/nuthan-hue/finetuning_llms/blob/master/src/agents/llm_agents/strategy_agent.py"
    click Orchestrator "https://github.com/nuthan-hue/finetuning_llms/blob/master/src/agents/orchestrator/orchestrator.py"
    click Phases "https://github.com/nuthan-hue/finetuning_llms/blob/master/src/agents/orchestrator/phases.py"
    click OptAI "https://github.com/nuthan-hue/finetuning_llms/blob/master/src/agents/orchestrator/optimization_ai.py"
    click DC_Collector "https://github.com/nuthan-hue/finetuning_llms/blob/master/src/agents/data_collector/collector.py"
    click DC_Analysis "https://github.com/nuthan-hue/finetuning_llms/blob/master/src/agents/data_collector/analysis.py"
    click DC_External "https://github.com/nuthan-hue/finetuning_llms/blob/master/src/agents/data_collector/external.py"
    click DC_Kaggle "https://github.com/nuthan-hue/finetuning_llms/blob/master/src/agents/data_collector/kaggle_data.py"
    click DC_Utils "https://github.com/nuthan-hue/finetuning_llms/blob/master/src/agents/data_collector/utils.py"
    click MT_Trainer "https://github.com/nuthan-hue/finetuning_llms/blob/master/src/agents/model_trainer/trainer.py"
    click MT_Pipeline "https://github.com/nuthan-hue/finetuning_llms/blob/master/src/agents/model_trainer/data_pipeline.py"
    click MT_Preproc "https://github.com/nuthan-hue/finetuning_llms/blob/master/src/agents/model_trainer/preprocessing.py"
    click MT_Models "https://github.com/nuthan-hue/finetuning_llms/blob/master/src/agents/model_trainer/models.py"
    click MT_Detect "https://github.com/nuthan-hue/finetuning_llms/blob/master/src/agents/model_trainer/detection.py"
    click SH_Submitter "https://github.com/nuthan-hue/finetuning_llms/blob/master/src/agents/submission/submitter.py"
    click SH_Formatting "https://github.com/nuthan-hue/finetuning_llms/blob/master/src/agents/submission/formatting.py"
    click SH_KaggleAPI "https://github.com/nuthan-hue/finetuning_llms/blob/master/src/agents/submission/kaggle_api.py"
    click SH_Predictions "https://github.com/nuthan-hue/finetuning_llms/blob/master/src/agents/submission/predictions.py"
    click SH_Preproc "https://github.com/nuthan-hue/finetuning_llms/blob/master/src/agents/submission/preprocessing.py"
    click SH_Utils "https://github.com/nuthan-hue/finetuning_llms/blob/master/src/agents/submission/utils.py"
    click LB_Monitor "https://github.com/nuthan-hue/finetuning_llms/blob/master/src/agents/leaderboard/monitor.py"
    click LB_Fetcher "https://github.com/nuthan-hue/finetuning_llms/blob/master/src/agents/leaderboard/fetcher.py"
    click LB_Analysis "https://github.com/nuthan-hue/finetuning_llms/blob/master/src/agents/leaderboard/analysis.py"
    click LB_Utils "https://github.com/nuthan-hue/finetuning_llms/blob/master/src/agents/leaderboard/utils.py"
    click AICaller "https://github.com/nuthan-hue/finetuning_llms/blob/master/src/utils/ai_caller.py"
    click CLI "https://github.com/nuthan-hue/finetuning_llms/blob/master/src/cli.py"
    click Main "https://github.com/nuthan-hue/finetuning_llms/blob/master/src/main.py"
    click Entry "https://github.com/nuthan-hue/finetuning_llms/blob/master/kaggle_agent.py"
    click RawData "https://github.com/nuthan-hue/finetuning_llms/tree/master/data/raw/"
    click ProcData "https://github.com/nuthan-hue/finetuning_llms/tree/master/data/processed/"
    click ModelsStore "https://github.com/nuthan-hue/finetuning_llms/blob/master/models/final/lightgbm_model.txt"
    click SubsStore "https://github.com/nuthan-hue/finetuning_llms/blob/master/submissions/titanic_submission.csv"
    click Prometheus "https://github.com/nuthan-hue/finetuning_llms/blob/master/monitoring/prometheus.yml"

    %% Styles
    classDef agent fill:#D0E8FF,stroke:#4A90E2,stroke-width:2px;
    classDef orchestrator fill:#DFF7D8,stroke:#34A853,stroke-width:2px;
    classDef worker fill:#FFE5B4,stroke:#FB8C00,stroke-width:2px;
    classDef utility fill:#F2E7FE,stroke:#9B51E0,stroke-width:2px;
    classDef cli fill:#E8E8E8,stroke:#666666,stroke-width:2px;
    classDef datastore fill:#E2E2E2,stroke:#8A8A8A,stroke-width:2px,stroke-dasharray: 5 5;
    classDef external fill:#FCE8E6,stroke:#D32F2F,stroke-width:2px;
```
