# macrocosm-os/finetuning architecture

> Architecture diagram of the GitHub repository macrocosm-os/finetuning, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/macrocosm-os/finetuning
- Repository: https://github.com/macrocosm-os/finetuning

## Overview

##### Project Type and Purpose
- This project is a specialized full-stack application developed to support fine-tuning of large language models (LLMs) under competitive scenarios. It integrates key Bittensor concepts like Miners, Validators, and on-chain incentive mechanisms.
- The README outlines that miners fine-tune models and push them to Hugging Face, while validators download and evaluate these models against continuously generated synthetic data. The evaluations are eventually used to update weights (incentives) on the Bittensor chain.

##### File Structure to Determine Architectural Components
- Top-level folders like “finetune”, “competitions”, “neurons”, “constants”, and “scripts” indicate how different concerns are separated:
- The “finetune” directory contains core logic for fine-tuning, dataset management, evaluation, and utilities.
- “competitions” and “constants” hold competition-specific configurations, parameters, and emission splits.
- The “neurons” directory comprises the components for managing Miner and Validator behaviors, including configuration and interfacing with the Bittensor chain.
- “scripts” provide CLI tools for running benchmarks, validating models, uploading, and other operational tasks.
- “tests” mirror the structure to ensure code quality through unit tests across datasets, evaluation, and the model’s behavior.
- CI/CD support is provided via the .circleci directory, underlining the continuous integration pipeline.

##### Key Components
a. Core Components:
- Finetuning Engine: This is the heart of the system (in the “finetune” module). It manages training workflows, dataset ingestion, evaluation (including if_eval submodules for metrics like bullet_count, casing, comma, etc.), and overall fine-tuning orchestration.
- Dataset Management: Under “finetune/datasets”, there are multiple loaders (generated synthetic data, Hugging Face loaders, subnet-specific prompt loaders) that prepare data for training and evaluation.
- Evaluation Module: The “finetune/eval” directory shows that there’s a robust evaluation pipeline where validators compute various metrics, compare models, and generate performance statistics.

b. External Interfaces and Supporting Components:
- Neurons Module: Acting as a bridge between internal logic (finetuning and evaluation) and external systems, the “neurons” module implements Miner and Validator functionalities that interact with the Bittensor blockchain (for metadata, mining, and validation operations).
- Competitions and Constants: Representing the configuration and parameter settings for each competition, they define the incentives and evaluation criteria.
- External Services:
- Hugging Face: Miners upload their fine-tuned models here.
- Bittensor Chain: Stores miners’ metadata and handles consensus via Yuma Consensus.
- WandB: For logging evaluation and benchmarking results.

c. Utility and Operational Scripts:
- Scripts for launching validators, checking compatibility, benchmarking, and uploading models help orchestrate the overall workflow.
- CI/CD integration ensures automated testing and deployment.

##### Illustrating Relationships and Interactions
- Feedback and logging pipelines should also be clearly outlined (e.g., connection to WandB).

##### Architectural Patterns & Technologies
- The architecture implies a modular design with clear separation of concerns (data ingestion, training, evaluation, and external communications).
- Patterns such as the separation of concerns and micro-modules enable isolated testing (as seen in the “tests” folder).
- The extensive use of scripts and configuration files (such as .circleci/config.yml) further emphasize automated deployment and continuous integration.
- Technologies to note are:
- Hugging Face for model hosting.
- Bittensor blockchain for incentive mechanisms.
- Possibly WandB for log aggregation.
- CI/CD tools (e.g., CircleCI) for automated builds and tests.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Internal Components
    subgraph "Backend"
        FT["Finetuning Engine"]:::internal
        DS["Dataset Loaders"]:::internal
        EV["Evaluation Module (if_eval)"]:::internal
        NM["Neurons Module (Miner & Validator)"]:::internal
    end

    %% Configuration Components
    subgraph "Configurations"
        CC["Competitions & Constants"]:::config
    end

    %% External Systems
    subgraph "External Services"
        HF["Hugging Face"]:::external
        BTC["Bittensor Chain"]:::external
        WB["WandB"]:::external
    end

    %% Utility Components
    subgraph "Utilities & Testing"
        UT["Utility and Operational Scripts"]:::utility
        CI["CI/CD Pipeline"]:::utility
        TS["Testing Infrastructure"]:::utility
    end

    %% Data Flow Connections
    FT -->|"data_ingestion"| DS
    FT -->|"model_evaluation"| EV
    EV -->|"logs_to"| WB
    EV -->|"feedback_loop"| NM
    NM -->|"upload_download"| HF
    NM -->|"chain_metadata"| BTC

    %% Operational and CI/CD Interactions
    UT -->|"orchestrates"| FT
    UT -->|"orchestrates"| NM
    CI -->|"triggers"| UT
    TS -->|"tests"| FT
    TS -->|"tests"| EV

    %% Click Events for Component Mapping
    click FT "https://github.com/macrocosm-os/finetuning/tree/main/finetune/"
    click DS "https://github.com/macrocosm-os/finetuning/tree/main/finetune/datasets/"
    click EV "https://github.com/macrocosm-os/finetuning/tree/main/finetune/eval/"
    click NM "https://github.com/macrocosm-os/finetuning/tree/main/neurons/"
    click CC "https://github.com/macrocosm-os/finetuning/tree/main/competitions/"
    click UT "https://github.com/macrocosm-os/finetuning/tree/main/scripts/"
    click CI "https://github.com/macrocosm-os/finetuning/tree/main/.circleci/"
    click TS "https://github.com/macrocosm-os/finetuning/tree/main/tests/"

    %% Styles
    classDef internal fill:#a7d8de,stroke:#333,stroke-width:2px;
    classDef external fill:#f9e79f,stroke:#333,stroke-width:2px;
    classDef config fill:#d5f5e3,stroke:#333,stroke-width:2px;
    classDef utility fill:#fad7a0,stroke:#333,stroke-width:2px;
```
