# aidc-ai/marco-o1 architecture

> Architecture diagram of the GitHub repository aidc-ai/marco-o1, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2024-12-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/aidc-ai/marco-o1
- Repository: https://github.com/aidc-ai/marco-o1

## Overview

##### Main Components

a) Core Model Components:
- Marco-o1-CoT (Base Model)
- MCTS Integration Layer
- Reasoning Action Strategy Module
- Reflection Mechanism

b) Data Processing Components:
- Dataset Handlers (for CoT datasets)
- Training Pipeline
- Inference Pipeline

c) API/Service Layer:
- FastAPI Service
- Client Interface
- Streaming Support

**[Client Layer]**

**[API Layer]**

**[Core Processing Layer]**

**[Model Layer]**

**[Data Layer]**

##### Specific Elements

a) Data Layer:
- Three dataset types:
- Open-O1 CoT Dataset
- Marco-o1 CoT Dataset
- Marco Instruction Dataset

b) Model Layer:
- Fine-tuned Marco-o1-CoT
- MCTS Components:
- Node Management
- Action Selection
- Confidence Scoring
- Reward Calculation

c) Core Processing Layer:
- Reasoning Strategy Handler
- Mini-step Processing
- Reflection Mechanism
- MCTS Tree Search Implementation

d) API Layer:
- FastAPI Endpoints
- vLLM Integration
- Stream Processing

e) Client Layer:
- Python Client Interface
- Streaming Client Interface

##### Component Details

Annotations for key components showing:
- Input/Output formats
- Key configurations
- Performance metrics
- Resource requirements

##### Integration Points

Connections with:
- Hugging Face model repository
- External APIs
- Monitoring systems
- Deployment infrastructure

- The modular nature of the system
- Clear separation between components
- Scalability points
- Data flow through the system

This system design should reflect the project's focus on reasoning capabilities and its innovative use of MCTS and reflection mechanisms while showing how these components interact with the base model and API services.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Style Definitions
    classDef model fill:#b3d9ff,stroke:#333,stroke-width:2px
    classDef data fill:#90EE90,stroke:#333,stroke-width:2px
    classDef api fill:#FFE5B4,stroke:#333,stroke-width:2px
    classDef client fill:#DDA0DD,stroke:#333,stroke-width:2px
    classDef processing fill:#F0F0F0,stroke:#333,stroke-width:2px

    %% Client Layer
    subgraph ClientLayer
        CI["Client Interface"]:::client
        SCI["Streaming Client Interface"]:::client
    end
    click CI "https://github.com/AIDC-AI/Marco-o1/blob/main/examples/client.py"
    click SCI "https://github.com/AIDC-AI/Marco-o1/blob/main/examples/stream_client.py"

    %% API Layer
    subgraph APILayer
        FAPI["FastAPI Service"]:::api
        SFAPI["Streaming FastAPI Service"]:::api
        VLLMI["vLLM Integration"]:::api
    end
    click FAPI "https://github.com/AIDC-AI/Marco-o1/blob/main/examples/vllm_fastapi.py"
    click SFAPI "https://github.com/AIDC-AI/Marco-o1/blob/main/examples/stream_vllm_fastapi.py"
    click VLLMI "https://github.com/AIDC-AI/Marco-o1/blob/main/src/talk_with_model_vllm.py"

    %% Core Processing Layer
    subgraph ProcessingLayer
        RSH["Reasoning Strategy Handler"]:::processing
        MSP["Mini-step Processing"]:::processing
        RM["Reflection Mechanism"]:::processing
        MCTS["MCTS Implementation"]:::processing
        MI["Model Interaction Layer"]:::processing
    end
    click MI "https://github.com/AIDC-AI/Marco-o1/blob/main/src/talk_with_model.py"

    %% Model Layer
    subgraph ModelLayer
        BM["Base Model\n(Qwen2-7B-Instruct)"]:::model
        FT["Fine-tuned Marco-o1-CoT"]:::model
        subgraph MCTSComponents
            NM["Node Management"]:::model
            AS["Action Selection"]:::model
            CS["Confidence Scoring"]:::model
            RC["Reward Calculation"]:::model
        end
    end

    %% Data Layer
    subgraph DataLayer
        OCD["Open-O1 CoT Dataset"]:::data
        MCD["Marco-o1 CoT Dataset"]:::data
        MID["Marco Instruction Dataset"]:::data
    end
    click MCD "https://github.com/AIDC-AI/Marco-o1/blob/main/data/CoT_demo.json"

    %% Connections
    ClientLayer --> APILayer
    APILayer --> ProcessingLayer
    ProcessingLayer --> ModelLayer
    ModelLayer --> DataLayer

    %% Detailed Connections
    CI --> FAPI
    SCI --> SFAPI
    FAPI --> MI
    SFAPI --> MI
    MI --> MCTS
    MCTS --> MCTSComponents
    MCTSComponents --> FT
    FT --> BM
    DataLayer --> FT

    %% Legend
    subgraph Legend
        L1["Model Components"]:::model
        L2["Data Components"]:::data
        L3["API Components"]:::api
        L4["Client Components"]:::client
        L5["Processing Components"]:::processing
    end
```
