# guilhermevaz8/macrec_v2 architecture

> Architecture diagram of the GitHub repository guilhermevaz8/macrec_v2, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-23). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/guilhermevaz8/macrec_v2
- Repository: https://github.com/guilhermevaz8/macrec_v2

## Overview

##### Project type and purpose
- This is a research‐grade Python framework (“MACRec”) for multi‐agent collaboration in recommender systems, combining LLM‐based agents, reward modeling, and RL training. It supports both batch CLI tasks and a web demo.

##### File structure
- Top‐level entry points:
- main.py – CLI entry, dispatches tasks.
- web_demo.py – Streamlit app for interactive demo.
- Core packages under macrec/:
- macrec/tasks: defines individual Task classes (e.g., EvaluateTask, GenerationTask).
- macrec/systems: orchestrates multi‐agent collaboration strategies (e.g., CollaborationSystem, ReflectionSystem).
- macrec/agents: individual agent roles (Analyst, Searcher, Reflector, Interpreter, Manager).
- macrec/llms: LLM wrappers (OpenAI, open‐source).
- macrec/tools: tool abstractions (e.g., web search, Wikipedia, summarization).
- macrec/rl: RL training support (offline dataset, PPO reward modules).
- macrec/dataset & evaluation: data loaders & metrics.
- macrec/pages: Streamlit page definitions.
- config/: JSON definitions for agents, prompts, systems, tools, training.
- Scripts/: shell wrappers for preprocessing, training, evaluation.

##### README
- Confirms architecture: modular “systems” built from “agents” communicating via Contract Net Protocol.
- Two modes: CLI batch tasks (via main.py + scripts) and Streamlit web demo.

1. User Interfaces
- CLI Interface (main.py + scripts)
- Web UI (web_demo.py + macrec/pages)
2. Task Manager
- macrec/tasks/*.py
- Parses task name, loads config, orchestrates execution.
3. System Orchestrator
- macrec/systems/*.py
- Implements collaboration patterns (analyse, chat, react, reflection).
4. Agents
- macrec/agents/*.py (Analyst, Searcher, Reflector, Interpreter, Manager)
- Each agent loads prompt (config/prompts) and communicates role‐based messages.
5. LLM Layer
- macrec/llms/basellm.py + openai/opensource wrappers
- Sends messages to external LLM APIs or local models.
6. Tools
- macrec/tools/*
- Provide external knowledge (web search, Wikipedia, summarization, info databases).
7. Data Pipeline
- macrec/dataset (load ML‐100k, Amazon)
- config/training & scripts/preprocess.sh
8. RL Training & Evaluation
- macrec/rl (reward shaping, PPO)
- macrec/evaluation (rank_metric, rating_metric)
9. Configuration Store
- config/agents, config/prompts, config/systems, config/tools, config/training
- User → (CLI or Web UI) → Task Manager → System Orchestrator → Agents
- Agents ↔ LLM Layer → external model APIs
- Agents → Tools → external services or local databases
- System Orchestrator → RL Training module (during train mode) → Dataset
- Task Manager ← Evaluation metrics ← System outputs
- All components read from Configuration Store at startup
c. Architectural patterns & design principles:
- Modular, plugin‐style for agents/systems/tools.
- Separation of concerns: UI vs. Task definitions vs. System logic vs. Agent roles vs. LLM/Tool interfaces.
- Data‐driven configuration through JSON.
- Contract Net Protocol for multi‐agent coordination.
- Support for both batch jobs and interactive demo.
d. Key technologies & libraries:
- Python 3.10+, PyTorch for RL and model integration.
- Streamlit for web UI.
- OpenAI SDK or Hugging Face for LLM calls.
- PPO (macrec/rl) for reinforcement learning.

##### Detailed checklist
- All nine components from step 4a.
- Footnotes or annotations for JSON config hierarchy (agents vs. prompts vs. systems).
- Notes indicating optional paths (batch vs. web demo, training vs. inference).
- Versioning or dependencies (Python 3.10+, PyTorch).

## Diagram (Mermaid)

```mermaid
graph TB
    %% UI Layer
    subgraph "UI Layer"
        MainPy["main.py"]:::ui
        ScriptsSh["scripts/"]:::ui
        WebDemo["web_demo.py"]:::ui
        Pages["macrec/pages/"]:::ui
    end

    %% Orchestration Layer
    subgraph "Orchestration Layer"
        TaskMgr["Task Manager"]:::orchestration
        SysOrch["System Orchestrator"]:::orchestration
    end

    %% Agent & Model Layer
    subgraph "Agent & Model Layer"
        Agents["Agents"]:::agent
        BaseLLM["basellm.py"]:::agent
        OpenAIWrapper["openai.py"]:::agent
        OSSWrapper["opensource.py"]:::agent
        Tools["macrec/tools/"]:::agent
    end

    %% Data & Training Layer
    subgraph "Data & Training Layer"
        DataPipe[(Data Pipeline)]:::data
        PreprocessSh["scripts/preprocess.sh"]:::data
        RL["RL Training"]:::data
        Eval[(Evaluation Metrics)]:::data
        Config[(Configuration Store)]:::data
    end

    %% External Services
    OpenAIAPI["OpenAI API"]:::external
    WebSearch["Web Search Service"]:::external

    %% Connections
    MainPy -->|dispatchTask| TaskMgr
    ScriptsSh -->|batchTask| TaskMgr
    WebDemo -->|invokeTask| TaskMgr
    Pages -->|invokeTask| TaskMgr

    TaskMgr -->|orchestrate| SysOrch
    SysOrch -->|invokeAgents| Agents
    Agents -->|LLMRequest| BaseLLM
    Agents -->|LLMRequest| OpenAIWrapper
    Agents -->|LLMRequest| OSSWrapper
    BaseLLM -->|apiCall| OpenAIAPI
    OpenAIWrapper -->|apiCall| OpenAIAPI
    OSSWrapper -->|localModel| OpenAIAPI
    Agents -->|useTool| Tools
    Tools -->|httpRequest| WebSearch

    SysOrch -->|trainLoop| RL
    RL -->|loadData| DataPipe
    PreprocessSh -->|prepareData| DataPipe

    SysOrch -->|outputMetrics| Eval
    Eval -->|sendResults| TaskMgr

    Config -->|loadAtStartup| MainPy
    Config -->|loadAtStartup| WebDemo
    Config -->|loadAtStartup| TaskMgr
    Config -->|loadAtStartup| SysOrch
    Config -->|loadAtStartup| Agents
    Config -->|loadAtStartup| BaseLLM
    Config -->|loadAtStartup| Tools
    Config -->|loadAtStartup| RL

    %% Click Events
    click MainPy "https://github.com/guilhermevaz8/macrec_v2/blob/main/main.py"
    click ScriptsSh "https://github.com/guilhermevaz8/macrec_v2/tree/main/scripts/"
    click WebDemo "https://github.com/guilhermevaz8/macrec_v2/blob/main/web_demo.py"
    click Pages "https://github.com/guilhermevaz8/macrec_v2/tree/main/macrec/pages/"
    click TaskMgr "https://github.com/guilhermevaz8/macrec_v2/tree/main/macrec/tasks/"
    click SysOrch "https://github.com/guilhermevaz8/macrec_v2/tree/main/macrec/systems/"
    click Agents "https://github.com/guilhermevaz8/macrec_v2/tree/main/macrec/agents/"
    click BaseLLM "https://github.com/guilhermevaz8/macrec_v2/blob/main/macrec/llms/basellm.py"
    click OpenAIWrapper "https://github.com/guilhermevaz8/macrec_v2/blob/main/macrec/llms/openai.py"
    click OSSWrapper "https://github.com/guilhermevaz8/macrec_v2/blob/main/macrec/llms/opensource.py"
    click Tools "https://github.com/guilhermevaz8/macrec_v2/tree/main/macrec/tools/"
    click DataPipe "https://github.com/guilhermevaz8/macrec_v2/tree/main/macrec/dataset/"
    click PreprocessSh "https://github.com/guilhermevaz8/macrec_v2/blob/main/scripts/preprocess.sh"
    click RL "https://github.com/guilhermevaz8/macrec_v2/tree/main/macrec/rl/"
    click Eval "https://github.com/guilhermevaz8/macrec_v2/tree/main/macrec/evaluation/"
    click Config "https://github.com/guilhermevaz8/macrec_v2/tree/main/config/"

    %% Styles
    classDef ui fill:#ADD8E6,stroke:#000,stroke-width:1px
    classDef orchestration fill:#90EE90,stroke:#000,stroke-width:1px
    classDef agent fill:#FFA500,stroke:#000,stroke-width:1px
    classDef data fill:#DDA0DD,stroke:#000,stroke-width:1px
    classDef external fill:#DDDDDD,stroke:#000,stroke-width:1px
```
