# betasecond/hoshi architecture

> Architecture diagram of the GitHub repository betasecond/hoshi, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/betasecond/hoshi
- Repository: https://github.com/betasecond/hoshi

## Overview

1. mofa_graph_rag: This is the backend ML/data pipeline component which implements a lightRAG system that leverages knowledge graphs to enhance Retrieval Augmented Generation (RAG). It’s primarily written in Python and organized around multiple configuration files, YAML dataflow definitions, and a set of scripts. This component interacts with external LLM APIs (using models such as 通义千问-qwen-turbo for dialogs and 通义千问-text-embedding-v2 for embeddings) and handles data preparation, question generation, RAG index management, and logging.

2. mofas-hoshi: This is the web developer kit (under kit/mofas-ui-master) that offers a complete full-stack frontend and backend solution. The frontend is built with Vue 3, TypeScript, and related modern frameworks (Element Plus, Vue Router, Pinia with Vite as the build tool), and the backend is an Express.js application in TypeScript (compiled into JavaScript), using RESTful API design, Pug templating for views, and integration for Python-based CLI tools.

##### Project Components
- mofa_graph_rag Data Pipeline: Mark this as the ML and data orchestration engine. Include submodules such as:
- Data Preparation and Question Generation operators (e.g., question_gen_operator.py, data_prep_operator.py)
- Knowledge Graph integration modules (under scripts/lightrag/kg, which include implementations based on Chroma, Faiss, Redis, Neo4j, etc.)
- LLM integration components (modules under scripts/lightrag/llm: openai, azure_openai, etc.)
- Configuration management (YAML, config files under mofa_graph_rag/configs).
- Execution orchestration using the Mofa/Dora environment and the graph_rag_dataflow.yml definition.

- mofas-hoshi Web Suite: This full-stack part has a clear frontend and backend:
- Frontend: Built using Vue 3 with TypeScript, including components, views, assets, and routing (look inside frontend/src for components like AgentEvaluation, AgentList, etc.).
- Backend: An Express.js server (using app.ts, server.js, bin/www.ts) that serves both API endpoints and rendered views (Pug templates in views directory), integrated with CLI tool invocations and possibly delegating some calls to Python scripts.

- External and Supporting Services:
- External LLM and embedding APIs (as configured in the mofa_graph_rag configs).
- MoFA and DORA environments that need to be set up as per the instructions.
- Optional database or storage components (implied by implementations like various KG connectors and shared_storage in the lightRAG scripts).

##### Depicting Component Relationships and Data Flow
- The Web UI (frontend) communicates with the Backend API (Express server) via REST calls.
- The Backend serves static files, renders Pug templates, and may trigger operations that invoke Python modules (e.g., calling CLI tools or orchestrating RAG tasks).
- The mofa_graph_rag module runs as an independent data flow process triggered via commands (using dora up/build/start commands) and logs outputs to the mofa_graph_rag/out directory.
- There is an integration point where user requests, agent management, or evaluations (as mentioned in the README “代理评估系统” and “代理管理”) may invoke functionalities in the mofa_graph_rag pipeline.
- The knowledge graph components and LLM components within the lightRAG code have several implementations. These should be grouped and connected to indicate that they are interchangeable strategies for handling embedding, similarity search, or API calls.

##### Important Architectural Patterns and Technologies
- The project follows a modular design—mofa_graph_rag is dedicated to the heavy lifting of data preparation and ML inference, while mofas-hoshi is the user-facing full-stack app.
- For the web part, note the separation of concerns (frontend vs. backend). The frontend adheres to a component-driven structure (Vue components, router, and stores), and the backend uses the MVC-like pattern (Express routes, controllers, Pug templating).
- In the ML/data module, the pipeline abstraction (via YAML configuration and modular scripts in Python) is a key point. Emphasize how different knowledge graph implementations are abstracted via a common interface, allowing flexibility in choosing vector databases or KG backends.
- Integration with external APIs (LLM and embedding models) is another crosscutting concern.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% mofa_graph_rag Data Pipeline Cluster
    subgraph "mofa_graph_rag Data Pipeline"
        O["Orchestration & Config (YAML & configs)"]:::pipeline
        subgraph "KG Branch"
            DPO["Data Preparation Operator"]:::pipeline
            KGM["KG Modules"]:::pipeline
            RIO["RAG Index Operator"]:::pipeline
        end
        subgraph "LLM Branch"
            QGO["Question Generation Operator"]:::pipeline
            LLMM["LLM Modules"]:::pipeline
            RQO["RAG Query Operator"]:::pipeline
        end
        RL["Result Logger Operator"]:::pipeline
        O --> DPO
        O --> QGO
        DPO --> KGM
        KGM --> RIO
        QGO --> LLMM
        LLMM --> RQO
        RIO --> RL
        RQO --> RL
    end

    %% mofas-hoshi Web Suite Cluster
    subgraph "mofas-hoshi Web Suite"
        subgraph "Frontend Layer"
            FE["Web Frontend (Vue 3, Vite, TypeScript)"]:::web
        end
        subgraph "Backend Layer"
            BE["Web Backend (Express, Node.js, TypeScript)"]:::web
            CLI["CLI Tools for Integration"]:::web
        end
    end

    %% Interactions between Web Suite and Data Pipeline
    FE -->|"REST_API"| BE
    BE -->|"CLI_call"| CLI
    BE -->|"Triggers"| O

    %% Click Events for mofa_graph_rag Data Pipeline
    click O "https://github.com/betasecond/hoshi/blob/master/mofa_graph_rag/graph_rag_dataflow.yml"
    click DPO "https://github.com/betasecond/hoshi/blob/master/mofa_graph_rag/scripts/data_prep_operator.py"
    click QGO "https://github.com/betasecond/hoshi/blob/master/mofa_graph_rag/scripts/question_gen_operator.py"
    click RIO "https://github.com/betasecond/hoshi/blob/master/mofa_graph_rag/scripts/rag_index_operator.py"
    click RQO "https://github.com/betasecond/hoshi/blob/master/mofa_graph_rag/scripts/rag_query_operator.py"
    click RL "https://github.com/betasecond/hoshi/blob/master/mofa_graph_rag/scripts/result_logger_operator.py"
    click KGM "https://github.com/betasecond/hoshi/tree/master/mofa_graph_rag/scripts/lightrag/kg/"
    click LLMM "https://github.com/betasecond/hoshi/tree/master/mofa_graph_rag/scripts/lightrag/llm/"

    %% Click Events for mofas-hoshi Web Suite
    click BE "https://github.com/betasecond/hoshi/tree/master/kit/mofas-ui-master/backend/"
    click FE "https://github.com/betasecond/hoshi/tree/master/kit/mofas-ui-master/frontend/src/"
    click CLI "https://github.com/betasecond/hoshi/tree/master/kit/mofas-ui-master/target/"

    %% Styling
    classDef pipeline fill:#FCE4EC,stroke:#C2185B,stroke-width:2px;
    classDef web fill:#E8F5E9,stroke:#388E3C,stroke-width:2px;
```
