# zjunlp/omnithink architecture

> Architecture diagram of the GitHub repository zjunlp/omnithink, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/zjunlp/omnithink
- Repository: https://github.com/zjunlp/omnithink

## Overview

##### Project Type and Purpose
- OmniThink is a machine writing framework that generates and polishes articles by performing tasks like outline generation, article drafting, and refinement. It also contains evaluation scripts that assess aspects such as Knowledge Density, Information Diversity, and Rubric Grading.

##### File Structure to Identify Main Components
- Top-Level Files and Scripts:
- run.sh: Looks like the main entry point that kicks off the article generation process.
- Dockerfile, environment.yml, requirements.txt: Indicate how the application is containerized and its dependencies are managed.
- Core Source Code (src directory):
- src/actions: Contains main functionality modules (article_generation.py, article_polish.py, outline_generation.py) which represent the primary behavioral components. These form the core processing pipeline.
- src/dataclass: Contains the data models including Article definitions and possibly interfaces used across modules.
- src/tools: Contains helper modules such as 'lm.py' and 'rm.py' that are in charge of interfacing with Language Model APIs and Search APIs, key for generating and refining content.
- src/utils: Provides utility functions related to text-processing, file I/O, web page handling, etc.
- Evaluation Module (eval directory): Independent scripts for evaluating generated content (Information_Diversity, Knowledge_Density, Rubric_Grading) which might be connected to third-party libraries (factscore).

##### Interactions and Relationships
- The run.sh entry script likely initializes the core workflow by invoking modules from src/actions.
- The src/actions modules call on utilities (in src/utils) for file management and text processing, and may rely on data models (in src/dataclass) for standardizing article representations.
- The tools (LM and search modules) connect with external APIs, with configuration provided via environment variables (LM_KEY, SEARCHKEY). This connection should be explicitly shown as interactions with an external service.
- Evaluation components in the eval folder are run either as separate processes or integrated pipelines, and their dependencies (like the factscore library from a cloned repo) should be highlighted.

##### Architectural Patterns and Technologies
- The design follows a modular, layered pattern:
- The Actions layer (for processing tasks) orchestrates operations.
- The Dataclass layer provides unified data models.
- The Tools and Utils layers encapsulate helper services and external integrations.
- How the iterative feedback is built into the system, with tightly linked generation, polishing, and outline phases.
- That the deployment supports containerization (Docker) and environment management (Conda), which could be represented as auxiliary components managing deployment and configuration.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Deployment and Configuration
    subgraph "Deployment / Configuration"
        RunEntry("run.sh (Entry Point)"):::deploy
        Dockerfile("Dockerfile"):::deploy
        Environment("environment.yml"):::deploy
        Requirements("requirements.txt"):::deploy
    end

    %% Core Processing Pipeline
    subgraph "Core Processing Pipeline"
        %% Actions Layer
        subgraph "Actions Layer"
            Outline("Outline Generation"):::actions
            ArticleGen("Article Generation"):::actions
            ArticlePolish("Article Polishing"):::actions
        end
        %% Data Models Layer
        subgraph "Data Models"
            ArticleModel("Article Model"):::dataclass
            Interface("Interface"):::dataclass
        end
        Utils("Utility Functions"):::utils
    end

    %% External Integrations
    subgraph "External Integrations"
        LM("LM API Connector"):::external
        Search("Search API Connector"):::external
    end

    %% Evaluation Suite
    subgraph "Evaluation Suite"
        InfoDiv("Information Diversity"):::evaluation
        KnowDen("Knowledge Density"):::evaluation
        Rubric("Rubric Grading"):::evaluation
    end

    %% Connections from Deployment to Core Pipeline
    RunEntry -->|"initiates"| Outline
    RunEntry -->|"initiates"| ArticleGen
    RunEntry -->|"initiates"| ArticlePolish

    %% Iterative and sequential flow within Actions Layer
    Outline -->|"produces outline for"| ArticleGen
    ArticleGen -->|"drafts content for"| ArticlePolish
    ArticlePolish -->|"iterative feedback"| ArticleGen

    %% Dependencies on Utility Functions and Data Models
    ArticleGen ---|"uses"| Utils
    ArticlePolish ---|"uses"| Utils
    Outline ---|"relies on"| ArticleModel
    ArticleGen ---|"relies on"| ArticleModel
    ArticlePolish ---|"relies on"| ArticleModel
    Outline ---|"relies on"| Interface
    ArticleGen ---|"relies on"| Interface
    ArticlePolish ---|"relies on"| Interface

    %% External API Integrations
    ArticleGen -->|"calls"| LM
    ArticlePolish -->|"calls"| Search

    %% Evaluation Flow
    ArticlePolish -->|"evaluated by"| InfoDiv
    ArticlePolish -->|"evaluated by"| KnowDen
    ArticlePolish -->|"evaluated by"| Rubric

    %% Deployment artifacts association
    RunEntry --- Dockerfile
    RunEntry --- Environment
    RunEntry --- Requirements

    %% Click Events for Component Mapping
    click RunEntry "https://github.com/zjunlp/omnithink/blob/main/run.sh"
    click Dockerfile "https://github.com/zjunlp/omnithink/tree/main/Dockerfile"
    click Environment "https://github.com/zjunlp/omnithink/blob/main/environment.yml"
    click Requirements "https://github.com/zjunlp/omnithink/blob/main/requirements.txt"
    click Outline "https://github.com/zjunlp/omnithink/blob/main/src/actions/outline_generation.py"
    click ArticleGen "https://github.com/zjunlp/omnithink/blob/main/src/actions/article_generation.py"
    click ArticlePolish "https://github.com/zjunlp/omnithink/blob/main/src/actions/article_polish.py"
    click ArticleModel "https://github.com/zjunlp/omnithink/blob/main/src/dataclass/Article.py"
    click Interface "https://github.com/zjunlp/omnithink/blob/main/src/dataclass/interface.py"
    click LM "https://github.com/zjunlp/omnithink/blob/main/src/tools/lm.py"
    click Search "https://github.com/zjunlp/omnithink/blob/main/src/tools/rm.py"
    click Utils "https://github.com/zjunlp/omnithink/tree/main/src/utils"
    click InfoDiv "https://github.com/zjunlp/omnithink/blob/main/eval/Information_Diversity.py"
    click KnowDen "https://github.com/zjunlp/omnithink/blob/main/eval/Knowledge_Density.py"
    click Rubric "https://github.com/zjunlp/omnithink/blob/main/eval/Rubric_Grading.py"

    %% Styles
    classDef deploy fill:#F4CCCC,stroke:#CC0000,stroke-width:2px;
    classDef actions fill:#C9DAF8,stroke:#1155CC,stroke-width:2px;
    classDef dataclass fill:#D9EAD3,stroke:#38761D,stroke-width:2px;
    classDef utils fill:#FFF2CC,stroke:#BF9000,stroke-width:2px;
    classDef external fill:#FCE5CD,stroke:#E69138,stroke-width:2px;
    classDef evaluation fill:#D9D2E9,stroke:#674EA7,stroke-width:2px;
```
