# tencent/cognitivekernel-pro architecture

> Architecture diagram of the GitHub repository tencent/cognitivekernel-pro, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tencent/cognitivekernel-pro
- Repository: https://github.com/tencent/cognitivekernel-pro

## Overview

##### Project type and purpose
- This is a research-oriented agent framework (Cognitive Kernel-Pro) implemented in Python, with a companion Node.js web-browsing microservice. It orchestrates multiple sub-agents (web agent, file agent, main agent) to perform multi-step tasks via LLM calls, tool invocations, and environment interactions.
- Purpose: provide an open-source, reproducible pipeline for training and running “deep research” agents, including data generation (SFT), evaluation, and real-world web/file operations.

##### File structure
- Top-level packaging:
- ck_pro/agents – core Agent abstractions and utilities.
- ck_pro/ck_file – file-operation agent (reading, conversion, prompts).
- ck_pro/ck_main – main agent orchestration, GAIA scorer, CLI entrypoint, analysis scripts.
- ck_pro/ck_web – web-agent interface and wrapper around Node.js Playwright server.
- A separate data/convert_sft.py script for SFT data post-processing.
- README, Dockerfile under ck_web/_web, package.json, run_local scripts → microservice component.

##### README for additional insights
- Architecture hints: three agent types (main, file, web) each with their own prompts and model calls.
- External dependencies: LLM APIs (OpenAI, vLLM, Claude), search APIs (Google or DuckDuckGo), browser service via Playwright.
- Data flow: tasks → ck_main.main → sub-agent sessions → JSONL outputs → analyze scripts.

##### Main components and their relationships
a. Components:
- CLI/Orchestrator (ck_pro/ck_main/main.py)
- Agent Core Library (ck_pro/agents)
- Main Agent Module (ck_pro/ck_main)
- File Agent Module (ck_pro/ck_file)
- Web Agent Module (ck_pro/ck_web/agent.py)
- Web-Browsing Microservice (Node.js, ck_web/_web/server.js)
- External LLM Services (HTTP endpoints or cloud APIs)
- External Search APIs (Google/DuckDuckGo)
- Local File System (data/, sessions storage)
- Analysis Scripts (ck_pro/ck_main/scripts/analyze.py, data/convert_sft.py)

b. Interactions:
- User invokes CLI → Orchestrator loads configuration → instantiates MainAgent
- MainAgent calls Agent Core → dispatches to FileAgent or WebAgent for each action
- WebAgent sends HTTP requests to Node.js microservice → controls headless browser → returns screenshots/text
- All agents call out to external LLM API endpoints for “think” or “act” steps
- Data flows into Session objects, written as JSONL to disk → analysis scripts consume them
- FileAgent uses utilities for file I/O, format conversion (markdownify, mammoth, pdfminer, etc.)

c. Architectural patterns and design principles:
- Modular “agent” abstraction – interchangeable sub-agents with common interface
- Pipeline of planning and action steps, captured in Session → batch JSONL storage
- Microservice pattern for sandboxed web browsing (Node.js container)
- Dependency injection via configuration dictionary (KwargsInitializable)
- Separation of concerns: core library vs. domain-specific modules (file, web, main)

d. Key technologies/frameworks:
- Python3.12, LangChain, SmolAgents
- Node.js + Playwright (browser automation)
- External HTTP APIs (OpenAI, Claude, vLLM)
- Search APIs (duckduckgo_search, Google API)
- File conversion: pdfminer-six, python-pptx, pydub, SpeechRecognition, etc.

##### Details
- Configuration injection mechanism → how MAIN_ARGS flows into agent constructors.
- Session lifecycle: creation, step loop, final output.
- Optional evaluator LLM for reflection (inference-time evaluation).
- Analysis scripts as separate consumers of the session data.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Python Orchestrator Components
    subgraph "Python Orchestrator"
        CLI["CLI / Orchestrator"]:::python
        Core["Agent Core Library"]:::python
        MainAgent["Main Agent Module"]:::python
        FileAgent["File Agent Module"]:::python
        WebAgent["Web Agent Module"]:::python
    end

    %% Node.js Microservice
    subgraph "Web-Browsing Microservice" 
        Server["server.js"]:::nodejs
        Dockerfile["Dockerfile"]:::nodejs
        PackageJSON["package.json"]:::nodejs
        PackageLock["package-lock.json"]:::nodejs
        RunLocal["run_local.sh / run_local_mac.sh"]:::nodejs
    end

    %% External Services
    LLM["External LLM APIs"]:::external
    SearchAPI["External Search APIs"]:::external

    %% Data Stores and Scripts
    Sessions["Session Storage (JSONL)"]:::data
    FileSystem["Local File System"]:::data
    Analyze["analyze.py"]:::python
    ConvertSMOL["convert_from_smolagent.py"]:::python
    ConvertSFT["convert_sft.py"]:::python
    TestData["simple_test.jsonl"]:::data

    %% Relationships
    CLI -->|"loads config\ninstantiates MainAgent"| MainAgent
    CLI -->|injects MAIN_ARGS| Core
    MainAgent -->|uses core abstractions| Core
    MainAgent -->|dispatches actions to| FileAgent
    MainAgent -->|dispatches actions to| WebAgent
    FileAgent -->|IO & conversion utils| Core
    WebAgent -->|prompts & utils| Core

    WebAgent -->|HTTP requests| Server
    Server -->|controls browser via Playwright| Server

    MainAgent -->|"LLM calls"| LLM
    FileAgent -->|"LLM calls"| LLM
    WebAgent -->|"LLM calls"| LLM

    MainAgent -->|search queries| SearchAPI
    WebAgent -->|search queries| SearchAPI

    Core -->|writes sessions to| Sessions
    Sessions -->|stored on| FileSystem

    Analyze -->|reads sessions| Sessions
    ConvertSMOL -->|reads sessions| Sessions
    ConvertSFT -->|reads raw data| FileSystem

    %% Click Events
    click CLI "https://github.com/tencent/cognitivekernel-pro/blob/main/ck_pro/ck_main/main.py"
    click Core "https://github.com/tencent/cognitivekernel-pro/tree/main/ck_pro/agents/"
    click MainAgent "https://github.com/tencent/cognitivekernel-pro/blob/main/ck_pro/ck_main/agent.py"
    click FileAgent "https://github.com/tencent/cognitivekernel-pro/blob/main/ck_pro/ck_file/agent.py"
    click WebAgent "https://github.com/tencent/cognitivekernel-pro/blob/main/ck_pro/ck_web/agent.py"
    click Server "https://github.com/tencent/cognitivekernel-pro/blob/main/ck_pro/ck_web/_web/server.js"
    click Dockerfile "https://github.com/tencent/cognitivekernel-pro/tree/main/ck_pro/ck_web/_web/Dockerfile"
    click PackageJSON "https://github.com/tencent/cognitivekernel-pro/blob/main/ck_pro/ck_web/_web/package.json"
    click PackageLock "https://github.com/tencent/cognitivekernel-pro/blob/main/ck_pro/ck_web/_web/package-lock.json"
    click RunLocal "https://github.com/tencent/cognitivekernel-pro/blob/main/ck_pro/ck_web/_web/run_local.sh"
    click Analyze "https://github.com/tencent/cognitivekernel-pro/blob/main/ck_pro/ck_main/scripts/analyze.py"
    click ConvertSMOL "https://github.com/tencent/cognitivekernel-pro/blob/main/ck_pro/ck_main/scripts/convert_from_smolagent.py"
    click ConvertSFT "https://github.com/tencent/cognitivekernel-pro/blob/main/data/convert_sft.py"
    click Sessions "https://github.com/tencent/cognitivekernel-pro/blob/main/ck_pro/agents/session.py"
    click TestData "https://github.com/tencent/cognitivekernel-pro/blob/main/ck_pro/ck_main/_test/simple_test.jsonl"
    click FileSystem "https://github.com/tencent/cognitivekernel-pro/tree/main/data/"

    %% Styles
    classDef python fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef nodejs fill:#90EE90,stroke:#333,stroke-width:1px
    classDef external fill:#FFA500,stroke:#333,stroke-width:1px
    classDef data fill:#D3D3D3,stroke:#333,stroke-width:1px
```
