How tencent/cognitivekernel-pro works
Project type and purpose
• This is a research-oriented agent framework (Cognitive Kernel-Pro) implemented in Python, with a companion Node.js web-browsing microservice. It orchestrates multiple sub-agents (web agent, file agent, main agent) to perform multi-step tasks via LLM calls, tool invocations, and environment interactions.
• Purpose: provide an open-source, reproducible pipeline for training and running “deep research” agents, including data generation (SFT), evaluation, and real-world web/file operations.
File structure
• Top-level packaging:
• ck_pro/agents – core Agent abstractions and utilities.
• ck_pro/ck_file – file-operation agent (reading, conversion, prompts).
• ck_pro/ck_main – main agent orchestration, GAIA scorer, CLI entrypoint, analysis scripts.
• ck_pro/ck_web – web-agent interface and wrapper around Node.js Playwright server.
• A separate data/convert_sft.py script for SFT data post-processing.
• README, Dockerfile under ck_web/_web, package.json, run_local scripts → microservice component.
README for additional insights
• Architecture hints: three agent types (main, file, web) each with their own prompts and model calls.
• External dependencies: LLM APIs (OpenAI, vLLM, Claude), search APIs (Google or DuckDuckGo), browser service via Playwright.
• Data flow: tasks → ck_main.main → sub-agent sessions → JSONL outputs → analyze scripts.
Main components and their relationships
a. Components:
• CLI/Orchestrator (ck_pro/ck_main/main.py)
• Agent Core Library (ck_pro/agents)
• Main Agent Module (ck_pro/ck_main)
• File Agent Module (ck_pro/ck_file)
• Web Agent Module (ck_pro/ck_web/agent.py)
• Web-Browsing Microservice (Node.js, ck_web/_web/server.js)
• External LLM Services (HTTP endpoints or cloud APIs)
• External Search APIs (Google/DuckDuckGo)
• Local File System (data/, sessions storage)
• Analysis Scripts (ck_pro/ck_main/scripts/analyze.py, data/convert_sft.py)
b. Interactions:
• User invokes CLI → Orchestrator loads configuration → instantiates MainAgent
• MainAgent calls Agent Core → dispatches to FileAgent or WebAgent for each action
• WebAgent sends HTTP requests to Node.js microservice → controls headless browser → returns screenshots/text
• All agents call out to external LLM API endpoints for “think” or “act” steps
• Data flows into Session objects, written as JSONL to disk → analysis scripts consume them
• FileAgent uses utilities for file I/O, format conversion (markdownify, mammoth, pdfminer, etc.)
c. Architectural patterns and design principles:
• Modular “agent” abstraction – interchangeable sub-agents with common interface
• Pipeline of planning and action steps, captured in Session → batch JSONL storage
• Microservice pattern for sandboxed web browsing (Node.js container)
• Dependency injection via configuration dictionary (KwargsInitializable)
• Separation of concerns: core library vs. domain-specific modules (file, web, main)
d. Key technologies/frameworks:
• Python3.12, LangChain, SmolAgents
• Node.js + Playwright (browser automation)
• External HTTP APIs (OpenAI, Claude, vLLM)
• Search APIs (duckduckgo_search, Google API)
• File conversion: pdfminer-six, python-pptx, pydub, SpeechRecognition, etc.
Details
• Configuration injection mechanism → how MAIN_ARGS flows into agent constructors.
• Session lifecycle: creation, step loop, final output.
• Optional evaluator LLM for reflection (inference-time evaluation).
• Analysis scripts as separate consumers of the session data.