How duy0699cat/worldquant-brain-mcp works
1. What this repo is
- worldquant-mcp is a local Python MCP server for agent-driven WorldQuant BRAIN alpha research.
- It sits between an MCP-aware client/agent and the external WorldQuant BRAIN API, while also hosting a local reference-ingestion/search pipeline for docs and notes.
2. Top-level architecture
- The project has two main subsystems:
- Online orchestration: auth, operator lookup, simulation, validation, submission against BRAIN.
- Offline knowledge pipeline: ingest PDFs/notes into searchable markdown/chunks and use that material for research assistance.
- The boundary is important: the repo does not contain proprietary platform documentation; it expects you to provide your own source PDFs.
3. Primary runtime entrypoint
- src/worldquant_mcp/__main__.py and server.py are the server entry points.
- The MCP server is intended to run over stdio and be launched by VS Code / an MCP client, not exposed as a general web service.
4. MCP tool surface
- src/worldquant_mcp/tools/ defines the functional API presented to agents.
- The tool groups map cleanly to workflow stages:
- Health/account/platform discovery: healthcheck, get_account, get_platform_operators, get_data_fields
- Reference lookup: search_local_reference, get_reference_operators
- Alpha lifecycle: create_simulation, batch_simulate, get_simulation, wait_simulation, get_alpha, get_alpha_submission
- Research/analysis: map_research_hypothesis, ingest_research_material, mutate_alpha_expression, analyze_alpha_novelty
- Governance: validate_alpha_submission_by_id, submit_alpha_by_id
- Local memory: log_alpha_experiment, list_alpha_experiments
5. External service boundary
- src/worldquant_mcp/client.py is the main integration point to api.worldquantbrain.com.
- It likely centralizes authentication, retries/rate-limit handling, polling, and request shaping so the tools stay thin and consistent.
- The README explicitly notes bounded waits and backoff behavior for simulations and rate limits, so the client is a key reliability boundary.
6. Configuration and secrets
- src/worldquant_mcp/config.py loads credentials and tunables from .env or WQ_BRAIN_* environment variables.
- This keeps secrets and timeout/rate-limit policy out of the tool layer and makes the server portable across local environments.
7. Expression and domain modeling
- src/worldquant_mcp/expression_ast.py and models.py define the repository’s internal representation of BRAIN expressions and API payloads.
- These modules are the bridge between plain-text hypotheses/expressions and structured operations like mutation, validation, and submission.
- This is where the repo likely normalizes alpha metadata, simulation results, and validation status into typed objects.
8. Research workflow layer
- src/worldquant_mcp/tools/research.py appears to connect user hypotheses and external research notes to concrete alpha ideas.
- It uses local reference material plus experiment memory to generate summaries, field proxies, and expression templates.
- This layer is the “reasoning” surface: it doesn’t just call the API, it helps transform research intent into testable expressions.
9. Simulation and submission pipeline
- The core operational flow is:
- generate or mutate an expression,
- submit it to simulation,
- wait/poll until completion,
- inspect alpha metrics,
- validate readiness,
- optionally submit the alpha.
- tools/lab.py and tools/submission.py likely own the mechanics of this lifecycle, while tools/reference.py supports discovery and tools/research.py supports synthesis.
10. Local reference ingestion and search
- src/worldquant_mcp/reference/ is a standalone document-processing subsystem.
- It converts source PDFs into:
- cleaned markdown,
- chunked JSON for retrieval,
- indexed/searchable artifacts.
- clean.py, extract.py, and index.py suggest a pipeline: extract text/OCR → normalize → chunk/index → query via MCP tools.
11. Persistent local memory
- src/worldquant_mcp/data/experiment_memory.json stores prior experiments in machine-readable form.
- This supports longitudinal research: the server can compare a candidate expression against historical experiments and recent alphas before submission.
- The repo therefore has a small local state layer in addition to transient API calls.
12. Supporting scripts and experiments
- The root scripts (run_*_sweep.py, probe_api.py, poll_self_corr.py, script.py) look like ad hoc research harnesses for batch experimentation and API probing.
- simplified_alpha/ is a separate lightweight example workflow with its own requirements and watchlist, useful as a sandbox or minimal reproduction path.
13. Documentation and process
- docs/ contains roadmap and research notes rather than product docs, reinforcing that this is a living research-engineering codebase.
- .github/workflows/validate.yml suggests CI validation, while .github/agents/ contains agent-specific guidance for workflow automation.
14. How the pieces fit together
- The main diagram should show:
- MCP client/agent → stdio MCP server → tool modules
- Tool modules → local reference store / experiment memory
- Tool modules → WorldQuant BRAIN API via shared client
- Reference ingestion pipeline → generated local markdown/chunks
- The key architectural distinction is between local knowledge management and remote alpha execution.
Main entry points: tools Tool Surface · research.py Research · client.py BRAIN Client · lab.py Alpha Lab
Components 21
MCP Runtime
Local stdio server entrypoints and tool dispatch.
- MCP Client · External MCP-aware agent or editor that drives the server.
- Server Entry · __main__.py and server.py launch the local MCP server.
src/worldquant_mcp/server.py - Tool Surface · Public function set exposed to agents.
src/worldquant_mcp/tools - Discovery · Account, platform, field, and health lookup tools.
src/worldquant_mcp/tools - Research · Hypothesis mapping, mutation, novelty analysis, and local ingestion tools.
src/worldquant_mcp/tools/research.py - Config · Loads credentials and tunables from environment or .env.
src/worldquant_mcp/config.py
BRAIN Orchestration
Remote alpha lifecycle, validation, and API integration.
- Alpha Lab · Simulation lifecycle tools for creating, polling, and reading alphas.
src/worldquant_mcp/tools/lab.py - Governance · Validation and submission tools for alpha readiness.
src/worldquant_mcp/tools/submission.py - BRAIN Client · Shared integration with api.worldquantbrain.com, including auth, retries, polling, and rate limits.
src/worldquant_mcp/client.py - Expression AST · Internal alpha expression representation and transformations.
src/worldquant_mcp/expression_ast.py - Domain Models · Structured API payloads, alpha metadata, and results.
src/worldquant_mcp/models.py
Knowledge Pipeline
Local reference ingestion, chunking, and search for research support.
- Reference Lookup · Search and operator reference tools for local and platform knowledge.
src/worldquant_mcp/tools/reference.py - Reference Pipeline · Offline pipeline that turns source documents into searchable local artifacts.
src/worldquant_mcp/reference - Extract/Clean · Text extraction, OCR normalization, and markdown cleanup.
src/worldquant_mcp/reference/extract.py - Indexing · Chunking and indexing for retrieval.
src/worldquant_mcp/reference/index.py
Local State
Persistent experiment memory and generated reference artifacts.
- Generated Docs · Cleaned markdown, chunks, and indexed outputs derived from source PDFs.
src/worldquant_mcp/reference/data - Experiment Memory · Persistent history of prior alpha experiments.
src/worldquant_mcp/data/experiment_memory.json
Supporting Workflows
Ad hoc scripts, sandbox workflows, docs, and CI guidance.
- Docs & Notes · Roadmap and research notes that guide the local workflow.
docs - Research Scripts · Sweep, probe, and polling scripts for experimentation.
run_b1_sweep.py - Sandbox Alpha · Minimal standalone alpha example and requirements.
simplified_alpha - CI & Agents · Validation workflow and agent instructions.
.github/workflows/validate.yml
Connections 26
- MCP Client → Server Entry: launches · An MCP-aware client starts the local stdio server.
- Server Entry → Tool Surface: dispatches · The server exposes the tool API to the client.
- Tool Surface → Discovery: includes · Health and discovery capabilities.
- Tool Surface → Reference Lookup: includes · Reference and operator lookup capabilities.
- Tool Surface → Research: includes · Research synthesis and local ingestion capabilities.
- Tool Surface → Alpha Lab: includes · Simulation and alpha lifecycle capabilities.
- Tool Surface → Governance: includes · Validation and submission capabilities.
- Alpha Lab → BRAIN Client: uses · Simulation and alpha retrieval flow through the shared API client.
- Governance → BRAIN Client: uses · Validation and submission go through the shared API client.
- Research → Experiment Memory: reads/writes · Research tools persist and compare prior experiments.
- Research → Generated Docs: queries · Research tools search local reference artifacts.
- Reference Lookup → Generated Docs: reads · Local reference search operates on generated artifacts.
- Reference Pipeline → Extract/Clean: stages · Document processing begins with extraction and cleanup.
- Reference Pipeline → Indexing: stages · Cleaned content is chunked and indexed for retrieval.
- Extract/Clean → Generated Docs: produces · Extraction and cleanup generate markdown artifacts.
- Indexing → Generated Docs: produces · Indexing generates chunk and search artifacts.
- BRAIN Client → Config: configured by · Credentials, timeouts, and rate-limit policy come from configuration.
- Research → Expression AST: builds · Hypotheses and mutations are converted into expression structure.
- Research → Domain Models: formats · Research helpers use typed domain models for summaries and payloads.
- Alpha Lab → Expression AST: submits · Expressions are prepared for simulation.
- Alpha Lab → Domain Models: returns · Simulation state and alpha results are represented as typed models.
- Governance → Domain Models: validates · Submission readiness is checked against structured alpha data.
- Docs & Notes → Research: guides · Roadmap and research notes inform the research workflow.
- Research Scripts → BRAIN Client: probes · Standalone scripts exercise the remote API and polling behavior.
- Sandbox Alpha → BRAIN Client: demonstrates · The sandbox provides a smaller alpha workflow built around the same backend.
- CI & Agents → Server Entry: supports · Validation workflow and agent guidance support repository automation.