# anselmjeong/deepresearch architecture

> Architecture diagram of the GitHub repository anselmjeong/deepresearch, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/anselmjeong/deepresearch
- Repository: https://github.com/anselmjeong/deepresearch

## Overview

##### Project type and purpose
- From the file tree, this is a standalone Python-based CLI/toolkit named “deepresearch.”
- It appears to automate literature ingestion and processing: PDF→markdown conversion, PubMed lookups (via DOI, BibTeX), citation “snowballing,” and stateful UI components for reviewing results.

##### File structure
- Top-level:
- deepresearch/             ← main package
- config.toml, rxconfig.py  ← application configuration
- pyproject.toml, requirements.txt ← build/dependency specs
- Inside deepresearch/:
- deepresearch.py          ← CLI entrypoint / orchestrator
- state.py                 ← application state management
- components/              ← UI rendering (Left/Middle/Right column)
- lib/                     ← core services & utilities
– pdf_process.py         ← PDF text extraction
– pdf_to_md/             ← text→markdown conversion
– process_directory.py   ← bulk directory traversal
– pubmedder/             ← PubMed API integration (DOI finder, BibTeX parser, etc.)
– researcher.py          ← high-level research workflows
– snowballer/            ← citation-expansion logic

##### Implicit architecture
- Orchestrator (deepresearch.py) initializes state, reads config, then invokes researcher workflows.
- researcher.py coordinates these sub-services: PDF ingestion → conversion → metadata lookup → snowballing.
- UI components subscribe to state changes and render columns of information.
- Config and state modules decouple environment settings from business logic.
- External systems: file system, PubMed API, local storage for markdown output.

a. Main components:
1) CLI Orchestrator
2) Configuration & State Manager
3) PDF Processor
4) Markdown Converter
5) Directory Scanner
6) PubMed API Client (pubmedder)
7) Citation Snowballer
8) UI Renderer (Left/Middle/Right columns)
9) File System / Assets / Output Store
b. Relationships & data flow:
- CLI → Config/State → Directory Scanner → PDF Processor → Markdown Converter → File System
- Markdown Converter → Researcher → PubMed API Client → Citation Snowballer → State
- State → UI Renderer → Terminal/Console
c. Architectural patterns & principles:
- Layered architecture: presentation (UI) / orchestration / services / infrastructure
- Single-responsibility: each lib/ submodule has one domain focus
- Dependency inversion: orchestrator depends on abstract service interfaces defined in lib/
- Extensibility: new components can be dropped into components/ or lib/ subpackages
d. Key technologies & libraries:
- Python 3.x (pyproject.toml)
- click or argparse (implied for CLI)
- pdfminer or PyPDF2 (in pdf_process.py)
- Markdown libraries (in pdf_to_md/fulltexter.py)
- Requests or Entrez API client (in pubmedder/*.py)

- Step 2: Beneath it, place Config/State Manager and Directory Scanner side by side.
- Step 3: Chain to PDF Processor → Markdown Converter in a vertical flow.
- Step 4: Branch from Markdown Converter to File System for output storage.

- The sequential data-processing pipeline (PDF → MD → storage)
- The research enrichment loop (metadata fetch & snowballing)
- The separation of concerns (orchestration vs. services vs. UI)
- Extension and integration points for future enhancements.

## Diagram (Mermaid)

```mermaid
flowchart TD
    CLI["CLI Orchestrator"]:::orchestration
    Config["Configuration Manager"]:::orchestration
    State["State Manager"]:::orchestration
    Scan["Directory Scanner"]:::core
    PDF["PDF Processor"]:::core
    MD["Markdown Converter"]:::core
    FS["File System / Assets"]:::io
    Researcher["Research Workflow Orchestrator"]:::orchestration

    subgraph External PubMed Service
        Pubmed["PubMed API Client"]:::external
    end
    DOI["DOI Finder"]:::external
    Bibtex["BibTeX Parser"]:::external
    CiteRef["Citation Ref Parser"]:::external
    DB["PubMed Local DB Interface"]:::external

    Snowball["Citation Snowballer"]:::core

    UILeft["UI Renderer - Left Column"]:::ui
    UIMid["UI Renderer - Middle Column"]:::ui
    UIRight["UI Renderer - Right Column"]:::ui

    CLI -->|initializes→| Config
    CLI -->|initializes→| State
    CLI -->|traverses→| Scan
    Scan -->|reads files→| PDF
    PDF -->|parses bytes→| MD
    MD -->|outputs MD text→| FS

    CLI -->|orchestrates→| Researcher
    Researcher -->|fetches metadata→| Pubmed
    Pubmed -->|uses→| DOI
    Pubmed -->|uses→| Bibtex
    Pubmed -->|uses→| CiteRef
    Pubmed -->|uses→| DB
    DOI -->|provides IDs→| Snowball
    Bibtex -->|parses entries→| Snowball
    CiteRef -->|extracts refs→| Snowball
    DB -->|stores/cache→| Snowball
    Snowball -->|updates→| State

    State -->|renders→| UILeft
    State -->|renders→| UIMid
    State -->|renders→| UIRight

    click CLI "https://github.com/anselmjeong/deepresearch/blob/main/deepresearch/deepresearch.py"
    click Config "https://github.com/anselmjeong/deepresearch/blob/main/deepresearch/lib/config.py"
    click State "https://github.com/anselmjeong/deepresearch/blob/main/deepresearch/state.py"
    click Scan "https://github.com/anselmjeong/deepresearch/blob/main/deepresearch/lib/process_directory.py"
    click PDF "https://github.com/anselmjeong/deepresearch/blob/main/deepresearch/lib/pdf_process.py"
    click MD "https://github.com/anselmjeong/deepresearch/blob/main/deepresearch/lib/pdf_to_md/fulltexter.py"
    click FS "https://github.com/anselmjeong/deepresearch/tree/main/assets/"
    click Researcher "https://github.com/anselmjeong/deepresearch/blob/main/deepresearch/lib/researcher.py"
    click Pubmed "https://github.com/anselmjeong/deepresearch/blob/main/deepresearch/lib/pubmedder/pubmedder.py"
    click DOI "https://github.com/anselmjeong/deepresearch/blob/main/deepresearch/lib/pubmedder/doi_finder.py"
    click Bibtex "https://github.com/anselmjeong/deepresearch/blob/main/deepresearch/lib/pubmedder/bibtex_parser.py"
    click CiteRef "https://github.com/anselmjeong/deepresearch/blob/main/deepresearch/lib/pubmedder/cite_ref.py"
    click DB "https://github.com/anselmjeong/deepresearch/blob/main/deepresearch/lib/pubmedder/db.py"
    click Snowball "https://github.com/anselmjeong/deepresearch/blob/main/deepresearch/lib/snowballer/snowballer.py"
    click UILeft "https://github.com/anselmjeong/deepresearch/blob/main/deepresearch/components/LeftColumn.py"
    click UIMid "https://github.com/anselmjeong/deepresearch/blob/main/deepresearch/components/MiddleColumn.py"
    click UIRight "https://github.com/anselmjeong/deepresearch/blob/main/deepresearch/components/RightColumn.py"

    classDef orchestration fill:#ADD8E6,stroke:#000,stroke-width:1px
    classDef core fill:#90EE90,stroke:#000,stroke-width:1px
    classDef io fill:#FFA500,stroke:#000,stroke-width:1px
    classDef ui fill:#DDA0DD,stroke:#000,stroke-width:1px
    classDef external fill:#FFDAB9,stroke:#000,stroke-width:1px
```
