# yrl-aida/docs_extractor architecture

> Architecture diagram of the GitHub repository yrl-aida/docs_extractor, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/yrl-aida/docs_extractor
- Repository: https://github.com/yrl-aida/docs_extractor

## Overview

This is a scientific-document extraction system: a user supplies a PDF, the demo orchestrates page/content parsing, article-field extraction and enrichment, then writes structured JSON. The README documents sections, references, figures and tables as output capabilities. Extractor source is partial, so the graph leaves internal substeps unspecified and does not assume wiring beyond the visible calls and documented integrations.

Main entry points: [`extractor/extractor.py`](https://github.com/yrl-aida/docs_extractor/blob/master/extractor/extractor.py) (Article extractor), [`demo.py`](https://github.com/yrl-aida/docs_extractor/blob/master/demo.py) (Demo runner), [`extractor/mineru_compact.py`](https://github.com/yrl-aida/docs_extractor/blob/master/extractor/mineru_compact.py) (MinerU parser), [`extractor/llm_extractor.py`](https://github.com/yrl-aida/docs_extractor/blob/master/extractor/llm_extractor.py) (LLM extractor)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_parsing["PDF parsing"]
  node_demo["Demo runner<br/>[demo.py]"]
  node_mineru["MinerU parser<br/>[mineru_compact.py]"]
  node_model["Local model"]
end

subgraph group_extraction["Article extraction"]
  node_article["Article extractor<br/>[extractor.py]"]
end

subgraph group_integrations["Enrichment services"]
  node_llm["LLM extractor<br/>[llm_extractor.py]"]
  node_prompts["Extraction prompts<br/>[prompts.yaml]"]
end

subgraph group_output["Structured output"]
  node_json["Article JSON"]
end

node_user(("PDF submitter"))
node_pdf["Input PDF"]
node_content["Parsed content"]
node_ollama["LLM service"]
node_grobid["GROBID service"]

node_user -->|"starts extraction"| node_demo
node_user -->|"provides PDF"| node_pdf
node_demo -->|"calls parse_doc"| node_mineru
node_mineru -->|"reads"| node_pdf
node_mineru -->|"uses model"| node_model
node_mineru -->|"returns content list"| node_content
node_demo -->|"calls extraction"| node_article
node_content -->|"supplies parsed content"| node_article
node_article -->|"requests extraction"| node_llm
node_llm -->|"reads prompts"| node_prompts
node_llm -->|"sends prompt"| node_ollama
node_article -.->|"requests author data"| node_grobid
node_demo -->|"calls JSON dump"| node_article
node_article -->|"writes result"| node_json

click node_demo "https://github.com/yrl-aida/docs_extractor/blob/master/demo.py"
click node_mineru "https://github.com/yrl-aida/docs_extractor/blob/master/extractor/mineru_compact.py"
click node_model "https://github.com/yrl-aida/docs_extractor/tree/master/model"
click node_article "https://github.com/yrl-aida/docs_extractor/blob/master/extractor/extractor.py"
click node_llm "https://github.com/yrl-aida/docs_extractor/blob/master/extractor/llm_extractor.py"
click node_prompts "https://github.com/yrl-aida/docs_extractor/blob/master/data/prompts/prompts.yaml"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_demo,node_mineru,node_model toneBlue
class node_article toneAmber
class node_llm,node_prompts toneMint
class node_json toneRose
class node_user,node_pdf,node_content,node_ollama,node_grobid toneIndigo
```

## Components

### PDF parsing

- [Demo runner](https://github.com/yrl-aida/docs_extractor/blob/master/demo.py): `demo.py`
- [MinerU parser](https://github.com/yrl-aida/docs_extractor/blob/master/extractor/mineru_compact.py): `extractor/mineru_compact.py`
- [Local model](https://github.com/yrl-aida/docs_extractor/tree/master/model): `model`

### Article extraction

- [Article extractor](https://github.com/yrl-aida/docs_extractor/blob/master/extractor/extractor.py): `extractor/extractor.py`

### Enrichment services

- [LLM extractor](https://github.com/yrl-aida/docs_extractor/blob/master/extractor/llm_extractor.py): `extractor/llm_extractor.py`
- [Extraction prompts](https://github.com/yrl-aida/docs_extractor/blob/master/data/prompts/prompts.yaml): `data/prompts/prompts.yaml`

### Structured output

- **Article JSON**

### Other components

- **PDF submitter**
- **Input PDF**
- **Parsed content**
- **LLM service**
- **GROBID service**

## Connections

- PDF submitter → Demo runner: starts extraction
- PDF submitter → Input PDF: provides PDF
- Demo runner → MinerU parser: calls parse_doc
- MinerU parser → Input PDF: reads
- MinerU parser → Local model: uses model
- MinerU parser → Parsed content: returns content list
- Demo runner → Article extractor: calls extraction
- Parsed content → Article extractor: supplies parsed content
- Article extractor → LLM extractor: requests extraction
- LLM extractor → Extraction prompts: reads prompts
- LLM extractor → LLM service: sends prompt
- Article extractor → GROBID service: requests author data
- Demo runner → Article extractor: calls JSON dump
- Article extractor → Article JSON: writes result