# wesroots/msc_project_refactor architecture

> Architecture diagram of the GitHub repository wesroots/msc_project_refactor, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/wesroots/msc_project_refactor
- Repository: https://github.com/wesroots/msc_project_refactor

## Overview

This is a biomedical knowledge-graph construction pipeline: it searches PubMed for abstracts, extracts entities and relations with an LLM, and converts extraction output into evaluation-ready predictions. Entity normalization is implemented separately; evaluation, relation validation, and graph construction are named in the pipeline but their modules are empty or the orchestration is unfinished. Edges reflect shown calls, reads, and API requests; the separate normalization module is not shown wired into the main script.

Main entry points: [`src/pipeline/extraction.py`](https://github.com/wesroots/msc_project_refactor/blob/main/src/pipeline/extraction.py) (LLM extractor), [`scripts/kg_pipeline.py`](https://github.com/wesroots/msc_project_refactor/blob/main/scripts/kg_pipeline.py) (Pipeline runner), [`src/pipeline/corpus.py`](https://github.com/wesroots/msc_project_refactor/blob/main/src/pipeline/corpus.py) (Corpus builder), [`src/pipeline/processing.py`](https://github.com/wesroots/msc_project_refactor/blob/main/src/pipeline/processing.py) (Output processor)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_collection["Corpus collection"]
  node_query_config["Search queries<br/>[queries.py]"]
  node_corpus_builder["Corpus builder<br/>[corpus.py]"]
end

subgraph group_extraction["LLM extraction"]
  node_path_config["Path configuration<br/>[paths.py]"]
  node_prompt_template["Prompt template"]
  node_guidelines["Extraction guidelines"]
  node_examples["Few-shot examples<br/>[fs_block.txt]"]
  node_extractor["LLM extractor<br/>[extraction.py]"]
end

subgraph group_normalization["Entity normalization"]
  node_normalizer["Entity normalizer"]
end

subgraph group_outputs["Processing and KG"]
  node_processor["Output processor<br/>[processing.py]"]
  node_extraction_log["Extraction log"]
  node_evaluation["Evaluation<br/>[evaluation.py]"]
  node_relation_validation["Relation validation"]
  node_graph_builder["Graph construction<br/>[graph.py]"]
end

node_runner["Pipeline runner<br/>[kg_pipeline.py]"]
node_pubmed(("PubMed"))
node_openai(("OpenAI API"))
node_ncbi(("NCBI E-utilities"))
node_cellosaurus(("Cellosaurus API"))

node_runner -->|"calls"| node_corpus_builder
node_runner -->|"calls"| node_extractor
node_runner -->|"calls"| node_processor
node_corpus_builder -->|"imports queries"| node_query_config
node_corpus_builder -->|"searches and fetches"| node_pubmed
node_extractor -->|"requests extraction"| node_openai
node_extractor -->|"loads paths from"| node_path_config
node_extractor -->|"loads prompt"| node_prompt_template
node_extractor -->|"loads guidelines"| node_guidelines
node_extractor -->|"loads examples"| node_examples
node_normalizer -->|"searches identifiers"| node_ncbi
node_normalizer -->|"searches cell lines"| node_cellosaurus
node_processor -->|"imports output path"| node_path_config
node_processor -->|"reads outputs"| node_extraction_log

click node_runner "https://github.com/wesroots/msc_project_refactor/blob/main/scripts/kg_pipeline.py"
click node_query_config "https://github.com/wesroots/msc_project_refactor/blob/main/configs/queries.py"
click node_corpus_builder "https://github.com/wesroots/msc_project_refactor/blob/main/src/pipeline/corpus.py"
click node_path_config "https://github.com/wesroots/msc_project_refactor/blob/main/configs/paths.py"
click node_prompt_template "https://github.com/wesroots/msc_project_refactor/blob/main/data/prompt_template.txt"
click node_guidelines "https://github.com/wesroots/msc_project_refactor/blob/main/data/br_ext_guidelines.txt"
click node_examples "https://github.com/wesroots/msc_project_refactor/blob/main/data/fs_block.txt"
click node_extractor "https://github.com/wesroots/msc_project_refactor/blob/main/src/pipeline/extraction.py"
click node_normalizer "https://github.com/wesroots/msc_project_refactor/blob/main/src/pipeline/entity_normalisation.py"
click node_processor "https://github.com/wesroots/msc_project_refactor/blob/main/src/pipeline/processing.py"
click node_extraction_log "https://github.com/wesroots/msc_project_refactor/blob/main/data/run/extraction_log.json"
click node_evaluation "https://github.com/wesroots/msc_project_refactor/blob/main/src/pipeline/evaluation.py"
click node_relation_validation "https://github.com/wesroots/msc_project_refactor/blob/main/src/pipeline/relation_validation.py"
click node_graph_builder "https://github.com/wesroots/msc_project_refactor/blob/main/src/pipeline/graph.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_query_config,node_corpus_builder toneBlue
class node_path_config,node_prompt_template,node_guidelines,node_examples,node_extractor toneAmber
class node_normalizer,node_openai,node_cellosaurus toneMint
class node_processor,node_extraction_log,node_evaluation,node_relation_validation,node_graph_builder toneRose
class node_runner toneTeal
class node_pubmed,node_ncbi toneIndigo
```

## Components

### Corpus collection

- [Search queries](https://github.com/wesroots/msc_project_refactor/blob/main/configs/queries.py): `configs/queries.py`
- [Corpus builder](https://github.com/wesroots/msc_project_refactor/blob/main/src/pipeline/corpus.py): `src/pipeline/corpus.py`

### LLM extraction

- [Path configuration](https://github.com/wesroots/msc_project_refactor/blob/main/configs/paths.py): `configs/paths.py`
- [Prompt template](https://github.com/wesroots/msc_project_refactor/blob/main/data/prompt_template.txt): `data/prompt_template.txt`
- [Extraction guidelines](https://github.com/wesroots/msc_project_refactor/blob/main/data/br_ext_guidelines.txt): `data/br_ext_guidelines.txt`
- [Few-shot examples](https://github.com/wesroots/msc_project_refactor/blob/main/data/fs_block.txt): `data/fs_block.txt`
- [LLM extractor](https://github.com/wesroots/msc_project_refactor/blob/main/src/pipeline/extraction.py): `src/pipeline/extraction.py`

### Entity normalization

- [Entity normalizer](https://github.com/wesroots/msc_project_refactor/blob/main/src/pipeline/entity_normalisation.py): `src/pipeline/entity_normalisation.py`

### Processing and KG

- [Output processor](https://github.com/wesroots/msc_project_refactor/blob/main/src/pipeline/processing.py): `src/pipeline/processing.py`
- [Extraction log](https://github.com/wesroots/msc_project_refactor/blob/main/data/run/extraction_log.json): `data/run/extraction_log.json`
- [Evaluation](https://github.com/wesroots/msc_project_refactor/blob/main/src/pipeline/evaluation.py): `src/pipeline/evaluation.py`
- [Relation validation](https://github.com/wesroots/msc_project_refactor/blob/main/src/pipeline/relation_validation.py): `src/pipeline/relation_validation.py`
- [Graph construction](https://github.com/wesroots/msc_project_refactor/blob/main/src/pipeline/graph.py): `src/pipeline/graph.py`

### Other components

- [Pipeline runner](https://github.com/wesroots/msc_project_refactor/blob/main/scripts/kg_pipeline.py): `scripts/kg_pipeline.py`
- **PubMed**
- **OpenAI API**
- **NCBI E-utilities**
- **Cellosaurus API**

## Connections

- Pipeline runner → Corpus builder: calls (evidence: `scripts/kg_pipeline.py`)
- Pipeline runner → LLM extractor: calls (evidence: `scripts/kg_pipeline.py`)
- Pipeline runner → Output processor: calls (evidence: `scripts/kg_pipeline.py`)
- Corpus builder → Search queries: imports queries (evidence: `src/pipeline/corpus.py`)
- Corpus builder → PubMed: searches and fetches (evidence: `src/pipeline/corpus.py`)
- LLM extractor → OpenAI API: requests extraction (evidence: `src/pipeline/extraction.py`)
- LLM extractor → Path configuration: loads paths from (evidence: `src/pipeline/extraction.py`)
- LLM extractor → Prompt template: loads prompt (evidence: `src/pipeline/extraction.py`)
- LLM extractor → Extraction guidelines: loads guidelines (evidence: `src/pipeline/extraction.py`)
- LLM extractor → Few-shot examples: loads examples (evidence: `src/pipeline/extraction.py`)
- Entity normalizer → NCBI E-utilities: searches identifiers (evidence: `src/pipeline/entity_normalisation.py`)
- Entity normalizer → Cellosaurus API: searches cell lines (evidence: `src/pipeline/entity_normalisation.py`)
- Output processor → Path configuration: imports output path (evidence: `src/pipeline/processing.py`)
- Output processor → Extraction log: reads outputs (evidence: `src/pipeline/processing.py`)