# semanticclimate/climateinsight architecture

> Architecture diagram of the GitHub repository semanticclimate/climateinsight, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/semanticclimate/climateinsight
- Repository: https://github.com/semanticclimate/climateinsight

## Overview

ClimateInsight turns climate reports and research papers into a searchable, cited question-answering service, with a parallel Encyclopedia and corpus concept graph. Users ask a question in the browser; the API coordinates retrieval, answer generation and session history, then returns citations that can lead to source passages. Separately, document ingestion builds the shared vector index, while Encyclopedia vocabulary supports graph matching and term exploration. The README documents these flows and major capabilities; several subsystem links below rely on that documentation because their implementation files were not sampled.

Main entry points: [`backend/retrieval/pipeline.py`](https://github.com/semanticclimate/climateinsight/blob/main/backend/retrieval/pipeline.py) (Passage retrieval), [`backend/api/routes.py`](https://github.com/semanticclimate/climateinsight/blob/main/backend/api/routes.py) (FastAPI routes), [`backend/knowledge_graph/engine.py`](https://github.com/semanticclimate/climateinsight/blob/main/backend/knowledge_graph/engine.py) (Corpus concept graph), [`backend/ingest/ingest.py`](https://github.com/semanticclimate/climateinsight/blob/main/backend/ingest/ingest.py) (Document ingestion)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_api_ui["User interface and API"]
  node_browser_ui["ClimateInsight UI<br/>[main.js]"]
  node_api["FastAPI routes<br/>[routes.py]"]
  node_sessions[("Conversation history<br/>[sessions.py]")]
end

subgraph group_answering["Search and answers"]
  node_retrieval["Passage retrieval<br/>[pipeline.py]"]
  node_context["Evidence context<br/>[context_builder.py]"]
  node_citations["Citation parsing<br/>[citation_parser.py]"]
  node_prompt["Answer prompts<br/>[prompt_builder.py]"]
  node_llm_client["Local model client<br/>[client.py]"]
end

subgraph group_corpus["Corpus and indexing"]
  node_ingest["Document ingestion<br/>[ingest.py]"]
  node_parsing["HTML and XML parsing<br/>[parser.py]"]
  node_chunking["Passage chunking<br/>[chunker.py]"]
  node_indexer["Embedding and indexing<br/>[indexer.py]"]
  node_embedder["Text embeddings<br/>[embedder.py]"]
  node_vector_store[("Chroma passages<br/>[chroma_client.py]")]
end

subgraph group_knowledge["Encyclopedia and graph"]
  node_ency_build["Encyclopedia build<br/>[build.py]"]
  node_term_pipeline["Term extraction and enrichment"]
  node_ency_service["Encyclopedia service<br/>[service.py]"]
  node_concept_match["Concept matching<br/>[concepts.py]"]
  node_graph_engine["Corpus concept graph<br/>[engine.py]"]
  node_graph_question["Question graph focus<br/>[question.py]"]
  node_graph_topics["Graph communities<br/>[topics.py]"]
end

node_research_user(("Research user"))
node_ollama["Ollama"]
node_ipcc_sources["IPCC and research sources"]

node_research_user -->|"asks questions"| node_browser_ui
node_browser_ui -->|"submits questions"| node_api
node_api -->|"requests answers"| node_retrieval
node_api -->|"stores history"| node_sessions
node_retrieval -->|"retrieves passages"| node_vector_store
node_retrieval -->|"embeds questions"| node_embedder
node_retrieval -->|"builds evidence"| node_context
node_context -->|"formats context"| node_prompt
node_retrieval -->|"generates answers"| node_llm_client
node_llm_client -->|"sends prompt"| node_ollama
node_retrieval -->|"parses citations"| node_citations
node_api -->|"serves graph data"| node_graph_engine
node_api -->|"requests graph focus"| node_graph_question
node_ipcc_sources -->|"supplies documents"| node_ingest
node_ingest -->|"parses documents"| node_parsing
node_parsing -->|"produces text"| node_chunking
node_ingest -->|"builds index"| node_indexer
node_indexer -->|"embeds passages"| node_embedder
node_indexer -->|"stores chunks"| node_vector_store
node_ipcc_sources -->|"supplies terms"| node_ency_build
node_ency_build -->|"builds vocabulary"| node_term_pipeline
node_term_pipeline -->|"creates entries"| node_ency_service
node_graph_engine -->|"matches concepts"| node_concept_match
node_graph_engine -->|"reads indexed passages"| node_vector_store
node_graph_engine -->|"groups concepts"| node_graph_topics
node_graph_question -->|"generates explanation"| node_llm_client
node_graph_question -->|"uses retrieved passages"| node_retrieval

click node_browser_ui "https://github.com/semanticclimate/climateinsight/blob/main/frontend/js/main.js"
click node_api "https://github.com/semanticclimate/climateinsight/blob/main/backend/api/routes.py"
click node_sessions "https://github.com/semanticclimate/climateinsight/blob/main/backend/core/sessions.py"
click node_retrieval "https://github.com/semanticclimate/climateinsight/blob/main/backend/retrieval/pipeline.py"
click node_context "https://github.com/semanticclimate/climateinsight/blob/main/backend/retrieval/context_builder.py"
click node_citations "https://github.com/semanticclimate/climateinsight/blob/main/backend/retrieval/citation_parser.py"
click node_prompt "https://github.com/semanticclimate/climateinsight/blob/main/backend/retrieval/prompt_builder.py"
click node_llm_client "https://github.com/semanticclimate/climateinsight/blob/main/backend/llm/client.py"
click node_ingest "https://github.com/semanticclimate/climateinsight/blob/main/backend/ingest/ingest.py"
click node_parsing "https://github.com/semanticclimate/climateinsight/blob/main/backend/ingest/parser.py"
click node_chunking "https://github.com/semanticclimate/climateinsight/blob/main/backend/ingest/chunker.py"
click node_indexer "https://github.com/semanticclimate/climateinsight/blob/main/backend/vectorstore/indexer.py"
click node_embedder "https://github.com/semanticclimate/climateinsight/blob/main/backend/vectorstore/embedder.py"
click node_vector_store "https://github.com/semanticclimate/climateinsight/blob/main/backend/vectorstore/chroma_client.py"
click node_ency_build "https://github.com/semanticclimate/climateinsight/blob/main/backend/encyclopedia/build.py"
click node_term_pipeline "https://github.com/semanticclimate/climateinsight/blob/main/encyclopedia/pipeline/query_to_encyclopedia.py"
click node_ency_service "https://github.com/semanticclimate/climateinsight/blob/main/backend/encyclopedia/service.py"
click node_concept_match "https://github.com/semanticclimate/climateinsight/blob/main/backend/encyclopedia/concepts.py"
click node_graph_engine "https://github.com/semanticclimate/climateinsight/blob/main/backend/knowledge_graph/engine.py"
click node_graph_question "https://github.com/semanticclimate/climateinsight/blob/main/backend/knowledge_graph/question.py"
click node_graph_topics "https://github.com/semanticclimate/climateinsight/blob/main/backend/knowledge_graph/topics.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_browser_ui,node_api,node_sessions,node_research_user toneBlue
class node_retrieval,node_context,node_citations,node_prompt,node_llm_client toneAmber
class node_ingest,node_parsing,node_chunking,node_indexer,node_embedder,node_vector_store toneMint
class node_ency_build,node_term_pipeline,node_ency_service,node_concept_match,node_graph_engine,node_graph_question,node_graph_topics toneRose
class node_ollama,node_ipcc_sources toneIndigo
```

## Components

### User interface and API

- [ClimateInsight UI](https://github.com/semanticclimate/climateinsight/blob/main/frontend/js/main.js): `frontend/js/main.js`
- [FastAPI routes](https://github.com/semanticclimate/climateinsight/blob/main/backend/api/routes.py): `backend/api/routes.py`
- [Conversation history](https://github.com/semanticclimate/climateinsight/blob/main/backend/core/sessions.py): `backend/core/sessions.py`

### Search and answers

- [Passage retrieval](https://github.com/semanticclimate/climateinsight/blob/main/backend/retrieval/pipeline.py): `backend/retrieval/pipeline.py`
- [Evidence context](https://github.com/semanticclimate/climateinsight/blob/main/backend/retrieval/context_builder.py): `backend/retrieval/context_builder.py`
- [Citation parsing](https://github.com/semanticclimate/climateinsight/blob/main/backend/retrieval/citation_parser.py): `backend/retrieval/citation_parser.py`
- [Answer prompts](https://github.com/semanticclimate/climateinsight/blob/main/backend/retrieval/prompt_builder.py): `backend/retrieval/prompt_builder.py`
- [Local model client](https://github.com/semanticclimate/climateinsight/blob/main/backend/llm/client.py): `backend/llm/client.py`

### Corpus and indexing

- [Document ingestion](https://github.com/semanticclimate/climateinsight/blob/main/backend/ingest/ingest.py): `backend/ingest/ingest.py`
- [HTML and XML parsing](https://github.com/semanticclimate/climateinsight/blob/main/backend/ingest/parser.py): `backend/ingest/parser.py`
- [Passage chunking](https://github.com/semanticclimate/climateinsight/blob/main/backend/ingest/chunker.py): `backend/ingest/chunker.py`
- [Embedding and indexing](https://github.com/semanticclimate/climateinsight/blob/main/backend/vectorstore/indexer.py): `backend/vectorstore/indexer.py`
- [Text embeddings](https://github.com/semanticclimate/climateinsight/blob/main/backend/vectorstore/embedder.py): `backend/vectorstore/embedder.py`
- [Chroma passages](https://github.com/semanticclimate/climateinsight/blob/main/backend/vectorstore/chroma_client.py): `backend/vectorstore/chroma_client.py`

### Encyclopedia and graph

- [Encyclopedia build](https://github.com/semanticclimate/climateinsight/blob/main/backend/encyclopedia/build.py): `backend/encyclopedia/build.py`
- [Term extraction and enrichment](https://github.com/semanticclimate/climateinsight/blob/main/encyclopedia/pipeline/query_to_encyclopedia.py): `encyclopedia/pipeline/query_to_encyclopedia.py`
- [Encyclopedia service](https://github.com/semanticclimate/climateinsight/blob/main/backend/encyclopedia/service.py): `backend/encyclopedia/service.py`
- [Concept matching](https://github.com/semanticclimate/climateinsight/blob/main/backend/encyclopedia/concepts.py): `backend/encyclopedia/concepts.py`
- [Corpus concept graph](https://github.com/semanticclimate/climateinsight/blob/main/backend/knowledge_graph/engine.py): `backend/knowledge_graph/engine.py`
- [Question graph focus](https://github.com/semanticclimate/climateinsight/blob/main/backend/knowledge_graph/question.py): `backend/knowledge_graph/question.py`
- [Graph communities](https://github.com/semanticclimate/climateinsight/blob/main/backend/knowledge_graph/topics.py): `backend/knowledge_graph/topics.py`

### Other components

- **Research user**
- **Ollama**
- **IPCC and research sources**

## Connections

- Research user → ClimateInsight UI: asks questions
- ClimateInsight UI → FastAPI routes: submits questions
- FastAPI routes → Passage retrieval: requests answers
- FastAPI routes → Conversation history: stores history (evidence: `backend/api/routes.py`)
- Passage retrieval → Chroma passages: retrieves passages
- Passage retrieval → Text embeddings: embeds questions
- Passage retrieval → Evidence context: builds evidence
- Evidence context → Answer prompts: formats context
- Passage retrieval → Local model client: generates answers
- Local model client → Ollama: sends prompt (evidence: `backend/llm/client.py`)
- Passage retrieval → Citation parsing: parses citations
- FastAPI routes → Corpus concept graph: serves graph data (evidence: `backend/api/routes.py`)
- FastAPI routes → Question graph focus: requests graph focus (evidence: `backend/api/routes.py`)
- IPCC and research sources → Document ingestion: supplies documents
- Document ingestion → HTML and XML parsing: parses documents
- HTML and XML parsing → Passage chunking: produces text
- Document ingestion → Embedding and indexing: builds index (evidence: `backend/ingest/ingest.py`)
- Embedding and indexing → Text embeddings: embeds passages
- Embedding and indexing → Chroma passages: stores chunks
- IPCC and research sources → Encyclopedia build: supplies terms
- Encyclopedia build → Term extraction and enrichment: builds vocabulary
- Term extraction and enrichment → Encyclopedia service: creates entries (evidence: `backend/encyclopedia/pipeline.py`)
- Corpus concept graph → Concept matching: matches concepts (evidence: `backend/knowledge_graph/engine.py`)
- Corpus concept graph → Chroma passages: reads indexed passages (evidence: `backend/knowledge_graph/engine.py`)
- Corpus concept graph → Graph communities: groups concepts (evidence: `backend/knowledge_graph/engine.py`)
- Question graph focus → Local model client: generates explanation
- Question graph focus → Passage retrieval: uses retrieved passages
