How semanticclimate/climateinsight works
ClimateInsight turns climate reports and research papers into a searchable, cited question-answering service, with a parallel Encyclopedia and corpus concept graph. Users ask a question in the browser; the API coordinates retrieval, answer generation and session history, then returns citations that can lead to source passages. Separately, document ingestion builds the shared vector index, while Encyclopedia vocabulary supports graph matching and term exploration. The README documents these flows and major capabilities; several subsystem links below rely on that documentation because their implementation files were not sampled.
Main entry points: pipeline.py Passage retrieval · routes.py FastAPI routes · engine.py Corpus concept graph · ingest.py Document ingestion
Components 24
User interface and API
- ClimateInsight UI
frontend/js/main.js - FastAPI routes
backend/api/routes.py - Conversation history
backend/core/sessions.py
Search and answers
- Passage retrieval
backend/retrieval/pipeline.py - Evidence context
backend/retrieval/context_builder.py - Citation parsing
backend/retrieval/citation_parser.py - Answer prompts
backend/retrieval/prompt_builder.py - Local model client
backend/llm/client.py
Corpus and indexing
- Document ingestion
backend/ingest/ingest.py - HTML and XML parsing
backend/ingest/parser.py - Passage chunking
backend/ingest/chunker.py - Embedding and indexing
backend/vectorstore/indexer.py - Text embeddings
backend/vectorstore/embedder.py - Chroma passages
backend/vectorstore/chroma_client.py
Encyclopedia and graph
- Encyclopedia build
backend/encyclopedia/build.py - Term extraction and enrichment
encyclopedia/pipeline/query_to_encyclopedia.py - Encyclopedia service
backend/encyclopedia/service.py - Concept matching
backend/encyclopedia/concepts.py - Corpus concept graph
backend/knowledge_graph/engine.py - Question graph focus
backend/knowledge_graph/question.py - Graph communities
backend/knowledge_graph/topics.py
Other components
- Research user
- Ollama
- IPCC and research sources
Connections 27
- Research user → ClimateInsight UI: asks questions
- ClimateInsight UI → FastAPI routes: submits questions
- FastAPI routes → Passage retrieval: requests answers
- FastAPI routes → Conversation history: stores history
backend/api/routes.py - Passage retrieval → Chroma passages: retrieves passages
- Passage retrieval → Text embeddings: embeds questions
- Passage retrieval → Evidence context: builds evidence
- Evidence context → Answer prompts: formats context
- Passage retrieval → Local model client: generates answers
- Local model client → Ollama: sends prompt
backend/llm/client.py - Passage retrieval → Citation parsing: parses citations
- FastAPI routes → Corpus concept graph: serves graph data
backend/api/routes.py - FastAPI routes → Question graph focus: requests graph focus
backend/api/routes.py - IPCC and research sources → Document ingestion: supplies documents
- Document ingestion → HTML and XML parsing: parses documents
- HTML and XML parsing → Passage chunking: produces text
- Document ingestion → Embedding and indexing: builds index
backend/ingest/ingest.py - Embedding and indexing → Text embeddings: embeds passages
- Embedding and indexing → Chroma passages: stores chunks
- IPCC and research sources → Encyclopedia build: supplies terms
- Encyclopedia build → Term extraction and enrichment: builds vocabulary
- Term extraction and enrichment → Encyclopedia service: creates entries
backend/encyclopedia/pipeline.py - Corpus concept graph → Concept matching: matches concepts
backend/knowledge_graph/engine.py - Corpus concept graph → Chroma passages: reads indexed passages
backend/knowledge_graph/engine.py - Corpus concept graph → Graph communities: groups concepts
backend/knowledge_graph/engine.py - Question graph focus → Local model client: generates explanation
- Question graph focus → Passage retrieval: uses retrieved passages