How lamm-mit/graphreasoning works
Project Type and Purpose
• This is a Python software library (“GraphReasoning”) for scientific knowledge‐graph construction, analysis, embedding, and LLM‐driven reasoning.
File‐Structure Analysis
• Top‐level modules under GraphReasoning/:
• graph_generation.py – build or extend a knowledge graph from text.
• graph_analysis.py – find paths, communities, centralities, scale‐free tests.
• graph_tools.py – embedding I/O, visualization, simplification, statistics.
• agents.py – conversational agent classes and simulators.
• openai_tools.py – wrappers for LLM calls (OpenAI, llama.cpp).
• utils.py – filesystem, plotting, helper routines.
• Notebooks/: Jupyter notebooks that demonstrate workflows end‐to‐end.
• setup.py, README.md, LICENSE: packaging & documentation.
Key Architectural Insights from README
• Data sources: GraphML files and pickled embeddings hosted on HuggingFace.
• External dependencies: transformers, llama‐cpp, powerlaw, wkhtmltopdf, huggingface_hub, networkx, pyvis.
• Core flows:
b. Multi‐agent loops for conversational reasoning.
Main Components & Interactions
A. Data Layer
• Raw Graph Data (GraphML) & Embeddings (PKL) – stored locally or fetched via huggingface_hub.
• Notebooks & Scripts – user‐facing clients that invoke library functions.
B. Core Processing Layer (Python Modules)
• graph_generation: “Text → Graph” pipeline using openai_tools.
• graph_analysis: “Graph → Metrics/Paths/Communities.”
• graph_tools: “Graph & Embeddings ↔ Visualization & Stats.”
• agents: “LLM Agents ↔ Conversation Simulation.”
• openai_tools: abstracts LLM API (OpenAI, llama.cpp) for all modules.
• utils: shared helpers (file I/O, plotting).
C. External Services
• HuggingFace Hub – storage & retrieval of models, data files.
• OpenAI API / llama.cpp – language model inference.
• wkhtmltopdf – HTML to PDF conversion for reports.
D. Presentation Layer
• HTML / PDF Reports (pyvis networks, matplotlib plots).
• Jupyter Notebooks – orchestrate end‐to‐end examples.
Data flows:
1. Notebooks call core modules → graph data loaded or generated.
2. graph_generation invokes openai_tools → receives LLM output → produces GraphML.
3. graph_tools loads embeddings (via huggingface_hub) → networkx graph → generate node embeddings (via openai_tools).
4. graph_analysis consumes graph & embeddings → computes metrics, paths (optionally calling openai_tools for reasoning).
5. agents orchestrate openai_tools calls for conversational reasoning.
6. graph_tools & utils produce HTML/PDF outputs saved to disk.
Architectural Patterns & Principles
• Modular separation: each module encapsulates one concern.
• Adapter pattern: openai_tools wraps external LLM APIs.
• Dependency injection: notebooks pass tokenizer, model, generate functions to modules.