How geeekai/atom works
Project Type and Purpose
• ATOM is a backend library/framework implemented in Python.
• Purpose: Ingest timestamped text, split it into atomic facts, extract 5-tuples via LLM calls, build small “atomic” Knowledge Graphs, and then merge them in parallel to form a continuously updatable Temporal Knowledge Graph (TKG).
File Structure
• Top-level directories and files
• atom/ — core modules
• datasets/ — input data files
• evaluation/ — Jupyter Notebooks for benchmarking
• requirements.txt — Python dependencies
• Key submodules inside atom/
• atom.py — orchestrates build/update of the TKG
• documents_distiller/ — splits docs into atomic facts
• graph_integration/ — merges atomic KGs and writes to Neo4j
• models/ — in-memory data models (KnowledgeGraph, Factoid, etc.)
• utils/ — LLM output parsing, schema definitions, entity/relation matching
README for Insights
• Confirms a pipeline architecture:
1. Document Distillation
2. 5-tuple Extraction (via LLM + embeddings)
3. Atomic KG Construction
4. Parallel Merging & Conflict Resolution (dual-time modeling)
5. Continuous Updates into an existing TKG
• External dependencies: OpenAI LLM API, OpenAIEmbeddings, LangChain wrappers, Neo4j for visualization
Main Components & Their Relationships
A. Data Ingestion Layer
• Datasets folder (Excel/Parquet files)
• Evaluation Notebooks (optional clients)
→ feeds raw text into
B. Document Distiller Module (atom/documents_distiller/documents_distiller.py)
→ outputs “atomic facts”
C. LLM & Embeddings Service (external)
• ChatOpenAI (LLM)
• OpenAIEmbeddings
→ called by atom/utils/llm_output_parser.py to extract subject–relation–object 5-tuples, t_start, t_end
D. Atomic KG Constructor (atom/atom.py)
• Takes factoids + embeddings
• Builds miniature KGs in memory (atom/models/knowledge_graph.py)
E. Parallel KG Merger (atom/atom.py)
• Uses vector similarity thresholds (ent_threshold, rel_threshold)
• Applies dual-time modeling, conflict resolution
→ produces a unified TKG data structure
F. Graph Integration Module (atom/graph_integration/graph_integrator.py)
• Persists final TKG to an external Neo4j database or other graph store
• Provides visualize_graph() for downstream use
G. Evaluation & Visualization
• Jupyter Notebooks in evaluation/ demonstrate latency, stability, scalability
• Plots and dashboards
Architectural Patterns and Design Principles
• Pipeline Pattern: clear, sequential steps from raw text → atomic facts → mini-KGs → merged TKG.
• Modular Design: each concern isolated in its own module.
• Parallelism: merging atomic KGs in parallel to scale.
• Dual-Time Modeling: handles inherent vs. observation timestamps.
• Extensibility: new data sources, new LLMs/embeddings providers can swap in via utils.
Detailed Labeling
• On the merge step, annotate the thresholds (ent_threshold, rel_threshold) and the “dual-time conflict resolution” logic.
• On the final graph store, label it “Neo4j (via GraphIntegrator)” and note possible Cypher queries or visualization calls.