# rahulnyk/knowledge_graph_maker architecture

> Architecture diagram of the GitHub repository rahulnyk/knowledge_graph_maker, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rahulnyk/knowledge_graph_maker
- Repository: https://github.com/rahulnyk/knowledge_graph_maker

## Overview

##### Main components
- User Input / Document Ingestion: The starting point where user texts are provided. This may include functionality for splitting texts into chunks and wrapping them into Document models (using pydantic).
- Ontology Definition Module: Shows that the ontology is defined as a pydantic model with prescribed labels and relationships.
- GraphMaker Engine: The core module (found in graph_maker.py) that takes documents, processes each chunk, calls the LLM for graph generation, and parses responses into graph edges (with Node and Edge pydantic models).
- LLM Client Abstraction: An abstraction representing the LLM client interface. This module supports multiple clients (OpenAIClient, GroqClient, OllamaClient) and follows a common abstract base class design.
- Logging Component: The logger component used across the library for tone and debugging information.
- Neo4j Integration Module (Optional): Represented by Neo4jGraphModel, which takes the generated graph edges and saves them into a Neo4j database (supporting index creation, transactional saving, etc.).
- Distribution/Packaging Component: Not directly part of runtime architecture but important for deployment and distribution (indicated by the “dist” folder and packaging files).

##### Relationships and Data Flows
- The text is split into chunks; each chunk is converted into a Document model that also carries metadata.
- Documents are fed to the GraphMaker engine that:
- Uses the defined Ontology as a prompt guide.
- Iterates through the Documents.
- For each one, it sends the text (with predefined prompt tuning) to the LLM client.
- The LLM client generates a response, which the GraphMaker then parses. Parsing includes handling JSON issues and splitting responses appropriately.
- Successfully parsed responses yield a list of Edge models (with embedded Node objects and metadata).
- Optionally, if configured, the graph is passed on to the Neo4j Integration Module, which handles indexing and saving into a Neo4j graph database.
- Logging occurs concurrently to document key steps (using the logger module).

##### Architectural Patterns and Design Principles
- Modularity: Each core functionality (document processing, LLM integration, graph creation, Neo4j persistence) is encapsulated in its own module.
- Abstraction: The LLM Client uses an abstract base class allowing extensibility (users can add their own LLM clients as long as they implement the abstract methods).
- Robust Error Handling and Resilience: The GraphMaker is designed to handle faulty LLM responses (e.g., malformed JSON) by applying auto-correction and multiple parsing attempts.
- Extendability: The use of pydantic models for ontology, documents, and graph components makes it easy to extend or modify the schemas without significant refactoring.

##### Relevant Technologies, Frameworks, or Libraries
- Python, with specific usage of pydantic for data modeling.
- LLM APIs (OpenAI, Groq) integrated as client libraries.
- Neo4j for graph database persistence.
- Packaging tools (pyproject.toml, poetry) for dependency management and distribution.
- GitHub Actions for CI/CD (as seen from the .github/workflows directory).

##### Instructing the Principal Software Engineer
- A Parser component that transforms LLM output into structured graph edges/models.
- An optional branch where the graph data is sent to the “Neo4j Integration Module,” indicating a clear dependency for storing and optionally visualizing the constructed graph.
- A small auxiliary module for “Logging & Error Management” that interacts with the GraphMaker Engine.
- Data flow from documents to GraphMaker.
- Communication from GraphMaker to the LLM Clients.
- The resulting flow from GraphMaker to the Neo4j module.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Input Layer
    subgraph "Input Layer"
        A1["User Input & Document Preparation"]:::input
    end

    %% Core Processing Layer
    subgraph "Core Processing"
        B1["GraphMaker Engine (Processes Documents and LLM Calls)"]:::engine
        B2["Ontology Definition Module"]:::engine
        B3["LLM Client Abstraction"]:::llm
        B4["Logging & Error Management"]:::logger
    end

    %% LLM Clients Subsystem
    subgraph "LLM Clients"
        C1["OpenAI Client"]:::llm
        C2["Groq Client"]:::llm
        C3["Ollama Client"]:::llm
    end

    %% Persistence Layer
    subgraph "Persistence Layer"
        D1["Neo4j Integration Module"]:::persistence
    end

    %% Distribution/Packaging Layer
    subgraph "Distribution Layer"
        E1["Distribution/Packaging Component"]:::packaging
    end

    %% Dataflow Connections
    A1 -->|"processes"| B1
    B1 -->|"uses"| B2
    B1 -->|"calls"| B3
    B1 -->|"logs_to"| B4
    B3 -->|"invokes"| C1
    B3 -->|"invokes"| C2
    B3 -->|"invokes"| C3
    C1 -->|"response"| B3
    C2 -->|"response"| B3
    C3 -->|"response"| B3
    B1 -->|"saves_graph_if_enabled"| D1

    %% Styles
    classDef input fill:#ffe6e6,stroke:#333,stroke-width:2px;
    classDef engine fill:#e6f7ff,stroke:#333,stroke-width:2px;
    classDef llm fill:#e6ffe6,stroke:#333,stroke-width:2px;
    classDef logger fill:#ffffcc,stroke:#333,stroke-width:2px;
    classDef persistence fill:#ffe6ff,stroke:#333,stroke-width:2px;
    classDef packaging fill:#e6e6e6,stroke:#333,stroke-width:2px;

    %% Click Events
    click A1 "https://github.com/rahulnyk/knowledge_graph_maker/blob/main/knowledge_graph_maker/graph_maker.py"
    click B1 "https://github.com/rahulnyk/knowledge_graph_maker/blob/main/knowledge_graph_maker/graph_maker.py"
    click B2 "https://github.com/rahulnyk/knowledge_graph_maker/blob/main/knowledge_graph_maker/types.py"
    click B3 "https://github.com/rahulnyk/knowledge_graph_maker/tree/main/knowledge_graph_maker/llm_clients/"
    click B4 "https://github.com/rahulnyk/knowledge_graph_maker/blob/main/knowledge_graph_maker/logger.py"
    click C1 "https://github.com/rahulnyk/knowledge_graph_maker/blob/main/knowledge_graph_maker/llm_clients/openai_client.py"
    click C2 "https://github.com/rahulnyk/knowledge_graph_maker/blob/main/knowledge_graph_maker/llm_clients/groq_client.py"
    click C3 "https://github.com/rahulnyk/knowledge_graph_maker/blob/main/knowledge_graph_maker/llm_clients/ollama_client.py"
    click D1 "https://github.com/rahulnyk/knowledge_graph_maker/blob/main/knowledge_graph_maker/neo4j_graph_model.py"
    click E1 "https://github.com/rahulnyk/knowledge_graph_maker/tree/main/dist"
```
