# gptechday/openai-academy-kg-recipe architecture

> Architecture diagram of the GitHub repository gptechday/openai-academy-kg-recipe, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/gptechday/openai-academy-kg-recipe
- Repository: https://github.com/gptechday/openai-academy-kg-recipe

## Overview

##### Project Type and Purpose
- This is a full-stack backend application focused on healthcare data processing. It leverages Synthea CSV healthcare data to build a knowledge graph, enriches it with vector indexing for semantic search, integrates with a Neo4j database, and exposes a FastAPI REST interface. Additionally, it incorporates natural language processing (NLP) through OpenAI models to translate user queries into actionable Cypher commands.

##### Analyzing the File Structure
- Key files and directories include:
- main.py – The entry point for the FastAPI server.
- utils/ – Contains modules like kg_builder.py (for constructing the graph), kg_to_vector.py (for vector indexing), neo4j_graph_builder.py (for integrating with Neo4j), synthea_ontology.py (which defines healthcare ontology), and cypher_writer.py (for generating Neo4j queries).
- prompts/ – Holds prompt files used with OpenAI models to process natural language queries.
- synthea_sample_data_csv_latest/ – Contains raw Synthea CSV datasets which are used to populate the knowledge graph.
- data/ – Likely used to store intermediary output such as the constructed knowledge graph (data/knowledge_graph.json).
- rebuild_index.py – A utility used to regenerate the vector index when modifications occur.
- Dockerfile and cloudbuild.yaml – For containerization and deployment.
- These components suggest clear divisions between data ingestion, graph construction, vector embedding creation, Cypher query generation, and API interfacing.

##### Core Components
a. API Layer:
- FastAPI application (main.py) exposing endpoints (e.g., /query and /rebuild_index).
b. Data Ingestion and Preprocessing:
- Synthea CSV data files (in synthea_sample_data_csv_latest).
- Data is read, preprocessed, and prepared for graph creation.
c. Knowledge Graph Construction:
- The utilities in utils/ (kg_builder.py, synthea_ontology.py) transform raw healthcare data into a structured graph.
d. Vector Indexing:
- The module kg_to_vector.py converts the knowledge graph into vector embeddings; rebuild_index.py triggers an update of this index.
e. Neo4j Integration:
- neo4j_graph_builder.py manages the interactions with the Neo4j database.
- cypher_writer.py handles translating natural language queries into Cypher queries.
f. NLP and Query Translation:
- The prompts in prompts/ work with OpenAI models to understand and convert natural language queries.
g. External Systems and Deployment:
- Neo4j Database: As the underlying graph database.
- OpenAI models: For natural language query processing.
- Docker container: For deployment and consistent runtime (Dockerfile).
- Cloud Build: Deployment automation (cloudbuild.yaml).

##### Relationships and Interactions
- API calls (especially /query) flow into the business logic where:
- The query is sent to the NLP processing module (using prompts/OpenAI models).
- The processed query is then converted to a Cypher command (cypher_writer.py) and executed against the Neo4j database (via neo4j_graph_builder.py).
- In parallel, vector search using the output of kg_to_vector.py may be performed to refine or support the query results.
- Data ingestion processes (reading Synthea CSV files) feed into the knowledge graph construction process.
- The vector indexing component interacts with the constructed graph, storing its summary in an index used during query resolution.
- Docker and cloud deployment components oversee containerization and continuous deployment flows.

##### Architectural Patterns and Technologies
- Component Separation: Dividing responsibilities across modules (API layer, data processing, graph building, NLP, and database integrations).
- Microservices-like Flow (even if within one service): Where data flows sequentially and components interact via well-defined interfaces.
- Use of REST APIs (via FastAPI) and containerization (via Docker) for deployment consistency.
- Leveraging external APIs (OpenAI’s language models) and databases (Neo4j).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Global External Entity
    client["Client Application"]:::external

    %% API Layer Subgraph
    subgraph "API Layer"
        api["FastAPI API (main.py)"]:::api
    end

    %% Data Ingestion Subgraph
    subgraph "Data Ingestion and Preprocessing"
        csv["Synthea CSV Data"]:::data
    end

    %% Knowledge Graph Construction Subgraph
    subgraph "Knowledge Graph Construction"
        kg["Knowledge Graph Builder (kg_builder.py & synthea_ontology.py)"]:::processing
    end

    %% Vector Indexing Subgraph
    subgraph "Vector Indexing"
        vector["Vector Indexer (kg_to_vector.py)"]:::processing
        rebuild["Rebuild Index (rebuild_index.py)"]:::processing
    end

    %% NLP and Query Translation Subgraph
    subgraph "Query Processing"
        nlp["NLP & Query Translator (prompts)"]:::nlp
        cypher["Cypher Query Generator (cypher_writer.py)"]:::processing
        neo4j["Neo4j Integration (neo4j_graph_builder.py)"]:::database
    end

    %% External Systems Subgraph
    subgraph "External Systems"
        openai["OpenAI Models/API"]:::external
        neo4jDB["Neo4j Database"]:::database
    end

    %% Deployment Subgraph
    subgraph "Deployment and Containerization"
        docker["Docker Container (Dockerfile)"]:::deployment
        cloudbuild["Cloud Build Pipeline (cloudbuild.yaml)"]:::deployment
    end

    %% Connections and Data Flows
    client -->|"APIRequest"| api
    api -->|"TriggerIngestion"| csv
    csv -->|"PreprocessData"| kg
    kg -->|"BuildGraph"| vector
    vector -->|"GenerateIndex"| rebuild

    api -->|"QueryEndpoint"| nlp
    nlp -->|"ConvertQuery"| cypher
    cypher -->|"ExecuteQuery"| neo4j
    neo4j -->|"InteractDB"| neo4jDB
    neo4jDB -->|"ReturnResults"| neo4j
    neo4j -->|"Results"| api
    api -->|"Response"| client

    api -->|"UseNLP"| openai

    api -->|"DeployedOn"| docker
    docker -->|"CI/CD"| cloudbuild

    %% Click Events
    click api "https://github.com/gptechday/openai-academy-kg-recipe/blob/main/main.py"
    click csv "https://github.com/gptechday/openai-academy-kg-recipe/tree/main/synthea_sample_data_csv_latest"
    click kg "https://github.com/gptechday/openai-academy-kg-recipe/blob/main/utils/kg_builder.py"
    click kg "https://github.com/gptechday/openai-academy-kg-recipe/blob/main/utils/synthea_ontology.py"
    click vector "https://github.com/gptechday/openai-academy-kg-recipe/blob/main/utils/kg_to_vector.py"
    click rebuild "https://github.com/gptechday/openai-academy-kg-recipe/blob/main/rebuild_index.py"
    click neo4j "https://github.com/gptechday/openai-academy-kg-recipe/blob/main/utils/neo4j_graph_builder.py"
    click cypher "https://github.com/gptechday/openai-academy-kg-recipe/blob/main/utils/cypher_writer.py"
    click nlp "https://github.com/gptechday/openai-academy-kg-recipe/blob/main/prompts/prompts.py"
    click docker "https://github.com/gptechday/openai-academy-kg-recipe/tree/main/Dockerfile"
    click cloudbuild "https://github.com/gptechday/openai-academy-kg-recipe/blob/main/cloudbuild.yaml"

    %% Styles
    classDef external fill:#f9f,stroke:#333,stroke-width:2px;
    classDef api fill:#b3e5fc,stroke:#0277bd,stroke-width:2px;
    classDef data fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px;
    classDef processing fill:#fff9c4,stroke:#f9a825,stroke-width:2px;
    classDef nlp fill:#d1c4e9,stroke:#6a1b9a,stroke-width:2px;
    classDef database fill:#ffe0b2,stroke:#e65100,stroke-width:2px;
    classDef deployment fill:#fce4ec,stroke:#880e4f,stroke-width:2px;
```
