# sohv/compliance-graph architecture

> Architecture diagram of the GitHub repository sohv/compliance-graph, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sohv/compliance-graph
- Repository: https://github.com/sohv/compliance-graph

## Overview

Based on a careful review of both the file tree and the README, this project is clearly a prototype for a financial compliance system that leverages knowledge graphs (with Neo4j) to extract, ingest, and validate compliance rules based on financial documents. The core idea is to extract relevant rules from documents, store them in a knowledge graph, and use semantic techniques (with SentenceTransformer NLP models) to assess transaction compliance.

##### Main Components
- Data Source & Document Input: Represent the source of your financial documents. Note the “data/docs” directory that holds financial documents (e.g., Synthetic_Financial_Rules.pdf).
- Rule Extraction: Highlight the process and modules responsible for extracting rules from PDFs. This involves the file “src/read_pdf.py” and “src/extract_rules.py” which likely use text processing and NLP techniques.
- Database Access & Ingestion: Show the process of taking the extracted rules and storing them in the Neo4j knowledge graph. The “src/db/connection.py” (and the entire “src/db” directory) is key here, as well as “src/ingest.py” which performs the ingestion.
- Compliance Module: Identify the functionality that retrieves rules from the knowledge graph and checks them against transactions. Use the “src/compliance/” directory components:
- rule_retrieve.py – retrieves relevant rules using semantic similarity.
- rule_check.py – evaluates if a transaction meets the compliance rules.
- check_compliance.py – orchestrates the overall compliance check.
- Evaluation and Search: Include evaluation (“src/evaluate_rules.py”) that assesses rule matching performance, and the “src/search_rules.py” which might be used for querying/assessing existing rules.

##### Relationships and Data Flow
- The extracted rules (saved as JSON files in the “data” directory) flow into the ingestion phase where they are loaded into the Neo4j knowledge graph (illustrate a connection from extraction module to database ingestion module).
- Once rules are in the graph, the compliance module retrieves these rules using semantic similarity (by leveraging the SentenceTransformer model) and checks incoming transactions for compliance.
- Finally, an evaluation module assesses the performance of the rule matching, with output possibly being logged in “data/output.txt” or evaluated via “src/evaluate_rules.py”.

##### Architectural Patterns & Design Principles
- Separation of Concerns: Components such as extraction, ingestion, compliance checking, and evaluation are clearly separated into different scripts and modules.
- Modular Design: Each functional area (compliance, database, extraction) is organized into its own subdirectory and Python file, allowing independent development and testing.
- Externalized Configuration: Use of a “config/config.py” for system parameters and environment variables (detailed in the README via the .env file) emphasizes centralized configuration management.
- Data Persistence through a Knowledge Graph: Emphasize the Neo4j database integration as a key technology which underpins the entire compliance rule retrieval and verification process.

##### Relevant Technologies & Tools
- Neo4j: As the target database for the knowledge graph.
- SentenceTransformer: For computing semantic similarity using the ‘all-MiniLM-L6-v2’ NLP model.
- Python: The primary programming language with modules split across directories.
- Various utility scripts (for ingest, extraction, compliance checking, etc.) that form the processing pipeline.

- Each component should have:
- A clearly labeled rectangle (e.g., "PDF Extraction (src/read_pdf.py & src/extract_rules.py)").

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Document Input & Extraction Layer
    subgraph "Document Input & Extraction"
        DOC["Document Input (data/docs)"]:::blue
        READ["Read PDF (src/read_pdf.py)"]:::blue
        EXTR["Extract Rules (src/extract_rules.py)"]:::blue
    end

    %% Knowledge Graph & Ingestion Layer
    subgraph "Knowledge Graph & Ingestion"
        ING["Ingestion (src/ingest.py)"]:::green
        DB["Knowledge Graph (Neo4j) - Connection (src/db/connection.py)"]:::green
    end

    %% Compliance Module Layer
    subgraph "Compliance Module"
        RETRIEVE["Rule Retrieval (src/compliance/rule_retrieve.py)"]:::orange
        CHECK["Rule Checking (src/compliance/rule_check.py)"]:::orange
        ORCH["Compliance Orchestration (src/compliance/check_compliance.py)"]:::orange
    end

    %% Evaluation & Search Layer
    subgraph "Evaluation & Search"
        EVAL["Evaluation (src/evaluate_rules.py)"]:::yellow
        SEARCH["Search (src/search_rules.py)"]:::yellow
    end

    %% External Configuration Node
    CONFIG["External Config (config/config.py)"]:::gray

    %% Data Flow Connections
    DOC -->|"processes"| READ
    DOC -->|"processes"| EXTR
    READ -->|"outputs rules"| ING
    EXTR -->|"outputs rules"| ING
    ING -->|"stores rules in"| DB
    DB -->|"supplies rules to"| RETRIEVE
    RETRIEVE -->|"passes rules to"| CHECK
    CHECK -->|"orchestrates"| ORCH
    ORCH -->|"triggers evaluation"| EVAL
    EVAL -->|"enables search"| SEARCH

    %% External Configuration influence
    CONFIG -->|"configures"| ING
    CONFIG -->|"configures"| RETRIEVE
    CONFIG -->|"configures"| CHECK

    %% Click Events for Component Mapping
    click DOC "https://github.com/sohv/compliance-graph/tree/main/data/docs"
    click READ "https://github.com/sohv/compliance-graph/blob/main/src/read_pdf.py"
    click EXTR "https://github.com/sohv/compliance-graph/blob/main/src/extract_rules.py"
    click ING "https://github.com/sohv/compliance-graph/blob/main/src/ingest.py"
    click DB "https://github.com/sohv/compliance-graph/blob/main/src/db/connection.py"
    click RETRIEVE "https://github.com/sohv/compliance-graph/blob/main/src/compliance/rule_retrieve.py"
    click CHECK "https://github.com/sohv/compliance-graph/blob/main/src/compliance/rule_check.py"
    click CHECK "https://github.com/sohv/compliance-graph/blob/main/src/compliance/check_compliance.py"
    click EVAL "https://github.com/sohv/compliance-graph/blob/main/src/evaluate_rules.py"
    click SEARCH "https://github.com/sohv/compliance-graph/blob/main/src/search_rules.py"
    click CONFIG "https://github.com/sohv/compliance-graph/blob/main/config/config.py"

    %% Styles for Visual Cues
    classDef blue fill:#cce5ff,stroke:#004085,stroke-width:2px;
    classDef green fill:#d4edda,stroke:#155724,stroke-width:2px;
    classDef orange fill:#ffe5b4,stroke:#ff7f00,stroke-width:2px;
    classDef yellow fill:#fff3cd,stroke:#856404,stroke-width:2px;
    classDef gray fill:#e2e3e5,stroke:#858796,stroke-width:2px;
```
