# sensein/brainkb architecture

> Architecture diagram of the GitHub repository sensein/brainkb, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-11-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sensein/brainkb
- Repository: https://github.com/sensein/brainkb

## Overview

##### Project type and purpose
- BrainKB is a full-fledged, microservices-based knowledge-base platform for neuroscience. It comprises multiple back-end services (ingestion, query, chat, ML, user management, authentication) communicating via REST and messaging, backed by GraphDB/RDF stores, Postgres caches, and RabbitMQ, and (formerly) a Django-based web UI.

##### File structure
- Top-level directories each represent one service or infra component:
- APItokenmanager (Django JWT user & scope manager)
- ingestion_service (producer + worker)
- query_service
- usermanagement_service
- chat_service
- ml_service
- graphdb (GraphDB triple store)
- oxigraph (alternative graph store)
- rabbit-mq (message broker)
- sparql_queries, linkml-models (ancillary artifacts)
- Each “_service” folder follows the same pattern: core/routers (API endpoints), core/models (ORM/pydantic), core/database, core/security, tests, Dockerfile, compose files for dev & prod.
- Presence of docker-compose files indicates containerized deployment; configuration and env files show external dependency wiring.

##### README for additional insights
- Confirms microservices: ingest (with RabbitMQ), query (GraphDB SPARQL), JWT manager, GraphDB, RabbitMQ, plus documentation link.
- WebApp is obsolete—focus on back-end services.

a. Main components
- API Gateway / Client Applications (future web UI, CLI, other clients)
- APItokenmanager (Django): issues and verifies JWTs, manages scopes
- usermanagement_service: user CRUD, roles
- ingestion_service:
- Producer API receives data → validates → enqueues to RabbitMQ
- Worker listens to RabbitMQ → processes messages → writes to GraphDB (or RDF store)
- query_service: handles SPARQL queries and updates against GraphDB
- chat_service: provides conversational interfaces, caches user context in Postgres, may query GraphDB
- ml_service: NLP/NER and GPT-based services, reads from GraphDB or other data stores, writes results back
- Data stores:
- GraphDB (primary RDF triple store)
- Oxigraph (secondary/experimental graph store)
- Postgres instances (per-service caches: chat_service, ml_service, usermanagement_service)
- RabbitMQ: message broker for ingestion pipeline
- SPARQL queries & LinkML schemas: offline artifacts used by query and ingestion services

b. Relationships and interactions
- Clients → APItokenmanager to obtain JWTs
- Clients → each service’s REST API (ingestion, query, chat, ml, usermgmt), passing JWT bearer tokens
- ingestion_service Producer → RabbitMQ → ingestion_service Worker → GraphDB
- query_service ↔ GraphDB (via SPARQL HTTP endpoint)
- chat_service ↔ GraphDB (for context) and ↔ its Postgres cache
- ml_service ↔ GraphDB and ↔ its Postgres database
- usermanagement_service ↔ its Postgres database
- All services validate JWTs via APItokenmanager’s public keys or introspection

c. Architectural patterns and principles
- Microservices with single-responsibility boundaries
- Event-driven ingestion pipeline using RabbitMQ
- Polyglot persistence: RDF store for KGs, relational caches for fast lookups
- JWT-based centralized authentication/authorization
- Containerization and environment-specific docker-compose setups

d. Key technologies
- Django (APItokenmanager)
- FastAPI (likely for other core services based on routers/pydantic)
- RabbitMQ (messaging)
- GraphDB & Oxigraph (triple stores)
- Postgres (caching and relational storage)
- Docker & Docker Compose

## Diagram (Mermaid)

```mermaid
flowchart TD
    ClientApps["Clients (Web UI, CLI)"]:::client

    subgraph "Authentication"
        direction TB
        APItoken["APItokenmanager"]:::auth
    end

    subgraph "Microservices"
        direction TB
        UM["usermanagement_service"]:::api
        ING_P["ingestion_service Producer"]:::api
        ING_W["ingestion_service Worker"]:::worker
        QRY["query_service"]:::api
        CHAT["chat_service"]:::api
        ML["ml_service"]:::api
    end

    subgraph "Messaging"
        direction TB
        Rabbit["RabbitMQ"]:::mq
    end

    subgraph "DataStores"
        direction TB
        GraphDB["GraphDB"]:::db
        Oxigraph["Oxigraph"]:::db
        PG_UM["Postgres (usermgmt)"]:::db
        PG_CHAT["Postgres (chat)"]:::db
        PG_ML["Postgres (ml)"]:::db
    end

    subgraph "Artifacts / Models"
        direction TB
        SPARQL["sparql_queries"]:::art
        LINKML["linkml-models"]:::art
    end

    %% Client to auth and services
    ClientApps -->|"HTTP + JWT"| APItoken
    ClientApps -->|"HTTP + JWT"| UM
    ClientApps -->|"HTTP + JWT"| ING_P
    ClientApps -->|"HTTP + JWT"| QRY
    ClientApps -->|"HTTP + JWT"| CHAT
    ClientApps -->|"HTTP + JWT"| ML

    %% JWT validation by services
    UM -.->|"Validate JWT"| APItoken
    ING_P -.->|"Validate JWT"| APItoken
    QRY -.->|"Validate JWT"| APItoken
    CHAT -.->|"Validate JWT"| APItoken
    ML -.->|"Validate JWT"| APItoken

    %% Service interactions
    UM -->|Reads/Writes| PG_UM
    ING_P -->|"AMQP"| Rabbit
    Rabbit -->|"AMQP"| ING_W
    ING_W -->|"SPARQL HTTP"| GraphDB
    QRY -->|"SPARQL HTTP"| GraphDB
    CHAT -->|"SPARQL HTTP"| GraphDB
    CHAT -->|Reads/Writes| PG_CHAT
    ML -->|"SPARQL HTTP"| GraphDB
    ML -->|Reads/Writes| PG_ML

    %% Build-time artifacts
    ING_P -.->|"Uses"| SPARQL
    ING_P -.->|"Uses"| LINKML
    QRY -.->|"Uses"| SPARQL

    %% Click events
    click APItoken "https://github.com/sensein/brainkb/tree/main/APItokenmanager/"
    click ING_P "https://github.com/sensein/brainkb/tree/main/ingestion_service/producer/"
    click ING_W "https://github.com/sensein/brainkb/tree/main/ingestion_service/worker/"
    click QRY "https://github.com/sensein/brainkb/tree/main/query_service/"
    click UM "https://github.com/sensein/brainkb/tree/main/usermanagement_service/"
    click CHAT "https://github.com/sensein/brainkb/tree/main/chat_service/"
    click ML "https://github.com/sensein/brainkb/tree/main/ml_service/"
    click GraphDB "https://github.com/sensein/brainkb/tree/main/graphdb/"
    click Oxigraph "https://github.com/sensein/brainkb/tree/main/oxigraph/"
    click Rabbit "https://github.com/sensein/brainkb/tree/main/rabbit-mq/"
    click SPARQL "https://github.com/sensein/brainkb/tree/main/sparql_queries/"
    click LINKML "https://github.com/sensein/brainkb/tree/main/linkml-models/"

    %% Styles
    classDef api fill:#D0E6FF,stroke:#0366d6,color:#0366d6
    classDef worker fill:#FFF4C1,stroke:#d69f00,color:#d69f00
    classDef auth fill:#E8F6EF,stroke:#28a745,color:#28a745
    classDef client fill:#F0F0F0,stroke:#333,color:#333
    classDef mq fill:#FFEFE6,stroke:#d73a49,color:#d73a49
    classDef db fill:#E6FFFA,stroke:#1f6feb,color:#1f6feb
    classDef art fill:#F5F5F5,stroke:#6a737d,color:#6a737d
```
