How kaizer-1/company-brain works
1. System boundary and purpose
- This repo is a full-stack “company brain” demo: a knowledge graph system that ingests raw company events, reconciles them into graph state, and answers grounded questions with citations.
- The architecture splits cleanly into a React frontend, a FastAPI backend, and two persistence layers: Neo4j for graph traversal and Postgres for event/audit/vector storage.
2. Frontend boundary
- The frontend is a React 18 + Vite + TypeScript app served behind Nginx in Docker.
- Main user entry points are /ask, /graph, /search, /ingest, and /audit.
- It does not own business logic; it primarily calls backend APIs and renders graph views, ingestion progress, search results, and agent traces.
3. Backend API boundary
- The backend is an async FastAPI service.
- Its major API areas are:
- agent endpoints for question answering and streaming,
- ingestion endpoints for reconciling new events,
- graph/query endpoints for typed graph reads,
- search endpoints for hybrid retrieval,
- audit/metrics endpoints for operational visibility.
- app/main.py wires app startup, middleware, and routing.
4. Two-store persistence model
- Neo4j is the system of record for graph structure: people, teams, services, systems, decisions, and their typed relationships.
- Postgres is the system of record for raw events, embeddings, ingestion run history, and merge/audit decisions.
- The split is architectural: multi-hop traversal belongs in the graph DB; event history, embeddings, and receipts belong in relational storage.
5. Ingestion pipeline
- New documents/messages enter through the ingestion orchestrator as raw events.
- The pipeline is staged and incremental: extraction, embedding, entity resolution, consolidation/projecting, temporal enrichment, contradiction detection, then final indexing/recording.
- Each stage writes to one or both stores and is designed to be idempotent, so repeated ingestion of the same event should not duplicate graph state.
6. Extraction and graph writing
- Extraction uses an external LLM via OpenRouter to turn raw text into structured entities, relationships, and evidence quotes.
- Parsed extraction output is validated, then written into Neo4j using MERGE-based graph writes.
- Extraction is provenance-heavy: graph nodes/edges carry references back to the originating event(s).
7. Entity resolution, consolidation, and temporal modeling
- Resolution is a 3-tier process: deterministic auto-merge, LLM adjudication for ambiguous matches, and “leave separate” below threshold.
- Consolidation is non-destructive: losers point to winners via merge relationships, and graph reachability is preserved by projecting edges onto canonical nodes.
- Temporal enrichment adds validity windows and supersession links so the graph can answer “what changed when” questions, not just current-state questions.
8. Contradiction detection
- A dedicated contradiction pass compares new messages against active decisions.
- It is scoped to newly created fragments rather than re-evaluating the entire corpus every time.
- Detected contradictions become graph facts, enabling queries about conflicts between discussion and decisions.
9. Search subsystem
- Search is hybrid: vector similarity from pgvector is blended with graph-density signals from the event’s extracted structure.
- Embeddings are generated locally using bge-small-en-v1.5 and stored in Postgres.
- Search reads from both Postgres and Neo4j, unlike pure graph queries which only read Neo4j.
10. Agent subsystem
- The question-answering agent is a LangGraph state machine that routes a user question to one of a fixed set of typed tools.
- It does not generate Cypher at runtime; instead it calls typed Python query functions and structural graph tools.
- After tool execution, the agent synthesizes an answer and verifies that every citation is grounded before returning it.
11. Query engine
- The backend includes specialized query modules for the “killer queries” described in the README: multi-hop ownership, temporal contradiction, blast radius, and change tracking.
- These are implemented as typed graph traversals over Neo4j, with explicit result schemas.
- This is the core reason the graph exists: the app needs typed multi-hop logic that document retrieval cannot express reliably.
12. Observability and auditability
- Every ingestion run is recorded, including timing and stage status.
- Merge decisions are also auditable, with reasoning and confidence preserved in Postgres.
- Metrics are exposed through a lightweight in-memory registry for demo-scale monitoring, while durable history lives in the database.
13. Synthetic corpus and test harness
- The repository is built around deterministic synthetic data generation, not live enterprise integrations.
- backend/app/synthetic seeds the corpus; scripts drive extraction, embedding, resolution, evals, and graph resets.
- The test suite covers backend APIs, ingestion behavior, resolution logic, extraction, search, queries, streaming, and frontend UI components.
14. External dependencies and runtime shape
- OpenRouter is the external LLM gateway used for extraction, routing, synthesis, and adjudication.
- Runtime is containerized with Docker Compose, typically bringing up Neo4j, Postgres, backend, and frontend together.
- The codebase is Python 3.12 on the backend, with React/TanStack Query/Tailwind on the frontend, but the architectural distinction that matters most is graph reasoning vs. event/audit storage.
Main entry points: api HTTP API · ingestion Ingestion · tests Test suite · search Search
Components 21
Frontend
React UI for asking questions, browsing graph/search, ingesting data, and reviewing audit state.
- React app · Vite + React 18 + TypeScript frontend served through Nginx.
frontend/src/App.tsx - Pages · Main user entry points for Ask, Graph, Search, Ingest, Audit, and Queries.
frontend/src/pages - API client · Typed browser clients for ask, graph, search, ingest, audit, and queries.
frontend/src/api - UI components · Presentation components for graph canvas, agent traces, ingestion progress, search cards, and audit tabs.
frontend/src/components
Backend
FastAPI service with agent, ingestion, query, search, resolution, temporal, and contradiction subsystems.
- App entry · FastAPI startup, middleware, routing, and service wiring.
backend/app/main.py - HTTP API · Public endpoints for health, events, graph, queries, search, audit, metrics, and ingestion.
backend/app/api - Agent · Question-answering state machine that routes typed tools, synthesizes answers, and verifies citations.
backend/app/agent - Query engine · Specialized Neo4j reads for multihop ownership, contradictions, blast radius, change tracking, and neighbors.
backend/app/queries - Ingestion · Incremental reconciliation pipeline that stages extraction, resolution, temporal enrichment, contradiction detection, and indexing.
backend/app/ingestion - Extraction · OpenRouter-backed structured extraction from raw text into entities, relationships, evidence, and graph writes.
backend/app/extraction - Resolution · Tiered entity matching, adjudication, consolidation, and projection onto canonical graph nodes.
backend/app/resolution - Temporal · Validity windows and supersession logic for answering change-over-time questions.
backend/app/temporal - Contradiction · Scoped pass that compares new messages against active decisions and records conflicts as graph facts.
backend/app/contradiction - Search · Vector-plus-graph retrieval using local embeddings in Postgres and structural signals from Neo4j.
backend/app/search - Metrics · Lightweight in-memory metrics for demo-scale monitoring and operational visibility.
backend/app/observability
Storage
Neo4j graph store and Postgres event/audit/vector store.
- Neo4j · System of record for typed nodes, relationships, projections, and traversal-oriented queries.
- Postgres · System of record for raw events, embeddings, ingestion runs, merge audits, and related history.
Ops
Docker, migrations, scripts, synthetic data, and evaluation harness.
- Synthetic data · Deterministic company corpus seeding and narrative generation used for demos and tests.
backend/app/synthetic - Migrations · Alembic and Cypher migrations for relational schema, graph constraints, and indexes.
backend/alembic - Scripts · Entry scripts for seeding, extraction, embedding, resolution, evals, and graph resets.
backend/scripts - Test suite · Backend and frontend tests covering APIs, ingestion, resolution, search, queries, streaming, and UI behavior.
backend/tests
Connections 39
- React app → Pages: renders · Top-level app mounts route pages and navigation.
- React app → UI components: composes · Pages are built from shared UI and domain components.
- Pages → API client: calls · Route views invoke typed API clients.
- UI components → API client: reads · Components display data returned by backend endpoints.
- App entry → HTTP API: wires · Application startup registers the service routes.
- HTTP API → Agent: exposes · Agent endpoints support question answering and streaming.
- HTTP API → Ingestion: exposes · Ingestion endpoints accept raw events and reconciliation requests.
- HTTP API → Query engine: exposes · Graph/query endpoints serve typed read models.
- HTTP API → Search: exposes · Search endpoints provide hybrid retrieval.
- HTTP API → Contradiction: exposes · Audit and operational routes surface contradiction-related state.
- HTTP API → Metrics: exposes · Metrics endpoints report runtime health and counters.
- Agent → Query engine: tools · The agent uses typed query functions rather than generating Cypher at runtime.
- Agent → Search: tools · The agent can retrieve supporting context through search tools.
- Agent → HTTP API: streams · Streaming responses are served through dedicated API routes.
- Ingestion → Extraction: stage · Ingestion begins with structured extraction from raw inputs.
- Ingestion → Resolution: stage · Extracted fragments are resolved and consolidated into canonical entities.
- Ingestion → Temporal: stage · Temporal enrichment annotates validity and supersession.
- Ingestion → Contradiction: stage · A scoped contradiction pass compares new messages to active decisions.
- Extraction → Neo4j: writes · Validated extraction output is merged into the graph store.
- Extraction → Postgres: records · Extraction provenance and supporting data are persisted relationally.
- Resolution → Neo4j: projects · Canonical nodes and merged relationships are written to Neo4j.
- Resolution → Postgres: audits · Merge decisions, confidence, and reasoning are stored durably.
- Temporal → Neo4j: annotates · Validity windows and supersession links enrich graph facts.
- Contradiction → Neo4j: adds facts · Detected contradictions become queryable graph state.
- Search → Postgres: reads · Hybrid retrieval consumes embeddings and event records from Postgres.
- Search → Neo4j: reads · Search uses graph structure signals from Neo4j.
- Query engine → Neo4j: reads · Typed traversals execute against the graph database.
- Metrics → Postgres: persists history · Durable run history lives in the database even when metrics are in-memory.
- Synthetic data → Ingestion: feeds · Deterministic demo corpus drives the ingestion pipeline.
- Migrations → Postgres: migrates · Alembic manages relational schema evolution.
- Migrations → Neo4j: constraints · Cypher migrations define graph constraints and indexes.
- Scripts → Synthetic data: seeds · Scripts generate and load synthetic company data.
- Scripts → Ingestion: runs · Scripts drive extraction, embedding, resolution, and ingestion workflows.
- Scripts → Query engine: evaluates · Helper scripts execute killer-query and query-eval workflows.
- Test suite → HTTP API: covers · API tests validate routes and contracts.
- Test suite → Ingestion: covers · Ingestion tests validate idempotency, stages, and orchestration.
- Test suite → Resolution: covers · Resolution tests cover adjudication, consolidation, and projection.
- Test suite → Search: covers · Search tests validate embedding and retrieval behavior.
- Test suite → Query engine: covers · Query tests validate typed traversal results.