How aaroncao06/projectnexus works
1. Overall shape
ProjectNexus is split into three runtime layers plus an offline preparation layer: an ingestion/authoring agent, a read/query backend, and a React frontend, all centered on a Neo4j graph. RAG and summarization are additive services that depend on Pinecone and OpenRouter.
2. Write path: the Agent
The agent/ package is the graph-building and indexing pipeline. It ingests email chains from CSV, synthetic data, or sample data, normalizes names through alias generation, extracts people/relationships, and writes the resulting entities into Neo4j. It also has separate flows for generating edge summaries and building Pinecone embeddings for later retrieval.
3. Graph data model
The core graph appears to be intentionally simple: Person nodes connected by COMMUNICATES_WITH edges. Edges can carry summary/commentary metadata, which becomes important for the summarization and RAG layers. This makes Neo4j the system of record for relationship structure.
4. Read path: the Backend
The backend/ package is a FastAPI service that reads from Neo4j and exposes graph-oriented APIs to the UI. It is not the primary writer; it acts as a query and analysis layer over the graph, plus a gateway to RAG and LLM-powered features.
5. Backend submodules and responsibilities
The backend is functionally split into:
- db.py / config.py: graph and service configuration
- vectorstore.py / rag.py: Pinecone-backed retrieval
- llm.py: OpenRouter integration
- clustering.py: graph clustering/reclustering logic for visualization
- insights.py: anomaly, bridge, and centrality analysis
- main.py: HTTP surface for graph, metadata, summarization, query, and insights endpoints
6. Frontend boundary
The frontend/ app is a Vite + React + TypeScript client. It consumes the backend over HTTP via api.ts, renders the graph in GraphView.tsx, and provides a chat/QA surface in ChatPanel.tsx. The UI is a pure consumer of backend APIs; it does not talk directly to Neo4j or Pinecone.
7. Visualization flow
The graph UI is driven by backend responses that include nodes, edges, and metadata. The backend can return either the full graph or a person-centered subgraph with configurable depth, and can optionally recluster nodes to improve layout/legibility before the frontend renders them.
8. RAG flow
RAG is a separate retrieval path: user questions from the frontend go to the backend /query endpoint, the backend embeds the query, searches Pinecone, and then sends retrieved context to OpenRouter for answer generation. This is the main place where unstructured email content is turned into natural-language answers.
9. Insights flow
The backend also produces graph analytics beyond raw traversal: structural anomalies, bridging nodes, and centrality-style insights. These can be returned as computed metrics or generated through LLM-backed commentary, depending on the endpoint used. This sits between pure graph querying and generative assistance.
10. Offline data preparation
Notebook_Data/ contains Jupyter notebooks that preprocess source email corpora into downstream assets. One notebook chunks large Enron-style email data into parquet files for later embedding/indexing; another works with a structured Epstein email JSON-like dataset. These notebooks feed both the graph-ingestion and RAG pipelines.
11. External services and operational dependencies
Three external systems materially shape the architecture:
- Neo4j for graph persistence
- Pinecone for semantic retrieval
- OpenRouter for summarization, answer generation, and insight narration
The app can still do basic graph exploration without the LLM/RAG services, but those features depend on environment configuration.
12. What to show in an architecture diagram
A clear diagram should place Neo4j in the center, with the Agent as the write-side producer, the Backend as the read/analysis API, and the Frontend as the client. Off to the side, show Pinecone + OpenRouter behind backend RAG/LLM calls, and show Notebook preprocessing as an offline source feeding the agent/indexing pipeline.
Main entry points: backend/main.py API · agent/main.py Agent · rag.py RAG · graph.py Graph build
Components 21
Offline prep
Notebooks and source data preparation
- Notebooks · Jupyter workflows that chunk and prepare source email corpora
Notebook_Data - Email data · Raw or structured email datasets used to build graph and retrieval assets
Notebook_Data/epstein_emails_insights_all.txt
Write path
Ingestion, normalization, and indexing pipeline
- Agent · Entry point for graph-building and indexing workflows
agent/main.py - Aliases · Name normalization and alias generation for entity resolution
agent/generate_aliases.py - Graph build · Creates people and communication relationships in the graph store
agent/graph.py - Edge index · Extracts relationship chains and writes communication edges
agent/index_relationships.py - Embeddings · Builds embeddings for later semantic retrieval
agent/embeddings.py - Pinecone client · Agent-side connector for semantic index writes
agent/pinecone_client.py
Read path
FastAPI query, analysis, and RAG service
- API · HTTP surface for graph, query, summarization, and insights
backend/main.py - DB layer · Neo4j connectivity and graph read helpers
backend/db.py - Clustering · Reclusters graph data for visualization and legibility
backend/clustering.py - Insights · Centrality, bridge, and anomaly-style analysis
backend/insights.py - RAG · Embeds questions and retrieves context for answers
backend/rag.py - Vector store · Pinecone-backed search helpers used by the backend
backend/vectorstore.py - LLM · OpenRouter integration for answers and commentary
backend/llm.py
UI
React client for graph exploration and chat
- Graph view · Renders nodes, edges, and metadata returned by the backend
frontend/src/GraphView.tsx - Chat panel · User-facing question and answer surface
frontend/src/ChatPanel.tsx - API client · Frontend adapter for backend requests
frontend/src/api.ts
External services
Persistence, retrieval, and LLM dependencies
- Neo4j · System of record for people and communication relationships
- Pinecone · Semantic retrieval store for RAG
- OpenRouter · Generates answers, summaries, and insight narration
Connections 22
- Notebooks → Email data: prepare · Offline notebooks transform source email corpora into downstream assets
- Email data → Agent: feed · Prepared corpora are consumed by the agent pipeline
- Agent → Aliases: normalize · Agent resolves names and aliases before graph construction
- Aliases → Graph build: resolve · Normalized identities are written as graph entities and relationships
- Agent → Edge index: extract · Agent derives communication chains and relationship edges
- Edge index → Graph build: write · Indexed relationships are persisted into the graph store
- Agent → Embeddings: index · Agent prepares semantic embeddings for retrieval
- Embeddings → Pinecone client: store · Embeddings are sent to the vector index client
- Graph build → Neo4j: persist · People and communication edges are stored in Neo4j
- Pinecone client → Pinecone: upsert · Semantic vectors are written to Pinecone
- API client → API: request · Frontend calls backend HTTP endpoints
- API → DB layer: read graph · Backend reads graph data through its DB layer
- DB layer → Neo4j: query · Neo4j powers graph traversal and subgraph responses
- API → Clustering: recluster · Backend can reshape graph data for visual clarity
- API → Insights: analyze · Backend computes graph-centric insights and anomalies
- API → RAG: answer · Backend routes user questions through the RAG flow
- RAG → Vector store: retrieve · Questions are embedded and matched against the vector store
- Vector store → Pinecone: search · Pinecone provides semantic retrieval context
- RAG → LLM: generate · Retrieved context is turned into a natural-language answer
- LLM → OpenRouter: invoke · OpenRouter generates summaries, answers, and commentary
- API → Graph view: serve graph · Graph responses power the visualization layer
- API → Chat panel: serve qa · Chat responses power the conversational UI