How lucasyzc/graphragx works
1. Overall shape
- GraphRAGX is a FastAPI-based backend with a thin server-rendered web console on top.
- The architecture is centered on three durable stores: PostgreSQL for system state, Neo4j for symbol graph relationships, and Qdrant for vector retrieval.
- Repository ingestion, knowledge ingestion, query answering, and impact analysis are separate flows that converge in the service layer.
2. Presentation boundary
- The UI lives under app/web as Jinja templates plus static JavaScript/CSS.
- Pages like Chat Home, Search, Project Management, Sync Management, Knowledge Management, and Impact Scenario call the API rather than owning business logic.
- This keeps the browser layer as a control surface for project creation, sync triggering, search, and answer review.
3. API layer
- app/api/routes splits the API by concern: projects, sync/jobs, query/chat, knowledge, symbols, analysis, web, and health.
- app/api/router.py and app/api/deps.py act as the composition point and dependency boundary.
- The API also carries a simple actor context through headers such as X-User and X-Role, which is the current RBAC scaffold.
4. Domain and persistence model
- PostgreSQL is the system of record for projects, sync jobs, chat sessions/messages, knowledge sources, memories, and related metadata.
- SQLAlchemy models live in app/db/models.py, with sessions and base wiring in app/db/session.py and app/db/base.py.
- Alembic migrations define the schema evolution; the current revisions show the initial baseline and later additions for project instructions and memories.
5. Sync and indexing pipeline
- The repository sync flow is the main ingestion path for codebases.
- app/services/scm_service.py handles local or Git-backed source access, including checkout into a workspace for remote repositories.
- app/services/sync_service.py orchestrates full and incremental sync, with incremental mode driven by base_sha, head_sha, and since_sha, falling back to full indexing when those are unavailable.
- Sync execution is tracked as jobs, and duplicate queued/running syncs are rejected.
6. Canonical intermediate representation
- Parsed code is normalized into a language-neutral IR under app/ir.
- Extractors exist for Python, Java, and C# under app/ir/extractors, with Python using AST-based extraction and Java/C# using lighter structural matching.
- This IR is the bridge between raw source files and downstream graph/vector persistence.
7. Graph layer
- Neo4j stores symbol nodes and relations such as CONTAINS, CALLS, IMPORTS, and REFERENCES.
- app/graph/repository.py is the graph persistence boundary, used primarily by sync and analysis services.
- The graph is not just a mirror of code; it powers traversal-based features like impact analysis and symbol-centric lookup.
8. Vector layer
- Qdrant stores embedded chunks for semantic retrieval.
- app/vector/repository.py and app/vector/types.py define the vector-store boundary.
- Symbol snippets and knowledge document chunks are embedded and upserted here, making vector search a shared retrieval substrate for both code and docs.
9. Embedding and chat gateways
- app/services/embedding_service.py supports local_hash for deterministic/local behavior and openai_compatible for external embedding APIs.
- app/services/chat_service.py similarly abstracts grounded answer generation with none and openai_compatible modes.
- This design isolates model/provider choice from the rest of the system, so retrieval and orchestration stay stable even when the LLM backend changes.
10. Query, retrieval, and chat flow
- app/services/retrieval_service.py and app/services/chunking_service.py unify search across vector chunks and metadata-backed sources.
- POST /query is the primary retrieval endpoint and can return contextual snippets plus retrieval metadata.
- Chat sessions are persisted separately from queries, so the system supports conversational state while still grounding each answer in retrieved evidence.
11. Knowledge ingestion as a second content source
- Knowledge sources are treated as first-class inputs, separate from source-code repositories.
- They support local directory and HTTP ingestion, then split supported documents into document_chunks with ACL scoping.
- This makes the architecture a two-lane RAG system: code repositories on one side, external/internal documentation on the other, both feeding the same retrieval stack.
12. Impact analysis
- app/services/impact_service.py uses the symbol graph to expand from a starting symbol or scope through graph hops.
- The dedicated /analysis/impact route and scenario page use Neo4j relations to answer “what breaks if this changes?” style questions.
- This is one of the clearest examples of why the graph store exists separately from the vector store.
13. Operational control plane
- app/services/job_service.py tracks async work and stale-job recovery.
- app/services/runtime_check.py enforces startup requirements: PostgreSQL, Neo4j, and Qdrant must be reachable when required, and SQLite is blocked outside test usage.
- app/services/audit_service.py and app/services/project_service.py suggest a central governance layer around project-level actions.
14. Execution environment and test posture
- Docker Compose packages the API with PostgreSQL, Neo4j, and Qdrant for local deployment.
- Tests cover sync, indexing, retrieval, chat, knowledge, web routes, SCM behavior, and incremental behavior, indicating that the main architectural seams are already enforced by automated tests.
- The repo is therefore structured as an integration-heavy application where the service layer is the real center of gravity.
Main entry points: router.py API router · indexing_service.py Indexing service · models.py DB models · retrieval_service.py Retrieval service
Components 31
API Layer
FastAPI routing, dependency wiring, and request boundaries.
- App entry · Application startup and server wiring.
app/main.py - API router · Central API assembly and dependency boundary.
app/api/router.py - API deps · Shared dependencies and actor context plumbing.
app/api/deps.py - Web routes · Thin server-rendered pages for console actions.
app/api/routes/web.py - Projects API · Project lifecycle and governance endpoints.
app/api/routes/projects.py - Sync API · Sync triggering, job control, and related operations.
app/api/routes/jobs.py - Query API · Primary retrieval and answer request endpoints.
app/api/routes/query.py - Chat API · Chat session and answer interaction endpoints.
app/api/routes/chat.py - Knowledge API · Knowledge source ingestion and management endpoints.
app/api/routes/knowledge.py - Symbols API · Symbol lookup and graph-oriented access endpoints.
app/api/routes/symbols.py - Analysis API · Impact analysis and graph expansion endpoints.
app/api/routes/analysis.py
Services
Orchestration, ingestion, retrieval, and analysis workflows.
- Project service · Project state, setup, and governance logic.
app/services/project_service.py - Sync service · Full and incremental repository sync coordination.
app/services/sync_service.py - SCM service · Local or Git-backed source checkout and file access.
app/services/scm_service.py - Indexing service · Transforms source into graph and vector-ready artifacts.
app/services/indexing_service.py - Retrieval service · Unifies vector and metadata-backed retrieval.
app/services/retrieval_service.py - Chat service · Grounded answer generation with provider abstraction.
app/services/chat_service.py - Knowledge service · Knowledge source ingestion and chunk preparation.
app/services/knowledge_service.py - Impact service · Graph traversal for change impact analysis.
app/services/impact_service.py - Job service · Job tracking, duplicate suppression, and recovery.
app/services/job_service.py - Embedding service · Embedding provider abstraction for local and external modes.
app/services/embedding_service.py - Runtime check · Enforces required backing-store availability.
app/services/runtime_check.py
Persistence
Database and external store boundaries.
- DB models · System-of-record schema for projects, jobs, chat, and knowledge.
app/db/models.py - DB session · Database session and engine configuration.
app/db/session.py - Graph repo · Persistence boundary for symbol nodes and relations.
app/graph/repository.py - Vector repo · Persistence boundary for embedded chunks and semantic search.
app/vector/repository.py
Domain Model
Schemas and normalized intermediate representation.
- IR models · Language-neutral intermediate representation for code.
app/ir/models.py - Python extractor · Python source extraction into the shared IR.
app/ir/extractors/python_extractor.py - Java extractor · Java source extraction into the shared IR.
app/ir/extractors/java_extractor.py - C# extractor · C# source extraction into the shared IR.
app/ir/extractors/csharp_extractor.py - Schemas · Request and response models for API boundaries.
app/schemas
Connections 45
- App entry → API router: mounts · Application startup composes the API surface.
- App entry → Runtime check: checks · Startup validates required infrastructure.
- API router → API deps: uses · Routes rely on shared dependency wiring.
- API router → Projects API: includes · Projects concern is routed into the API.
- API router → Sync API: includes · Sync and job control routes are registered.
- API router → Query API: includes · Retrieval endpoints are registered.
- API router → Chat API: includes · Conversation endpoints are registered.
- API router → Knowledge API: includes · Knowledge management endpoints are registered.
- API router → Symbols API: includes · Symbol access endpoints are registered.
- API router → Analysis API: includes · Impact analysis endpoints are registered.
- API router → Web routes: includes · Server-rendered console routes are registered.
- API router → Schemas: contracts · API endpoints use shared request and response models.
- Web routes → Project service: calls · UI actions delegate to project operations.
- Web routes → Sync service: calls · UI actions trigger sync workflows.
- Web routes → Retrieval service: calls · Search and review pages use retrieval.
- Projects API → Project service: calls · Project endpoints are backed by project orchestration.
- Sync API → Sync service: calls · Sync endpoints start ingestion flows.
- Sync API → Job service: calls · Job lifecycle and duplicate control are centralized.
- Query API → Retrieval service: calls · Primary retrieval endpoint is serviced here.
- Chat API → Chat service: calls · Conversation handling delegates to answer generation.
- Chat API → Retrieval service: calls · Chat is grounded through shared retrieval.
- Knowledge API → Knowledge service: calls · Knowledge ingestion and management flow through this service.
- Symbols API → Graph repo: calls · Symbol lookup is backed by graph persistence.
- Analysis API → Impact service: calls · Impact analysis requests use graph traversal logic.
- Project service → DB models: persists · Projects and governance state live in PostgreSQL.
- Job service → DB models: persists · Job state and recovery metadata are stored in PostgreSQL.
- Chat service → DB models: persists · Chat sessions and messages are stored in PostgreSQL.
- Knowledge service → DB models: persists · Knowledge sources and metadata are stored in PostgreSQL.
- Retrieval service → Vector repo: queries · Semantic retrieval uses the vector store.
- Retrieval service → DB models: reads · Metadata-backed search also uses relational state.
- Chat service → Embedding service: uses · Answers rely on embeddings and provider abstraction.
- Knowledge service → Embedding service: uses · Document chunks are embedded before upsert.
- Sync service → SCM service: uses · Sync reads code through source checkout and file access.
- Sync service → Indexing service: orchestrates · Sync delegates parsing and indexing stages.
- Indexing service → IR models: produces · Source is normalized into the shared IR.
- Indexing service → Python extractor: uses · Python code is extracted via AST-based parsing.
- Indexing service → Java extractor: uses · Java code is extracted via structural matching.
- Indexing service → C# extractor: uses · C# code is extracted via structural matching.
- Indexing service → Graph repo: writes · Symbol relations are persisted to Neo4j.
- Indexing service → Vector repo: writes · Chunks and snippets are upserted to Qdrant.
- Indexing service → Embedding service: uses · Vector preparation depends on embeddings.
- Impact service → Graph repo: traverses · Impact analysis expands over graph relationships.
- Knowledge service → Vector repo: writes · Knowledge chunks are indexed into the shared vector layer.
- Runtime check → DB session: verifies · Startup checks backing-store connectivity.
- DB session → DB models: supports · Session wiring serves the relational model.