abhi9199-tech42/scdl-rag-enhanced-architecture

This repository is a hybrid monorepo for a contradiction-aware RAG system. The main production-facing code is in src/ (TypeScript), while api/ and embedding/ contain Python service components, and isre/, urcm/, and reference/ hold reference implementations and migration/validation material.

1. External entry points

The system is consumed by Python and JavaScript SDKs (sdk/python, sdk/javascript), direct HTTP clients, and the test/load scripts. The documented API surface is a detection service that accepts a question, answer, and source documents, then returns a hallucination verdict, score, and explanation.

2. HTTP/API boundary

The TypeScript server in src/api/ is the main request edge: server.ts, routes.ts, middleware/auth.ts, middleware/rateLimit.ts, validation.ts, swagger.ts, and metrics.ts define request handling, auth, throttling, schema validation, API docs, and observability. Async endpoints are part of this boundary as well.

3. Orchestration layer

src/system/core.ts and src/system/performance.ts coordinate runtime behavior and performance-sensitive paths. This is the point where request-level work is handed off to retrieval, context assembly, and contradiction analysis, and where async/background processing is likely coordinated.

4. Context and retrieval pipeline

src/context/assembler.ts and src/retrieval/engine.ts form the evidence-preparation path. Documents and query material are normalized, assembled into a working context, and then scored/retrieved before the reasoning layers operate on them.

5. Storage and persistence

src/storage/ provides two backing modes (memory_store.ts and file_store.ts) plus deduplication logic. The checked-in data/vector_store.json indicates a local vector/cache artifact rather than a separate database service, so persistence is lightweight and embedded in-process or file-backed.

6. Hallucination/contradiction reasoning

The core reasoning subsystems are split into src/isre/ and src/urcm/. src/isre/bridge.ts, src/isre/processor.ts, and the compression/graph/multilang modules implement semantic reasoning, while src/urcm/processor.ts and src/urcm/contradiction/* handle contradiction detection and resolution. This is the main analytic boundary of the system.

7. Explainability and audit trail

src/audit/explainable.ts and src/audit/trail.ts capture why a result was produced, not just the result itself. This is important because the product’s output is meant to be human-readable and auditable, especially for detected contradictions and auto-fix suggestions.

8. Shared primitives and utilities

Common types, IDs, logging, and vector utilities live in src/types, src/utils/id.ts, src/utils/logger.ts, and src/utils/vector.ts. These are cross-cutting support modules used by API, retrieval, storage, and reasoning code.

9. Python service components

The api/ and embedding/ directories are separate Python service runtimes with their own requirements and Dockerfiles. The README describes them as the API service and the embedding/NLI inference service, so in deployment diagrams they should be treated as distinct services connected over HTTP rather than as libraries.

10. Reference implementations and compatibility layers

isre/, urcm/, and the large reference/ subtree mirror the same architecture in Python and include validation suites, docs, and examples. These are useful for understanding intent and for migration/benchmarking, but they are not the only runtime path.

11. Offline evaluation and performance harness

experiments/ and loadtest/ contain benchmark runners, datasets, CSV/MD reports, and pressure tests. These are an offline analytics boundary: they exercise the API and model services to measure recall, throughput, endurance, and auto-fix behavior.

12. Deployment and packaging

Docker is first-class (docker/api.Dockerfile, docker/embedding.Dockerfile, docker/docker-compose.yml), and CI is defined in .github/workflows/ci.yml. The repo also ships docs/landing pages and SDK packaging, so the architecture spans service runtime, client libraries, and documentation/static site delivery.

In diagram terms, the key flow is: client/SDK → API gateway/server → validation/auth/rate limit → context assembly/retrieval/storage → ISRE/URCM reasoning and contradiction detection → audit/explanation → response, with a separate embedding/inference service and offline evaluation/load-testing tooling around it.