How thamizh-arasu/contextiva-prototype-bmad works
Contextiva is organized as a layered, domain-driven Python service with a single FastAPI application at the edge, a use-case/application layer in the middle, domain models as the business core, and infrastructure adapters for persistence, cache, crawling, and LLM providers.
What the system is
The repository implements a knowledge engine for AI agents. Its main purpose is to ingest project documents and external content, store and version that knowledge, and expose it through:
• a REST API under /api/v1
• a planned/partial MCP integration surface for agent tooling
• RAG-oriented search and synthesis workflows
The system is not a general CMS; it is centered around knowledge organization for projects and retrieval for downstream AI consumers.
Main runtime boundary
The runtime entry point is the FastAPI app in src/api/main.py. Around that edge are:
• src/api/v1/routes/* for HTTP endpoints
• src/api/middleware/logging_middleware.py for request tracing / structured logging
• src/api/dependencies.py for dependency injection into routes
• src/api/v1/schemas/* for request/response validation and API contracts
This API layer is the external contract for clients, including human users, scripts, and AI agents.
Core layers and responsibilities
1) API layer
This layer is purely transport and validation:
• routes for auth, projects, documents, tasks, knowledge, and rag
• Pydantic schemas for each resource
• middleware for logging and request correlation
The API layer translates HTTP requests into application use cases and returns typed responses. It should not contain domain rules.
2) Application layer
This is the orchestration layer that coordinates workflows across domain objects and infrastructure.
It is split into:
• src/application/use_cases/projects/create_project.py
• src/application/use_cases/knowledge/query_knowledge.py
• src/application/use_cases/ingest_knowledge.py
• src/application/use_cases/crawl_knowledge.py
and reusable services in:
• src/application/services/text_extractor.py
• src/application/services/text_chunker.py
• src/application/services/hybrid_search_service.py
• src/application/services/reranking_service.py
• src/application/services/synthesis_service.py
This layer owns multi-step business flows such as:
• creating projects
• ingesting documents or crawled content
• breaking text into chunks
• searching and reranking knowledge
• synthesizing an answer for RAG queries
3) Domain layer
The domain layer contains the core business entities:
• project
• document
• task
• knowledge
• user
These are the stable concepts around which the application is built. The README and schema/migration names indicate that:
• projects are the root organizing concept
• documents are versioned source artifacts
• tasks are project-scoped work items
• knowledge items are indexed/searchable derived records
• users support auth and ownership
This is the business boundary that should remain independent of FastAPI, the database, Redis, or any provider vendor.
4) Infrastructure layer
Infrastructure provides concrete implementations for persistence and external systems:
Database repositories
Under src/infrastructure/database/repositories/ there are repositories for:
• projects
• documents
• tasks
• knowledge
• users
These are the persistence adapters that isolate the rest of the code from PostgreSQL details.
Cache
src/infrastructure/cache/redis_cache.py is the Redis adapter. Cache is optional according to configuration, but it is part of the performance architecture for repeated retrievals and possibly RAG lookups.
External integrations
• src/infrastructure/external/crawler/crawler_client.py handles crawling external URLs or documentation sources
• src/infrastructure/external/llm/provider_factory.py selects among LLM backends
• provider implementations exist for:
- OpenAI
- Anthropic
- Ollama
- OpenRouter
This external boundary is important because RAG, synthesis, and ingestion depend on third-party AI services and web sources.
5) Shared cross-cutting utilities
src/shared/ contains reusable platform concerns:
• shared/config/settings.py for environment-driven configuration
• shared/config/logging.py for log setup
• shared/infrastructure/database/connection.py for the async database pool
• shared/utils/security.py for auth/security helpers
• shared/utils/errors.py for app-wide error handling
• shared/utils/versioning.py for semantic/version-related helpers
These are not business features; they are shared operational concerns used across layers.
Main data flows
Project management flow
Request path:
• API route in projects
• application use case, especially create project
• project repository
• PostgreSQL
Projects act as the anchor for other entities. Documents, tasks, and knowledge records are all organized in relation to a project, so the project boundary is central to the data model.
Document ingestion and versioning flow
Request path:
• API route in documents or knowledge upload/crawl entrypoints
• document-related application logic
• text extraction and chunking services
• document repository and knowledge repository
• PostgreSQL, with pgvector-enabled tables for retrieval-oriented storage
Documents are treated as source artifacts, while knowledge items are the retrievable derived records. That separation is important for diagramming the system: raw content and searchable knowledge are distinct storage concerns.
Knowledge crawl / upload flow
Request path:
• /api/v1/knowledge/crawl or /api/v1/knowledge/upload
• crawl_knowledge or ingest_knowledge use case
• crawler client or file/text extraction service
• chunking service
• persistence into document/knowledge tables
• later retrieval through search services
This flow bridges external content sources into the internal knowledge index.
RAG query flow
Request path:
• /api/v1/rag/query
• knowledge query use case
• hybrid search service
• optional reranking service
• synthesis service
• knowledge repository and database search
• optional Redis cache
• LLM provider for answer generation or synthesis
This is the most important end-user AI workflow. The architecture supports a layered retrieval pipeline rather than a single database lookup:
1. retrieve candidate knowledge
2. combine semantic and keyword signals
3. rerank if enabled
4. synthesize a final response if enabled
Authentication and security flow
There is an auth route and security utilities under shared/utils/security.py. The README describes JWT-based authentication and RBAC support/stubs. In architecture terms, auth is a cross-cutting boundary that protects the API layer and is enforced through dependencies rather than being embedded in business logic.
Persistence and schema boundary
The database is PostgreSQL with pgvector enabled by migration. Migrations show the schema evolution clearly:
• enable pgvector
• create projects
• create users
• create documents
• create tasks
• create knowledge_items
• add a full-text search index
This tells you the system’s storage model is hybrid:
• relational tables for core entities
• vector search for semantic retrieval
• full-text indexing for keyword-based retrieval
That hybrid storage design is the foundation for the RAG/search layer.
Infrastructure and deployment model
The repo is container-first:
• Dockerfile for the application image
• docker-compose.yml for the local multi-service stack
• PostgreSQL with pgvector as the primary datastore
• Redis as the cache layer
The application is configured entirely through environment variables, including:
• app host/port
• PostgreSQL connection details
• Redis connection details
• LLM provider selection and model settings
• embedding and RAG behavior flags
• CORS and observability options
This means deployment architecture is environment-driven and service-oriented, not embedded in code.
Observability and operational concerns
The system includes:
• structured logging
• request logging middleware
• health checking with database connectivity
• error-handling utilities
• configuration-driven behavior
These concerns sit outside the core business layer but are essential to runtime architecture. They support debugging, readiness checks, and production operations.
Testing topology
The test layout mirrors the architecture:
• unit tests for domain models, shared utilities, and application services
• integration tests for repositories, provider factory, and cache
• e2e tests for API endpoints
That test stratification reinforces the layer boundaries and indicates which parts are intended to be isolated versus integration-tested.
Inferred subsystem map
For an architecture diagram, the main boxes should be:
1. Clients
- human API consumers
- AI agents
- future MCP/tooling clients
2. API edge
- FastAPI app
- versioned HTTP routes
- validation schemas
- middleware
3. Application orchestration
- project use cases
- knowledge ingestion/query use cases
- RAG/search services
- extraction/chunking/reranking/synthesis services
4. Domain core
- project, document, task, knowledge, user models
5. Infrastructure adapters
- PostgreSQL repositories
- Redis cache
- crawler client
- LLM provider factory and vendor providers
6. Shared platform services
- config
- logging
- security
- database connection
- errors/versioning
7. Operational systems
- PostgreSQL + pgvector
- Redis
- external LLM APIs
- external web sources for crawling
Key architectural takeaway
The repository is built around a clean separation between:
• knowledge/workflow orchestration
• domain entities
• persistence/search infrastructure
• external AI and crawling services
The most important domain boundary is between raw content management (documents, crawl/upload) and derived retrievable knowledge (knowledge_items, RAG query). The most important technical boundary is between the application layer and the vendor-backed infrastructure layer, especially for LLMs, vector search, and caching.
Main entry points: projects.py HTTP Routes · dependencies.py Dependencies · ingest_knowledge.py Ingest Knowledge · connection.py DB Connection
Components 34
Clients
API consumers and agent tooling
- Humans · Human users of the REST API
- Agents · AI agents and future MCP tooling
API Edge
FastAPI entrypoint, routes, schemas, and middleware
- FastAPI App · Application entrypoint and router assembly
src/api/main.py - HTTP Routes · Versioned REST endpoints for auth, projects, documents, tasks, knowledge, and rag
src/api/v1/routes/projects.py - API Schemas · Pydantic request and response models
src/api/v1/schemas/project.py - Logging Middleware · Request logging and correlation
src/api/middleware/logging_middleware.py - Dependencies · Route dependency wiring for services and repositories
src/api/dependencies.py
Application
Use cases and orchestration services
- Create Project · Coordinates project creation workflows
src/application/use_cases/projects/create_project.py - Ingest Knowledge · Orchestrates upload and ingestion flows
src/application/use_cases/ingest_knowledge.py - Crawl Knowledge · Orchestrates crawling and intake of external content
src/application/use_cases/crawl_knowledge.py - Query Knowledge · Coordinates retrieval and answer generation
src/application/use_cases/knowledge/query_knowledge.py - Text Extractor · Extracts text from source artifacts
src/application/services/text_extractor.py - Text Chunker · Splits content into retrieval-friendly chunks
src/application/services/text_chunker.py - Hybrid Search · Combines keyword and semantic retrieval
src/application/services/hybrid_search_service.py - Reranking · Reorders candidate results by relevance
src/application/services/reranking_service.py - Synthesis · Builds final RAG responses from retrieved context
src/application/services/synthesis_service.py
Domain Core
Business entities and invariants
- Project · Root organizing concept for knowledge work
src/domain/models/project.py - Document · Versioned source artifact
src/domain/models/document.py - Task · Project-scoped work item
src/domain/models/task.py - Knowledge · Derived searchable knowledge record
src/domain/models/knowledge.py - User · Authenticated user and ownership subject
src/domain/models/user.py
Infrastructure
Persistence and external adapters
- Project Repo · Persistence adapter for projects
src/infrastructure/database/repositories/project_repository.py - Document Repo · Persistence adapter for documents
src/infrastructure/database/repositories/document_repository.py - Task Repo · Persistence adapter for tasks
src/infrastructure/database/repositories/task_repository.py - Knowledge Repo · Persistence adapter for knowledge items and retrieval queries
src/infrastructure/database/repositories/knowledge_repository.py - User Repo · Persistence adapter for users
src/infrastructure/database/repositories/user_repository.py - Redis Cache · Optional cache for retrieval and repeated lookups
src/infrastructure/cache/redis_cache.py - Crawler Client · Fetches external web or documentation content
src/infrastructure/external/crawler/crawler_client.py - LLM Factory · Selects the active LLM backend
src/infrastructure/external/llm/provider_factory.py - LLM Providers · OpenAI, Anthropic, Ollama, and OpenRouter integrations
src/infrastructure/external/llm/providers/base.py
Shared Platform
Cross-cutting configuration and utilities
- Settings · Environment-driven application configuration
src/shared/config/settings.py - Logging · Shared logging setup
src/shared/config/logging.py - DB Connection · Async database pool and connection management
src/shared/infrastructure/database/connection.py - Security Utils · Auth and JWT helper functions
src/shared/utils/security.py
Connections 41
- Humans → FastAPI App: REST requests · Humans call the HTTP API
- Agents → FastAPI App: REST / MCP · Agents interact through the API and planned tool surface
- FastAPI App → HTTP Routes: mounts · Registers versioned routes
- FastAPI App → Logging Middleware: wraps · Adds request logging and correlation
- HTTP Routes → API Schemas: validates · Uses typed request and response contracts
- HTTP Routes → Dependencies: injects · Resolves services and repositories per request
- HTTP Routes → Create Project: creates · Project endpoints invoke project use cases
- HTTP Routes → Ingest Knowledge: ingests · Document and knowledge upload flows
- HTTP Routes → Crawl Knowledge: crawls · Knowledge crawl entrypoints
- HTTP Routes → Query Knowledge: queries · RAG query endpoints
- Dependencies → Project Repo: wires · Provides persistence adapters
- Dependencies → Document Repo: wires · Provides persistence adapters
- Dependencies → Task Repo: wires · Provides persistence adapters
- Dependencies → Knowledge Repo: wires · Provides persistence adapters
- Dependencies → User Repo: wires · Provides persistence adapters
- Dependencies → Hybrid Search: wires · Provides retrieval service dependencies
- Dependencies → Settings: reads · Uses environment configuration
- Create Project → Project: uses · Operates on project entities
- Create Project → Project Repo: persists · Stores projects
- Ingest Knowledge → Text Extractor: extracts · Pulls text from source documents
- Ingest Knowledge → Text Chunker: chunks · Prepares content for indexing
- Ingest Knowledge → Document Repo: stores · Persists source artifacts
- Ingest Knowledge → Knowledge Repo: indexes · Creates retrievable knowledge records
- Crawl Knowledge → Crawler Client: fetches · Retrieves external content
- Crawl Knowledge → Ingest Knowledge: delegates · Hands crawled content to ingestion flow
- Query Knowledge → Hybrid Search: retrieves · Collects candidate knowledge
- Query Knowledge → Reranking: reranks · Optionally improves result ordering
- Query Knowledge → Synthesis: synthesizes · Generates final answer text
- Hybrid Search → Knowledge Repo: searches · Queries the knowledge index
- Hybrid Search → Redis Cache: consults · Uses cache when enabled
- Synthesis → LLM Factory: selects · Obtains the configured LLM backend
- LLM Factory → LLM Providers: routes to · Chooses one of the vendor adapters
- Knowledge Repo → DB Connection: queries · Uses the shared async database connection
- Project Repo → DB Connection: queries · Uses the shared async database connection
- Document Repo → DB Connection: queries · Uses the shared async database connection
- Task Repo → DB Connection: queries · Uses the shared async database connection
- User Repo → DB Connection: queries · Uses the shared async database connection
- HTTP Routes → Security Utils: protects · Auth dependencies and JWT helpers secure routes
- Settings → DB Connection: configures · Drives database connectivity
- Settings → Redis Cache: configures · Drives cache behavior
- Settings → LLM Factory: configures · Selects provider and model settings