How eblict-sd20/cogopscb works
1. Overall shape
- This repo is a two-service chatbot system for Bangladesh government services.
- The main app is a FastAPI-backed chatbot/API + UI service.
- Retrieval is split out into a separate jiggasha search service, so answering is not done in one monolith.
2. Primary request path
- User input enters the chatbot API at api.py.
- The core flow is deterministic: sanitize → router → retrieval/rerank → composer → verifier.
- The system is built to answer only from a fixed government-services corpus and to refuse unsupported claims rather than hallucinate.
3. Main chatbot service boundary
- govtchat.service runs the main app on port 9000.
- It serves both the API endpoints and the browser UI from cogops/ui/index.html.
- This service owns conversation handling, prompt assembly, streaming responses, verification, and session state.
4. Search/retrieval service boundary
- jiggasha-search.service runs independently on port 10000.
- It handles passage retrieval for Bengali queries using embeddings, Qdrant, and an LLM-based reranker.
- The chatbot service calls it over HTTP; it is a hard dependency for the retrieval stage.
5. Input normalization and routing
- cogops/pipeline/sanitize.py removes unsafe or malformed input patterns before any LLM call.
- cogops/pipeline/router.py classifies intent and produces Bengali sub-queries.
- The router can fast-path some pure Bengali domain queries without invoking the LLM.
6. Retrieval and reranking
- The jiggasha package is the retrieval subsystem: query expansion, embedding, Qdrant top-K lookup, and rerank.
- The search service returns vetted passages grouped by sub-query, with a cosine-based fallback if rerank fails.
- This subsystem is the only place where the corpus is searched; the chatbot does not search directly.
7. Answer generation
- cogops/agents/pipeline.py is the composer stage.
- It builds the final prompt from the retrieved context, optional disambiguation text, and the user query.
- The primary LLM streams the answer and cites evidence using [S#] source tags allocated by the application.
8. Post-answer verification
- cogops/verifier/nli.py performs a post-flight NLI check.
- It is fail-soft and non-blocking at the architecture level, but it can redact or refuse unsupported cited claims depending on policy.
- This is the final grounding boundary before the answer is returned to the user.
9. Disambiguation behavior
- The system can ask for clarification when a short intent maps to passages from multiple service categories.
- This prevents the composer from guessing across ambiguous government-service topics.
- The disambiguation logic is part of the pipeline rather than a separate user-flow service.
10. Session and conversation state
- cogops/session/* stores per-user history and query logs.
- A Redis-backed store is present for persistence and cross-request continuity.
- This is what allows the test harness and live chatbot sessions to accumulate context server-side.
11. Eventing and streaming
- cogops/events/* defines the event types and channels used to stream stage-by-stage progress.
- The UI debug pane and the test harness consume these events as NDJSON/streamed updates.
- This event layer exposes internal pipeline boundaries without changing the actual request flow.
12. LLM integration
- cogops/llm/clients.py centralizes primary/secondary model access.
- The repo distinguishes the primary answer model from the secondary reasoning/rerank models.
- cogops/llm/reasoning_loop.py suggests supporting logic for controlled LLM interactions, though the main production path is explicitly non-ReAct.
13. Prompts, tools, and parsing support
- cogops/prompts/* contains stage-specific prompt templates for composer, system messages, time reminders, and ReAct-related helpers.
- cogops/tools/* and cogops/utils/thinking_parser.py provide supporting abstractions, but the shipped architecture centers on the fixed pipeline rather than free-form tool use.
- These components mostly define the contract around LLM behavior and output parsing.
14. Operational and test boundaries
- Configuration comes from configs/config.yml plus environment files, with separate settings for pipeline, verifier, and Jiggasha.
- Tests cover normalization, input handling, query expansion, grounding, and end-to-end concurrent sessions.
- The test harness is architecture-relevant because it validates session isolation, streaming events, retrieval quality, and citation grounding under load.
Main entry points: orchestrator.py Orchestrator · pipeline.py Composer · service.py Search API · api.py API entry
Components 19
Main chatbot
FastAPI chatbot/API service, UI, session handling, prompts, events, and verification.
- API entry · Main request entrypoint serving chatbot endpoints and the UI.
api.py - Web UI · Served static interface for interactive chat and debug streaming.
cogops/ui/index.html - Sanitize · Removes unsafe or malformed input before any model call.
cogops/pipeline/sanitize.py - Router · Classifies intent and produces Bengali sub-queries with fast-path handling.
cogops/pipeline/router.py - Orchestrator · Coordinates the staged chatbot flow and stage transitions.
cogops/agents/orchestrator.py - Composer · Builds the final prompt and streams grounded responses with citations.
cogops/agents/pipeline.py - Verifier · Post-answer grounding check that can redact or refuse unsupported claims.
cogops/verifier/nli.py - Citations · Assigns and formats evidence tags used by the generated answer.
cogops/verifier/citations.py - Session store · Redis-backed history and query logging for cross-request continuity.
cogops/session/redis_store.py - Events · Stage-by-stage event types and channels consumed by UI and tests.
cogops/events/types.py
Retrieval service
Separate Jiggasha search service for query expansion, embedding search, and reranking.
- Query expand · Expands and normalizes user queries before retrieval and routing.
jiggasha/embedder.py - Search API · Independent Jiggasha service that exposes passage search and rerank.
jiggasha/service.py - Search config · Search-service configuration and instruction tuning for retrieval behavior.
jiggasha/config.yml
Shared support
Configuration, LLM access, tools, and utility helpers used across the pipeline.
- LLM clients · Centralized primary and secondary model clients.
cogops/llm/clients.py - Prompts · Stage-specific system, composer, and reminder prompts.
cogops/prompts/composer.py - Thinking parser · Parses structured reasoning or intermediate LLM output when needed.
cogops/utils/thinking_parser.py - Tool registry · Registered helper tools and adapters, including retrieval integration.
cogops/tools/registry.py - Jiggasha tool · HTTP client wrapper used by the chatbot to call the search service.
cogops/tools/jiggasha.py - App config · Main application configuration for pipeline, verifier, and service wiring.
configs/config.yml
Connections 21
- API entry → Sanitize: accept input · Incoming user text is first sanitized.
- Sanitize → Router: clean query · Cleaned input is routed into intent classification and sub-query generation.
- Router → Orchestrator: route intent · Router hands control to the pipeline orchestrator.
- Orchestrator → Jiggasha tool: retrieve · Orchestrator invokes retrieval through the Jiggasha client.
- Jiggasha tool → Search API: HTTP call · Chatbot service depends on the separate search service for retrieval.
- Search API → Query expand: expand/search · Search service expands queries and performs indexed passage lookup.
- Query expand → LLM clients: embed/rerank · Retrieval uses embeddings and LLM-based reranking support.
- Search API → Composer: passages · Retrieved vetted passages are returned to the composer stage.
- Composer → Prompts: assemble prompt · Composer builds the final grounded prompt from templates and context.
- Composer → LLM clients: stream answer · Primary model streams the response back through the application.
- Composer → Citations: tag evidence · Evidence tags are allocated and attached to answer text.
- Composer → Verifier: post-flight check · Generated output is checked for groundedness and policy compliance.
- Verifier → Citations: validate cites · Verifier uses citation state to assess supported claims.
- Orchestrator → Session store: load/save · Conversation history and query logs are persisted per session.
- Orchestrator → Events: emit stages · Pipeline stages are streamed as events for UI and harness consumers.
- Web UI → Events: consume stream · Browser UI listens to streamed pipeline progress updates.
- API entry → Web UI: serve UI · Main service serves the static browser interface.
- Orchestrator → Thinking parser: parse output · Structured reasoning or intermediate outputs can be parsed when used.
- Orchestrator → Tool registry: use helpers · Pipeline can access registered helpers and adapters.
- App config → API entry: configure · Shared configuration drives the main service wiring and policies.
- Search config → Search API: configure · Search-service behavior is controlled by its own configuration.