How nishadmahmud/queuestorm-api works
1. What this service is
- QueueStorm Investigator is a single-purpose FastAPI backend for investigating support tickets about financial transactions.
- The main business boundary is one HTTP request: a customer complaint plus optional recent transaction_history goes in, and a strict structured case assessment comes out.
2. Public API surface
- app/main.py is the entry point and exposes three routes: /, /health, and POST /analyze-ticket.
- /analyze-ticket is the only business endpoint; it returns a fixed JSON contract with fields like case_type, severity, department, evidence_verdict, and customer_reply.
3. Request/response contract
- app/schemas.py defines the canonical request and response models.
- The service is intentionally tolerant on optional context: unknown enum-like context values degrade to null, and malformed transaction entries are tolerated rather than blocking the whole request.
- Invalid top-level structure returns 400, empty complaints return 422, and the successful path always returns a schema-valid response.
4. Core request lifecycle
- The processing path is: validation → evidence matching → classification/routing → reply drafting → safety filtering → final response.
- The complaint is treated as untrusted input throughout; the design assumes prompt injection may appear inside user text and must not alter system behavior.
5. Reasoning subsystem
- app/reasoning/ contains the decision logic split into evidence extraction, classification, escalation, and routing.
- This layer decides whether the complaint is supported by the provided transactions, whether the evidence is inconsistent or insufficient, which case type applies, and whether human review is required.
- Escalation is a separate concern: high-risk cases, especially phishing/social-engineering, are forced toward human review.
6. Evidence-first transaction analysis
- The service does not classify from complaint text alone.
- It tries to match a relevant_transaction_id using clues such as amount, time, transaction type, and counterparty, then labels the evidence as consistent, inconsistent, or insufficient.
- Duplicate-payment detection is part of the reasoning boundary, not just a prompt instruction.
7. LLM abstraction and provider fallback
- app/llm/ abstracts model access behind a provider layer with Gemini, Groq, and NVIDIA clients plus shared prompt logic.
- The model order is configurable, and the README shows a deliberate resilience chain: multiple Gemini Flash/Flash-Lite models first, then Groq, then NVIDIA.
- This layer is outbound-only HTTPS to external AI services; the application itself does not embed models locally.
8. Deterministic offline fallback
- If all providers fail or time out, the service falls back to a rules-only engine in app/reasoning/classify.py.
- That fallback is important architecturally: it preserves availability, keeps the response schema valid, and ensures the service can still be tested offline with no API keys.
- The fallback is conservative and lower-confidence by design.
9. Safety and guardrail layer
- app/safety/ is a final post-processing choke point that scans generated text fields for unsafe content.
- It blocks credential requests, unauthorized refund/promises, suspicious external channels, and prompt-injection style misuse.
- Safety is field-scoped: for example, customer_reply is more restricted than agent_summary, and violations force human_review_required = true.
10. Configuration and runtime dependencies
- app/config.py centralizes environment-driven settings such as API keys, model names, provider order, and timeout budgets.
- The runtime stack is FastAPI served by Uvicorn, typically behind Nginx in production.
- The design is built around short per-attempt timeouts and a total budget, because latency and rate-limit resilience are part of the architecture.
11. Deployment topology
- Production traffic flows through deploy/nginx.conf, which terminates HTTPS and reverse proxies to a local Uvicorn process on 127.0.0.1:8000.
- A systemd unit in deploy/preli-api.service keeps the API process supervised and restartable.
- Deployment targets a DigitalOcean Droplet, with setup and redeploy automation in deploy/setup-droplet.sh and deploy/redeploy.sh.
12. CI/CD and operational validation
- .github/workflows/deploy.yml runs tests on every push to main and only deploys if the offline test suite passes.
- tests/conftest.py intentionally clears API keys so tests exercise the deterministic fallback path instead of depending on live LLM access.
- scripts/replay_samples.py and scripts/benchmark.py are operational tools for validating the official sample cases and latency targets.
13. Documentation and diagram assets
- The repository includes a strong architecture-documentation track under docs/diagrams/, with separate diagrams for system layout, request lifecycle, AI fallback, safety, CI/CD, infrastructure, and data contract.
- Those artifacts mirror the actual code boundaries and are useful for drawing a diagram that matches the implementation rather than just the README narrative.
14. Primary external dependencies
- The only runtime dependencies that materially shape the architecture are the AI providers: Google Gemini, Groq, and NVIDIA NIM.
- Everything else is internal infrastructure: FastAPI/Uvicorn for serving, Nginx/systemd for production hosting, and GitHub Actions for deployment automation.
Main entry points: main.py Main API · base.py LLM Base · filter.py Safety Filter · deploy.yml Deploy Workflow
Components 23
API Surface
FastAPI entrypoint and request/response contract
Core Logic
Request lifecycle, reasoning, and fallback classification
- Config · Environment-driven model, timeout, and provider configuration
app/config.py - Evidence · Extracts relevant transaction signals and labels evidence quality
app/reasoning/evidence.py - Classify · Deterministic case type and severity decision path
app/reasoning/classify.py - Routing · Maps cases to department and human review need
app/reasoning/routing.py - Escalate · Raises human review for high-risk or ambiguous cases
app/reasoning/escalate.py
LLM Providers
Provider abstraction and external model clients
- Prompts · Shared prompt assembly for model calls
app/llm/prompts.py - LLM Base · Common model abstraction and provider orchestration
app/llm/base.py - Gemini · Gemini model client used in the fallback chain
app/llm/gemini.py - Groq · Groq client used after Gemini in the provider order
app/llm/groq_client.py - NVIDIA · NVIDIA NIM client used as a later fallback provider
app/llm/nvidia.py - Compat · OpenAI-compatible transport helpers shared by clients
app/llm/_openai_compat.py
Safety
Post-processing guardrails for generated output
- Safety Filter · Scans generated fields for unsafe or disallowed content
app/safety/filter.py - Credentials · Blocks credential requests and secret harvesting
app/safety/credentials.py - Promises · Blocks unauthorized refunds and payment promises
app/safety/promises.py - Channels · Blocks suspicious external contact channels
app/safety/channels.py
Operations
Deployment, CI, and validation tooling
- Nginx · Terminates HTTPS and reverse proxies to the app
deploy/nginx.conf - Systemd · Supervises the API process on the host
deploy/preli-api.service - Deploy Workflow · Runs tests and deploys from GitHub Actions
.github/workflows/deploy.yml - Replay Samples · Runs official sample cases through the service
scripts/replay_samples.py - Benchmark · Checks latency and operational performance
scripts/benchmark.py
Docs
Architecture diagrams and reference artifacts
- Diagrams · Architecture diagrams matching the implemented boundaries
docs/diagrams/01-architecture.mmd
Connections 26
- Main API → Schemas: uses contract · Validates requests and returns the structured response schema
- Main API → Config: loads settings · Reads runtime configuration for provider order and budgets
- Main API → Evidence: analyzes complaint · Starts the evidence-first request lifecycle
- Evidence → Classify: feeds signals · Hands matched evidence and transaction context to classification
- Classify → Routing: assigns case · Determines department and handling path
- Routing → Escalate: escalation check · Forces human review for high-risk or unclear cases
- Main API → LLM Base: LLM path · Uses model abstraction when provider-backed reasoning is available
- LLM Base → Prompts: builds prompts · Assembles shared prompts for external model calls
- LLM Base → Gemini: tries first · Primary provider in the configured fallback chain
- LLM Base → Groq: falls back · Secondary provider in the configured fallback chain
- LLM Base → NVIDIA: falls back · Later provider in the configured fallback chain
- Gemini → Compat: uses transport · Shares OpenAI-compatible client helpers
- Groq → Compat: uses transport · Shares OpenAI-compatible client helpers
- NVIDIA → Compat: uses transport · Shares OpenAI-compatible client helpers
- Main API → Safety Filter: post-processes · Applies final guardrails to generated output fields
- Safety Filter → Credentials: checks · Rejects credential-seeking text
- Safety Filter → Promises: checks · Rejects unauthorized refund or promise language
- Safety Filter → Channels: checks · Rejects suspicious external contact instructions
- Main API → Deploy Workflow: deployed by · CI validates main branch before deployment
- Deploy Workflow → Replay Samples: validates · Exercises sample cases in the offline path
- Deploy Workflow → Benchmark: measures · Checks latency and throughput targets
- Nginx → Systemd: fronts · Routes public traffic to the supervised app process
- Deploy Workflow → Nginx: deploys via · Operational deployment targets the Nginx-fronted host
- Diagrams → Main API: documents · Reference diagrams mirror the implemented architecture
- Diagrams → Evidence: documents · Request lifecycle and evidence flow are diagrammed
- Diagrams → Safety Filter: documents · Safety guardrail boundaries are diagrammed