# sisig-ai/doctor architecture

> Architecture diagram of the GitHub repository sisig-ai/doctor, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sisig-ai/doctor
- Repository: https://github.com/sisig-ai/doctor

## Overview

##### Main Components
- Client / LLM Agent
– The user’s HTTP client or MCP-capable editor (e.g. VSCode) requesting crawl/search.
- API Layer
– Amazon API Gateway or AWS Application Load Balancer fronting the FastAPI service.
- Compute Cluster
– Amazon ECS (Fargate) or EKS cluster hosting two services:
a. Web Service (FastAPI in Docker)
b. Crawl Worker (as a background task processor)
- Message Broker
– Amazon ElastiCache (Redis) for Celery-style task queuing.
- Data Store
– DuckDB file stored on Amazon EFS (volume mounted into the containers)
- External API
– OpenAI Embedding endpoint via public internet.
- Monitoring & Logging
– Amazon CloudWatch Logs & Metrics for container and Redis metrics.

##### Relationships & Data Flows
a. Client → API Gateway (HTTPS) → ECS-Fargate (Web Service)
- “POST /fetch_url” enqueues a crawl job to Redis.
- “GET /search_docs” queries DuckDB via the Web Service.
b. Web Service → Redis (Enqueue)
c. Crawl Worker (polls Redis) → Crawls target URL → Chunker → Embedder → OpenAI API → Writes embeddings + pages to DuckDB on EFS
d. Web Service → EFS-backed DuckDB (Reads/Writes)
e. Web Service → Client via Server-Sent Events (MCP SSE endpoint)

##### Architectural Patterns
– Containerized microservices using ECS Fargate (no servers to manage).
– Asynchronous task queue with Redis (managed ElastiCache).
– Serverless-style data store for analytical workloads (DuckDB on EFS or Aurora).
– Third-party integration (OpenAI) abstracted in the embedder component.
– FastAPI as an API gateway inside the container, but fronted by API Gateway / ALB in AWS.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Network Layer
    Client["User/LLM Agent"]:::network
    API_Gateway["API Gateway"]:::network

    %% ECS Cluster
    subgraph "ECS Fargate Cluster"
        direction TB
        WebService["ECS Fargate\ndoctor-web_service"]:::compute
        CrawlWorker["ECS Fargate\ndoctor-crawl_worker"]:::compute
    end

    %% Data Layer
    Redis["ElastiCache Redis"]:::datastore
    DuckDB["EFS DuckDB / Aurora Vector DB"]:::datastore

    %% External Services
    OpenAI["OpenAI Embedding API"]:::external

    %% Monitoring & Security
    CloudWatch["CloudWatch Logs & Metrics"]:::monitoring
    SecretsMgr["Secrets Manager / IAM"]:::security

    %% Connections
    Client -->|HTTPS| API_Gateway
    API_Gateway --> WebService

    WebService -->|POST /fetch_url\nENQUEUE| Redis
    WebService -->|GET /search_docs| DuckDB
    WebService -->|SSE| Client

    CrawlWorker -->|POLL/DEQUEUE| Redis
    CrawlWorker -->|WRITE Embeddings & Pages| DuckDB
    CrawlWorker -->|REQUEST Embed| OpenAI

    WebService --> CloudWatch
    CrawlWorker --> CloudWatch
    Redis --> CloudWatch

    SecretsMgr --> WebService
    SecretsMgr --> CrawlWorker
    SecretsMgr --> OpenAI

    %% Click Events
    click WebService "https://github.com/sisig-ai/doctor/tree/main/src/web_service"
    click CrawlWorker "https://github.com/sisig-ai/doctor/tree/main/src/crawl_worker"
    click Redis "https://github.com/sisig-ai/doctor/blob/main/docker-compose.yml"
    click DuckDB "https://github.com/sisig-ai/doctor/tree/main/src/lib/database"

    %% Styles
    classDef compute fill:#f90,stroke:#333,stroke-width:1px;
    classDef datastore fill:#0af,stroke:#333,stroke-width:1px;
    classDef network fill:#0a0,stroke:#333,stroke-width:1px;
    classDef external fill:#999,stroke:#333,stroke-width:1px;
    classDef monitoring fill:#bb0,stroke:#333,stroke-width:1px;
    classDef security fill:#ff0,stroke:#333,stroke-width:1px;
```
