# mrnkim/surgical-tool-detection architecture

> Architecture diagram of the GitHub repository mrnkim/surgical-tool-detection, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mrnkim/surgical-tool-detection
- Repository: https://github.com/mrnkim/surgical-tool-detection

## Overview

Surgical Tool Detection is a web platform for indexing surgical videos, detecting instruments, and exploring video intelligence results. The principal workflow is upload → video indexing → YOLO detection → persisted results → timeline/statistics and analysis. Search, SOAP notes, and clip creation are separate product capabilities. The frontend API and backend excerpts show key upload and detection handoffs; README statements support broader features. Some UI wiring and several API implementations were not sampled, so those relationships are omitted or cited as documented capabilities.

Main entry points: [`frontend/src/app/[factoryId]/page.js`](https://github.com/mrnkim/surgical-tool-detection/blob/main/frontend/src/app/%5BfactoryId%5D/page.js) (Video workspace), [`frontend/src/app/api/upload/route.js`](https://github.com/mrnkim/surgical-tool-detection/blob/main/frontend/src/app/api/upload/route.js) (Video upload API), [`frontend/src/app/api/detect-tools/[videoId]/route.js`](https://github.com/mrnkim/surgical-tool-detection/blob/main/frontend/src/app/api/detect-tools/%5BvideoId%5D/route.js) (Detection result access), [`frontend/src/app/api/analysis/route.js`](https://github.com/mrnkim/surgical-tool-detection/blob/main/frontend/src/app/api/analysis/route.js) (Video chat API)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_web["Web experience"]
  node_webapp["Video workspace<br/>[page.js]"]
  node_clipui["Clip workspace<br/>[page.js]"]
  node_overlay["Detection overlay"]
  node_timelineui["Tool timeline"]
  node_statsui["Tool statistics"]
  node_searchui["Search results"]
  node_clips["Clip creation<br/>[ClipBento.js]"]
end

subgraph group_ingest["Video ingestion"]
  node_uploadapi["Video upload API<br/>[route.js]"]
  node_indexing["Video indexing"]
end

subgraph group_detection["Tool detection"]
  node_detectroute["Detection gateway<br/>[route.js]"]
  node_detector["YOLO detection API<br/>[main.py]"]
  node_toolresults["Detection result access<br/>[route.js]"]
  node_rtsp["RTSP stream worker<br/>[main.py]"]
end

subgraph group_intelligence["Video intelligence"]
  node_timelineapi["Chapter timeline API<br/>[route.js]"]
  node_analysisapi["Video chat API<br/>[route.js]"]
  node_searchapi["Video search API<br/>[route.js]"]
end

subgraph group_results["Analysis and results"]
  node_blob[("Video and result blobs")]
  node_soapapi["SOAP notes API<br/>[route.js]"]
  node_analysisstore["Analysis data API<br/>[route.js]"]
end

node_surgeon(("Video analyst"))
node_twelvelabs["TwelveLabs"]

node_surgeon -->|"uses"| node_webapp
node_surgeon -->|"uploads video"| node_uploadapi
node_uploadapi -->|"stores video"| node_blob
node_uploadapi -->|"indexes video"| node_twelvelabs
node_uploadapi -->|"starts detection"| node_detectroute
node_detectroute -->|"requests detection"| node_detector
node_detector -->|"saves results"| node_detectroute
node_detectroute -->|"persists results"| node_blob
node_webapp -.->|"views detections"| node_toolresults
node_toolresults -.->|"supplies detections"| node_overlay
node_toolresults -.->|"feeds timeline"| node_timelineui
node_toolresults -.->|"feeds statistics"| node_statsui
node_webapp -.->|"requests chapters"| node_timelineapi
node_timelineapi -->|"generates chapters"| node_twelvelabs
node_webapp -.->|"asks video questions"| node_analysisapi
node_analysisapi -->|"analyzes video"| node_twelvelabs
node_analysisapi -->|"reads detections"| node_blob
node_analysisstore -->|"reads and writes"| node_blob
node_soapapi -.->|"edits analysis"| node_analysisstore
node_webapp -.->|"searches videos"| node_searchapi
node_searchapi -->|"queries index"| node_twelvelabs
node_searchui -.->|"creates clips"| node_clips
node_surgeon -->|"reviews clips"| node_clipui
node_rtsp -.->|"runs CV analysis"| node_detector

click node_webapp "https://github.com/mrnkim/surgical-tool-detection/blob/main/frontend/src/app/%5BfactoryId%5D/page.js"
click node_clipui "https://github.com/mrnkim/surgical-tool-detection/blob/main/frontend/src/app/clips/%5Bid%5D/page.js"
click node_uploadapi "https://github.com/mrnkim/surgical-tool-detection/blob/main/frontend/src/app/api/upload/route.js"
click node_detectroute "https://github.com/mrnkim/surgical-tool-detection/blob/main/frontend/src/app/api/detect-tools/%5BvideoId%5D/route.js"
click node_detector "https://github.com/mrnkim/surgical-tool-detection/blob/main/backend/main.py"
click node_toolresults "https://github.com/mrnkim/surgical-tool-detection/blob/main/frontend/src/app/api/detect-tools/%5BvideoId%5D/route.js"
click node_overlay "https://github.com/mrnkim/surgical-tool-detection/blob/main/frontend/src/app/components/ToolDetectionOverlay.js"
click node_timelineui "https://github.com/mrnkim/surgical-tool-detection/blob/main/frontend/src/app/components/ToolUsageTimeline.js"
click node_statsui "https://github.com/mrnkim/surgical-tool-detection/blob/main/frontend/src/app/components/ToolUsageStatistics.js"
click node_timelineapi "https://github.com/mrnkim/surgical-tool-detection/blob/main/frontend/src/app/api/timeline/route.js"
click node_analysisapi "https://github.com/mrnkim/surgical-tool-detection/blob/main/frontend/src/app/api/analysis/route.js"
click node_soapapi "https://github.com/mrnkim/surgical-tool-detection/blob/main/frontend/src/app/api/analysis/%5BvideoId%5D/soap/route.js"
click node_analysisstore "https://github.com/mrnkim/surgical-tool-detection/blob/main/frontend/src/app/api/analysis/%5BvideoId%5D/route.js"
click node_searchapi "https://github.com/mrnkim/surgical-tool-detection/blob/main/frontend/src/app/api/search/route.js"
click node_searchui "https://github.com/mrnkim/surgical-tool-detection/blob/main/frontend/src/app/components/SearchResultModal.js"
click node_clips "https://github.com/mrnkim/surgical-tool-detection/blob/main/frontend/src/app/components/ClipBento.js"
click node_rtsp "https://github.com/mrnkim/surgical-tool-detection/blob/main/rtsp-stream-worker/main.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_webapp,node_clipui,node_overlay,node_timelineui,node_statsui,node_searchui,node_clips toneBlue
class node_uploadapi,node_indexing toneAmber
class node_detectroute,node_detector,node_toolresults,node_rtsp toneMint
class node_timelineapi,node_analysisapi,node_searchapi toneRose
class node_blob,node_soapapi,node_analysisstore,node_surgeon,node_twelvelabs toneIndigo
```

## Components

### Web experience

- [Video workspace](https://github.com/mrnkim/surgical-tool-detection/blob/main/frontend/src/app/%5BfactoryId%5D/page.js): `frontend/src/app/[factoryId]/page.js`
- [Clip workspace](https://github.com/mrnkim/surgical-tool-detection/blob/main/frontend/src/app/clips/%5Bid%5D/page.js): `frontend/src/app/clips/[id]/page.js`
- [Detection overlay](https://github.com/mrnkim/surgical-tool-detection/blob/main/frontend/src/app/components/ToolDetectionOverlay.js): `frontend/src/app/components/ToolDetectionOverlay.js`
- [Tool timeline](https://github.com/mrnkim/surgical-tool-detection/blob/main/frontend/src/app/components/ToolUsageTimeline.js): `frontend/src/app/components/ToolUsageTimeline.js`
- [Tool statistics](https://github.com/mrnkim/surgical-tool-detection/blob/main/frontend/src/app/components/ToolUsageStatistics.js): `frontend/src/app/components/ToolUsageStatistics.js`
- [Search results](https://github.com/mrnkim/surgical-tool-detection/blob/main/frontend/src/app/components/SearchResultModal.js): `frontend/src/app/components/SearchResultModal.js`
- [Clip creation](https://github.com/mrnkim/surgical-tool-detection/blob/main/frontend/src/app/components/ClipBento.js): `frontend/src/app/components/ClipBento.js`

### Video ingestion

- [Video upload API](https://github.com/mrnkim/surgical-tool-detection/blob/main/frontend/src/app/api/upload/route.js): `frontend/src/app/api/upload/route.js`
- **Video indexing**

### Tool detection

- [Detection gateway](https://github.com/mrnkim/surgical-tool-detection/blob/main/frontend/src/app/api/detect-tools/%5BvideoId%5D/route.js): `frontend/src/app/api/detect-tools/[videoId]/route.js`
- [YOLO detection API](https://github.com/mrnkim/surgical-tool-detection/blob/main/backend/main.py): `backend/main.py`
- [Detection result access](https://github.com/mrnkim/surgical-tool-detection/blob/main/frontend/src/app/api/detect-tools/%5BvideoId%5D/route.js): `frontend/src/app/api/detect-tools/[videoId]/route.js`
- [RTSP stream worker](https://github.com/mrnkim/surgical-tool-detection/blob/main/rtsp-stream-worker/main.py): `rtsp-stream-worker/main.py`

### Video intelligence

- [Chapter timeline API](https://github.com/mrnkim/surgical-tool-detection/blob/main/frontend/src/app/api/timeline/route.js): `frontend/src/app/api/timeline/route.js`
- [Video chat API](https://github.com/mrnkim/surgical-tool-detection/blob/main/frontend/src/app/api/analysis/route.js): `frontend/src/app/api/analysis/route.js`
- [Video search API](https://github.com/mrnkim/surgical-tool-detection/blob/main/frontend/src/app/api/search/route.js): `frontend/src/app/api/search/route.js`

### Analysis and results

- **Video and result blobs**
- [SOAP notes API](https://github.com/mrnkim/surgical-tool-detection/blob/main/frontend/src/app/api/analysis/%5BvideoId%5D/soap/route.js): `frontend/src/app/api/analysis/[videoId]/soap/route.js`
- [Analysis data API](https://github.com/mrnkim/surgical-tool-detection/blob/main/frontend/src/app/api/analysis/%5BvideoId%5D/route.js): `frontend/src/app/api/analysis/[videoId]/route.js`

### Other components

- **Video analyst**
- **TwelveLabs**

## Connections

- Video analyst → Video workspace: uses
- Video analyst → Video upload API: uploads video (evidence: `frontend/src/app/api/upload/route.js`)
- Video upload API → Video and result blobs: stores video (evidence: `frontend/src/app/api/upload/route.js`)
- Video upload API → TwelveLabs: indexes video (evidence: `frontend/src/app/api/upload/route.js`)
- Video upload API → Detection gateway: starts detection (evidence: `frontend/src/app/api/upload/route.js`)
- Detection gateway → YOLO detection API: requests detection (evidence: `frontend/src/app/api/detect-tools/[videoId]/route.js`)
- YOLO detection API → Detection gateway: saves results (evidence: `backend/main.py`)
- Detection gateway → Video and result blobs: persists results (evidence: `frontend/src/app/api/detect-tools/[videoId]/route.js`)
- Video workspace → Detection result access: views detections
- Detection result access → Detection overlay: supplies detections
- Detection result access → Tool timeline: feeds timeline
- Detection result access → Tool statistics: feeds statistics
- Video workspace → Chapter timeline API: requests chapters
- Chapter timeline API → TwelveLabs: generates chapters (evidence: `frontend/src/app/api/timeline/route.js`)
- Video workspace → Video chat API: asks video questions
- Video chat API → TwelveLabs: analyzes video (evidence: `frontend/src/app/api/analysis/route.js`)
- Video chat API → Video and result blobs: reads detections (evidence: `frontend/src/app/api/analysis/route.js`)
- Analysis data API → Video and result blobs: reads and writes (evidence: `frontend/src/app/api/analysis/[videoId]/route.js`)
- SOAP notes API → Analysis data API: edits analysis
- Video workspace → Video search API: searches videos
- Video search API → TwelveLabs: queries index (evidence: `frontend/src/app/api/search/route.js`)
- Search results → Clip creation: creates clips
- Video analyst → Clip workspace: reviews clips
- RTSP stream worker → YOLO detection API: runs CV analysis