# apurvsinghgautam/robin architecture

> Architecture diagram of the GitHub repository apurvsinghgautam/robin, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/apurvsinghgautam/robin
- Repository: https://github.com/apurvsinghgautam/robin

## Overview

1. **What this repository is**
   Robin is a single-process Streamlit application for dark-web OSINT investigations. The app is not split into separate backend/frontend services; instead, `ui.py` acts as the top-level orchestrator and calls into search, scraping, and LLM modules.

2. **User-facing entry point**
   The main runtime entrypoint is the Streamlit UI launched with `streamlit run ui.py`. In Docker, `entrypoint.sh` and the `Dockerfile` package that same app into a container exposing port `8501`.

3. **Core application flow**
   The user provides an investigation query in the Streamlit UI. That query is refined by an LLM, sent to dark-web search engines, the resulting pages are scraped, and the extracted content is summarized back through the LLM. The final output is shown in the UI and can be saved as an investigation artifact.

4. **LLM subsystem**
   `llm.py` and `llm_utils.py` encapsulate model access and prompt/response handling. The README indicates support for multiple providers: OpenAI, Claude, Gemini, Ollama, and other OpenAI-compatible APIs. This layer is the main abstraction boundary for model choice, so the rest of the app can remain provider-agnostic.

5. **Search subsystem**
   `search.py` is responsible for querying dark-web search engines. This is the part most tightly coupled to network access and Tor. The architecture implies search is an independent step before scraping, rather than directly crawling arbitrary sites from the UI.

6. **Scraping subsystem**
   `scrape.py` handles retrieving page content from search results or investigation targets. This sits between the search stage and the LLM summary stage, turning raw web pages into text that can be analyzed. It is the main content-ingestion boundary.

7. **Configuration and environment**
   `config.py` centralizes runtime settings such as API keys, provider URLs, and other app options. `.env.example` documents the expected environment variables. This repo is designed so secrets and provider settings come from environment variables rather than hardcoded values.

8. **Operational dependencies**
   Tor is a material dependency for the search flow; the README explicitly requires it for dark-web lookups. The app also depends on third-party LLM APIs or local model servers, so external connectivity is part of the runtime architecture, not just a deployment detail.

9. **Persistence and investigation history**
   The README describes saved investigations and a “Past Investigations” panel. That means the app has a local persistence layer for investigation outputs, likely filesystem-based and mounted into the container when using Docker. This is a key boundary between ephemeral UI sessions and reusable case history.

10. **Health and deployment support**
   `health.py` suggests a lightweight container health-check or runtime readiness probe. Together with the `Dockerfile` and `entrypoint.sh`, this indicates the project is deployment-friendly and optimized for a single-container runtime.

11. **Streamlit-specific presentation layer**
   `.streamlit/config.toml` configures the Streamlit app’s appearance/behavior. This is not a separate subsystem, but it affects the deployment/runtime experience and confirms that the UI is the primary interaction surface.

12. **Release and distribution pipeline**
   The GitHub Actions workflows under `.github/workflows` handle release automation and Docker image publication. So the repo has a clear build-and-publish boundary: source code is packaged into a versioned container image and GitHub release artifacts.

13. **Architectural summary for diagramming**
   A good diagram would show: user → Streamlit UI (`ui.py`) → orchestration layer → LLM provider (`llm.py`/`llm_utils.py`) and Tor-backed search (`search.py`) → scraper (`scrape.py`) → summarized results back to UI, with config injected from environment and optional local file persistence for saved investigations.

Main entry points: [`ui.py`](https://github.com/apurvsinghgautam/robin/blob/main/ui.py) (Streamlit UI), [`llm.py`](https://github.com/apurvsinghgautam/robin/blob/main/llm.py) (LLM core), [`search.py`](https://github.com/apurvsinghgautam/robin/blob/main/search.py) (Search engine), [`scrape.py`](https://github.com/apurvsinghgautam/robin/blob/main/scrape.py) (Scraper)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_g1["Interface"]
  node_n1(("Investigator<br/>user"))
  node_n2["Streamlit UI<br/>app entrypoint<br/>[ui.py]"]
  node_n12["Streamlit runtime<br/>ui runtime"]
  node_n17["Streamlit config<br/>ui settings<br/>[config.toml]"]
end

subgraph group_g2["Core flow"]
  node_n3["App config<br/>settings<br/>[config.py]"]
  node_n4["LLM core<br/>model adapter<br/>[llm.py]"]
  node_n5["Prompt utils<br/>prompt helpers<br/>[llm_utils.py]"]
  node_n6["Search engine<br/>dark-web search<br/>[search.py]"]
  node_n7["Scraper<br/>content fetch<br/>[scrape.py]"]
  node_n8[("Case history<br/>local persistence")]
end

subgraph group_g3["External deps"]
  node_n9{{"Tor network<br/>network dependency"}}
  node_n10{{"LLM providers<br/>external APIs"}}
  node_n11{{"Search targets<br/>dark-web sources"}}
end

subgraph group_g4["Operations"]
  node_n13["Docker image<br/>container build"]
  node_n14["Entrypoint<br/>container startup<br/>[entrypoint.sh]"]
  node_n15["Health check<br/>readiness probe<br/>[health.py]"]
  node_n16["Release CI<br/>github actions<br/>[workflows]"]
end

node_n1 -->|"uses"| node_n2
node_n2 -->|"reads"| node_n3
node_n2 -->|"orchestrates"| node_n4
node_n4 -->|"uses"| node_n5
node_n2 -->|"triggers"| node_n6
node_n6 -->|"routes via"| node_n9
node_n6 -->|"queries"| node_n11
node_n2 -->|"passes leads to"| node_n7
node_n7 -->|"fetches"| node_n11
node_n7 -->|"summarizes with"| node_n4
node_n4 -->|"calls"| node_n10
node_n2 -->|"saves to"| node_n8
node_n2 -->|"runs in"| node_n12
node_n13 -->|"starts"| node_n14
node_n13 -->|"exposes"| node_n15
node_n16 -->|"builds"| node_n13
node_n2 -->|"styled by"| node_n17

click node_n2 "https://github.com/apurvsinghgautam/robin/blob/main/ui.py"
click node_n3 "https://github.com/apurvsinghgautam/robin/blob/main/config.py"
click node_n4 "https://github.com/apurvsinghgautam/robin/blob/main/llm.py"
click node_n5 "https://github.com/apurvsinghgautam/robin/blob/main/llm_utils.py"
click node_n6 "https://github.com/apurvsinghgautam/robin/blob/main/search.py"
click node_n7 "https://github.com/apurvsinghgautam/robin/blob/main/scrape.py"
click node_n13 "https://github.com/apurvsinghgautam/robin/tree/main/Dockerfile"
click node_n14 "https://github.com/apurvsinghgautam/robin/blob/main/entrypoint.sh"
click node_n15 "https://github.com/apurvsinghgautam/robin/blob/main/health.py"
click node_n16 "https://github.com/apurvsinghgautam/robin/blob/main/.github/workflows"
click node_n17 "https://github.com/apurvsinghgautam/robin/blob/main/.streamlit/config.toml"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_n1,node_n2,node_n12,node_n17 toneBlue
class node_n3,node_n4,node_n5,node_n6,node_n7,node_n8 toneAmber
class node_n9,node_n10,node_n11 toneMint
class node_n13,node_n14,node_n15,node_n16 toneRose
```

## Components

### Interface

User-facing Streamlit entrypoint and presentation runtime.

- **Investigator**: user · Human analyst running an OSINT investigation.
- [Streamlit UI](https://github.com/apurvsinghgautam/robin/blob/main/ui.py): app entrypoint · Top-level orchestrator for query intake, investigation flow, and result presentation. · `ui.py`
- **Streamlit runtime**: ui runtime · Streamlit execution environment launched with the app.
- [Streamlit config](https://github.com/apurvsinghgautam/robin/blob/main/.streamlit/config.toml): ui settings · Streamlit appearance and behavior configuration. · `.streamlit/config.toml`

### Core flow

Orchestration, model access, search, scraping, and persisted case data.

- [App config](https://github.com/apurvsinghgautam/robin/blob/main/config.py): settings · Central runtime settings and environment-driven configuration. · `config.py`
- [LLM core](https://github.com/apurvsinghgautam/robin/blob/main/llm.py): model adapter · Provider-agnostic model access and response handling. · `llm.py`
- [Prompt utils](https://github.com/apurvsinghgautam/robin/blob/main/llm_utils.py): prompt helpers · Prompt shaping and LLM request/response utilities. · `llm_utils.py`
- [Search engine](https://github.com/apurvsinghgautam/robin/blob/main/search.py): dark-web search · Queries dark-web search sources for investigation leads. · `search.py`
- [Scraper](https://github.com/apurvsinghgautam/robin/blob/main/scrape.py): content fetch · Retrieves and extracts page content from result targets. · `scrape.py`
- **Case history**: local persistence · Saved investigations and prior output reused across sessions.

### External deps

Network and service dependencies used during investigations.

- **Tor network**: network dependency · Required network layer for dark-web lookups and related traffic.
- **LLM providers**: external APIs · OpenAI, Claude, Gemini, Ollama, and compatible model endpoints.
- **Search targets**: dark-web sources · Search engines and reachable pages returned for investigation.

### Operations

Containerization, health, and release automation.

- [Docker image](https://github.com/apurvsinghgautam/robin/tree/main/Dockerfile): container build · Container packaging for the single-process app. · `Dockerfile`
- [Entrypoint](https://github.com/apurvsinghgautam/robin/blob/main/entrypoint.sh): container startup · Container start command that launches the Streamlit app. · `entrypoint.sh`
- [Health check](https://github.com/apurvsinghgautam/robin/blob/main/health.py): readiness probe · Lightweight runtime or container health validation. · `health.py`
- [Release CI](https://github.com/apurvsinghgautam/robin/blob/main/.github/workflows): github actions · Automated release and Docker image publication workflows. · `.github/workflows`

## Connections

- Investigator → Streamlit UI: uses · The investigator interacts with the Streamlit app to start and inspect cases.
- Streamlit UI → App config: reads · UI behavior and provider settings are sourced from central configuration.
- Streamlit UI → LLM core: orchestrates · UI calls into the model layer for query refinement and summarization.
- LLM core → Prompt utils: uses · Model access relies on prompt/response utilities.
- Streamlit UI → Search engine: triggers · Investigation queries are sent to dark-web search.
- Search engine → Tor network: routes via · Search depends on Tor-backed network access.
- Search engine → Search targets: queries · Search engines return candidate pages and leads.
- Streamlit UI → Scraper: passes leads to · Search results are handed to the scraper for content extraction.
- Scraper → Search targets: fetches · Scraper retrieves the underlying pages and artifacts.
- Scraper → LLM core: summarizes with · Extracted content is condensed and interpreted by the LLM layer.
- LLM core → LLM providers: calls · The model layer talks to configurable provider endpoints.
- Streamlit UI → Case history: saves to · Final investigation artifacts can be stored for later review.
- Streamlit UI → Streamlit runtime: runs in · The UI executes inside the Streamlit runtime.
- Docker image → Entrypoint: starts · The container image launches the app through the entrypoint script.
- Docker image → Health check: exposes · Containerized deployment includes a health/readiness hook.
- Release CI → Docker image: builds · Release workflows package and publish the container image.
- Streamlit UI → Streamlit config: styled by · Streamlit presentation settings are applied to the UI.