# reconurge/flowsint architecture

> Architecture diagram of the GitHub repository reconurge/flowsint, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/reconurge/flowsint
- Repository: https://github.com/reconurge/flowsint

## Overview

#### Purpose and deployment

Flowsint is a self-hosted OSINT investigation system: users create investigations, model evidence as entity/relationship graphs, run enrichment workflows, and retain analyses, activity, and encrypted third-party keys locally. Docker Compose is the operational entry point: `docker-compose.prod.yml` exposes only the frontend/proxy on port 5173; `flowsint-app/nginx.conf` proxies backend traffic and applies Host-header protection.

#### Application entry points

The browser application starts at `flowsint-app/src/main.tsx`, with TanStack-style generated routing rooted in `flowsint-app/src/routes/__root.tsx`. Authenticated dashboard routes lead to investigations, sketches, flows, enrichers, vault, and analyses.

The Python HTTP boundary is FastAPI at `flowsint-api/app/main.py`. Route modules expose domain APIs—especially `flowsint-api/app/api/routes/scan.py`, `flowsint-api/app/api/routes/sketches.py`, and `flowsint-api/app/api/routes/events.py`—while schemas define request/response contracts.

#### Core domain and persistence boundary

`flowsint-core` centralizes business logic behind services and repositories. Services coordinate permissions, validation, persistence, and execution for investigations, flows, scans, sketches, keys, chat, and analysis; repositories isolate PostgreSQL persistence.

Graph-shaped investigation data is separately handled through Neo4j-oriented graph abstractions: `flowsint-core/src/flowsint_core/core/graph/service.py`, `flowsint-core/src/flowsint_core/core/graph/repository.py`, and `flowsint-core/src/flowsint_core/core/graph/serializer.py`. Neo4j migrations reside in `neo4j-migrations/001_indexes.cypher` and related migration files; relational schema evolution is Alembic under `flowsint-api/alembic/versions`.

#### Type system and graph semantics

`flowsint-types` provides the Pydantic entity vocabulary used as graph nodes and enricher inputs/outputs, including `flowsint-types/src/flowsint_types/flowsint_base.py`, `flowsint-types/src/flowsint_types/domain.py`, and `flowsint-types/src/flowsint_types/ip.py`. Core graph types and serialization mediate this typed model from enrichers through persistence to frontend graph payloads.

Custom types extend the built-in vocabulary through `flowsint-core/src/flowsint_core/core/services/custom_type_service.py` and API/UI surfaces rather than requiring a package-level entity class.

#### Enrichment and asynchronous execution

An investigation graph supplies entities to enrichers or visual flows. The orchestrator and task layer dispatch execution, persist scan/log results, mutate graph state, and emit events: `flowsint-core/src/flowsint_core/core/orchestrator.py`, `flowsint-core/src/flowsint_core/tasks/enricher.py`, and `flowsint-core/src/flowsint_core/tasks/flow.py`. Celery configuration in `flowsint-core/src/flowsint_core/core/celery.py` makes Redis a material asynchronous-work dependency.

Enrichers are modular transformations registered in `flowsint-enrichers/src/flowsint_enrichers/registry.py`, implemented per input domain (for example `flowsint-enrichers/src/flowsint_enrichers/domain/to_dns.py`), and built on the common contract in `flowsint-core/src/flowsint_core/core/enricher_base.py`. Tool adapters may invoke network utilities or containerized tools, e.g. `flowsint-enrichers/src/tools/dockertool.py`.

#### Visual investigation workspace

The React frontend separates remote API access from interactive state. Typed service clients under `flowsint-app/src/api/api.ts` and `flowsint-app/src/api/scan-service.ts` call FastAPI; server-sent events in `flowsint-app/src/api/sse.ts` update scans/logs in real time.

Sketches render and edit the persisted graph through `flowsint-app/src/components/sketches/graph/index.tsx`, transformation logic in `flowsint-app/src/components/sketches/graph/utils/graph-data-transformer.ts`, and graph state in `flowsint-app/src/stores/graph-store.ts`. Layout and pathfinding run off the UI thread in `flowsint-app/src/workers/layout.worker.ts` and `flowsint-app/src/workers/pathfinder.worker.ts`. The flow editor builds visual enrichment pipelines at `flowsint-app/src/components/flows/editor.tsx`.

#### Security and external integrations

Authentication/authorization is enforced at API dependencies and core services, with permissions in `flowsint-api/app/api/deps.py` and `flowsint-api/app/security/permissions.py`. User API keys are encrypted through `flowsint-core/src/flowsint_core/core/vault.py` and managed by `flowsint-core/src/flowsint_core/core/services/vault_service.py`.

Material infrastructure is PostgreSQL for application records, Neo4j for investigation graphs, Redis/Celery for background execution, and Nginx for frontend serving/proxying. Optional external services are invoked by enricher tools (DNS, WHOIS, breach, crawler, social, blockchain providers) and chat/template generation can select OpenAI or Mistral providers through `flowsint-core/src/flowsint_core/core/llm/factory.py`.

Main entry points: [`flowsint-core/src/flowsint_core/tasks/enricher.py`](https://github.com/reconurge/flowsint/blob/main/flowsint-core/src/flowsint_core/tasks/enricher.py) (Celery enricher and flow tasks), [`flowsint-app/src/main.tsx`](https://github.com/reconurge/flowsint/blob/main/flowsint-app/src/main.tsx) (React application), [`flowsint-core/src/flowsint_core/core/graph/service.py`](https://github.com/reconurge/flowsint/blob/main/flowsint-core/src/flowsint_core/core/graph/service.py) (Graph service and repository), [`flowsint-api/app/api/routes/scan.py`](https://github.com/reconurge/flowsint/blob/main/flowsint-api/app/api/routes/scan.py) (Scan, sketch and event routes)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ui["Browser workspace"]
  node_nginx{{"Nginx proxy<br/>edge proxy<br/>[nginx.conf]"}}
  node_frontend["React application<br/>browser SPA<br/>[main.tsx]"]
  node_api_clients["API and SSE clients<br/>frontend data layer<br/>[sse.ts]"]
  node_sketch_workspace["Sketch graph workspace<br/>graph editor<br/>[index.tsx]"]
  node_graph_state["Graph state and workers<br/>interactive state<br/>[graph-store.ts]"]
  node_flow_editor["Visual flow editor<br/>pipeline editor<br/>[editor.tsx]"]
end

subgraph group_api["HTTP boundary"]
  node_fastapi{{"FastAPI application<br/>HTTP API<br/>[main.py]"}}
  node_api_routes["Scan, sketch and event routes<br/>route modules<br/>[scan.py]"]
  node_authz["Authorization dependencies<br/>API security<br/>[deps.py]"]
end

subgraph group_core["Domain core"]
  node_domain_services["Domain services<br/>business layer"]
  node_graph_service["Graph service and repository<br/>graph persistence layer<br/>[service.py]"]
  node_entity_types["Typed entity vocabulary<br/>Pydantic models<br/>[flowsint_base.py]"]
  node_vault["Encrypted key vault<br/>secrets service<br/>[vault.py]"]
end

subgraph group_execution["Enrichment execution"]
  node_orchestrator["Execution orchestrator<br/>workflow dispatcher<br/>[orchestrator.py]"]
  node_celery_tasks["Celery enricher and flow tasks<br/>background workers<br/>[enricher.py]"]
  node_enricher_registry["Enricher registry and tools<br/>plugin transformations<br/>[registry.py]"]
  node_external_providers["OSINT and LLM providers<br/>external integrations<br/>[factory.py]"]
end

subgraph group_storage["Persistence &amp; infrastructure"]
  node_postgres[("PostgreSQL<br/>application database")]
  node_neo4j[("Neo4j<br/>investigation graph database")]
  node_redis["Redis<br/>task broker"]
end

node_nginx -->|"serves"| node_frontend
node_nginx -->|"proxies API traffic"| node_fastapi
node_frontend -->|"uses"| node_api_clients
node_api_clients -->|"HTTP and SSE"| node_fastapi
node_frontend -->|"hosts"| node_sketch_workspace
node_sketch_workspace -->|"updates"| node_graph_state
node_frontend -->|"hosts"| node_flow_editor
node_fastapi -->|"mounts"| node_api_routes
node_api_routes -->|"authorizes via"| node_authz
node_authz -->|"permits service calls"| node_domain_services
node_domain_services -->|"persists records"| node_postgres
node_domain_services -->|"manages investigation graph"| node_graph_service
node_graph_service -->|"serializes"| node_entity_types
node_graph_service -->|"reads and writes"| node_neo4j
node_domain_services -->|"manages keys"| node_vault
node_domain_services -->|"starts executions"| node_orchestrator
node_orchestrator -->|"dispatches"| node_celery_tasks
node_celery_tasks -->|"uses broker"| node_redis
node_celery_tasks -->|"runs enrichers"| node_enricher_registry
node_enricher_registry -->|"consumes and produces"| node_entity_types
node_celery_tasks -->|"mutates graph state"| node_graph_service
node_celery_tasks -->|"stores scan and log results"| node_postgres
node_enricher_registry -->|"invokes"| node_external_providers
node_api_routes -->|"emits events"| node_api_clients

click node_nginx "https://github.com/reconurge/flowsint/blob/main/flowsint-app/nginx.conf"
click node_frontend "https://github.com/reconurge/flowsint/blob/main/flowsint-app/src/main.tsx"
click node_api_clients "https://github.com/reconurge/flowsint/blob/main/flowsint-app/src/api/sse.ts"
click node_sketch_workspace "https://github.com/reconurge/flowsint/blob/main/flowsint-app/src/components/sketches/graph/index.tsx"
click node_graph_state "https://github.com/reconurge/flowsint/blob/main/flowsint-app/src/stores/graph-store.ts"
click node_flow_editor "https://github.com/reconurge/flowsint/blob/main/flowsint-app/src/components/flows/editor.tsx"
click node_fastapi "https://github.com/reconurge/flowsint/blob/main/flowsint-api/app/main.py"
click node_api_routes "https://github.com/reconurge/flowsint/blob/main/flowsint-api/app/api/routes/scan.py"
click node_authz "https://github.com/reconurge/flowsint/blob/main/flowsint-api/app/api/deps.py"
click node_graph_service "https://github.com/reconurge/flowsint/blob/main/flowsint-core/src/flowsint_core/core/graph/service.py"
click node_entity_types "https://github.com/reconurge/flowsint/blob/main/flowsint-types/src/flowsint_types/flowsint_base.py"
click node_vault "https://github.com/reconurge/flowsint/blob/main/flowsint-core/src/flowsint_core/core/vault.py"
click node_orchestrator "https://github.com/reconurge/flowsint/blob/main/flowsint-core/src/flowsint_core/core/orchestrator.py"
click node_celery_tasks "https://github.com/reconurge/flowsint/blob/main/flowsint-core/src/flowsint_core/tasks/enricher.py"
click node_enricher_registry "https://github.com/reconurge/flowsint/blob/main/flowsint-enrichers/src/flowsint_enrichers/registry.py"
click node_external_providers "https://github.com/reconurge/flowsint/blob/main/flowsint-core/src/flowsint_core/core/llm/factory.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_nginx,node_frontend,node_api_clients,node_sketch_workspace,node_graph_state,node_flow_editor toneBlue
class node_fastapi,node_api_routes,node_authz toneAmber
class node_domain_services,node_graph_service,node_entity_types,node_vault toneMint
class node_orchestrator,node_celery_tasks,node_enricher_registry,node_external_providers toneRose
class node_postgres,node_neo4j,node_redis toneIndigo
```

## Components

### Browser workspace

- [Nginx proxy](https://github.com/reconurge/flowsint/blob/main/flowsint-app/nginx.conf): edge proxy · Only public Compose endpoint on port 5173. · `flowsint-app/nginx.conf`
- [React application](https://github.com/reconurge/flowsint/blob/main/flowsint-app/src/main.tsx): browser SPA · Authenticated investigation workspace and route shell. · `flowsint-app/src/main.tsx`
- [API and SSE clients](https://github.com/reconurge/flowsint/blob/main/flowsint-app/src/api/sse.ts): frontend data layer · Calls HTTP APIs and receives live scan/log updates. · `flowsint-app/src/api/sse.ts`
- [Sketch graph workspace](https://github.com/reconurge/flowsint/blob/main/flowsint-app/src/components/sketches/graph/index.tsx): graph editor · Edits and renders persisted investigation graphs. · `flowsint-app/src/components/sketches/graph/index.tsx`
- [Graph state and workers](https://github.com/reconurge/flowsint/blob/main/flowsint-app/src/stores/graph-store.ts): interactive state · Transforms graph data and offloads layout/pathfinding. · `flowsint-app/src/stores/graph-store.ts`
- [Visual flow editor](https://github.com/reconurge/flowsint/blob/main/flowsint-app/src/components/flows/editor.tsx): pipeline editor · `flowsint-app/src/components/flows/editor.tsx`

### HTTP boundary

- [FastAPI application](https://github.com/reconurge/flowsint/blob/main/flowsint-api/app/main.py): HTTP API · Domain API boundary for dashboard clients. · `flowsint-api/app/main.py`
- [Scan, sketch and event routes](https://github.com/reconurge/flowsint/blob/main/flowsint-api/app/api/routes/scan.py): route modules · Expose execution, graph, and event APIs. · `flowsint-api/app/api/routes/scan.py`
- [Authorization dependencies](https://github.com/reconurge/flowsint/blob/main/flowsint-api/app/api/deps.py): API security · Enforces authentication and permissions before service calls. · `flowsint-api/app/api/deps.py`

### Domain core

- **Domain services**: business layer · Coordinates investigations, flows, scans, sketches, vault, and analysis.
- [Graph service and repository](https://github.com/reconurge/flowsint/blob/main/flowsint-core/src/flowsint_core/core/graph/service.py): graph persistence layer · Serializes typed entities between graph storage and frontend payloads. · `flowsint-core/src/flowsint_core/core/graph/service.py`
- [Typed entity vocabulary](https://github.com/reconurge/flowsint/blob/main/flowsint-types/src/flowsint_types/flowsint_base.py): Pydantic models · Defines entity and enricher input/output semantics. · `flowsint-types/src/flowsint_types/flowsint_base.py`
- [Encrypted key vault](https://github.com/reconurge/flowsint/blob/main/flowsint-core/src/flowsint_core/core/vault.py): secrets service · Encrypts and manages user third-party API keys. · `flowsint-core/src/flowsint_core/core/vault.py`

### Enrichment execution

- [Execution orchestrator](https://github.com/reconurge/flowsint/blob/main/flowsint-core/src/flowsint_core/core/orchestrator.py): workflow dispatcher · Dispatches flows and enrichers and records execution results. · `flowsint-core/src/flowsint_core/core/orchestrator.py`
- [Celery enricher and flow tasks](https://github.com/reconurge/flowsint/blob/main/flowsint-core/src/flowsint_core/tasks/enricher.py): background workers · Run asynchronous enrichment and graph mutation. · `flowsint-core/src/flowsint_core/tasks/enricher.py`
- [Enricher registry and tools](https://github.com/reconurge/flowsint/blob/main/flowsint-enrichers/src/flowsint_enrichers/registry.py): plugin transformations · Selects domain transformations and optional network/container tools. · `flowsint-enrichers/src/flowsint_enrichers/registry.py`
- [OSINT and LLM providers](https://github.com/reconurge/flowsint/blob/main/flowsint-core/src/flowsint_core/core/llm/factory.py): external integrations · DNS, WHOIS, breach, crawler, social, blockchain, OpenAI, and Mistral services. · `flowsint-core/src/flowsint_core/core/llm/factory.py`

### Persistence & infrastructure

- **PostgreSQL**: application database · Stores relational records, scans, logs, and analyses.
- **Neo4j**: investigation graph database · Stores entity and relationship graphs.
- **Redis**: task broker · Celery asynchronous-work dependency.

## Connections

- Nginx proxy → React application: serves
- Nginx proxy → FastAPI application: proxies API traffic
- React application → API and SSE clients: uses
- API and SSE clients → FastAPI application: HTTP and SSE
- React application → Sketch graph workspace: hosts
- Sketch graph workspace → Graph state and workers: updates
- React application → Visual flow editor: hosts
- FastAPI application → Scan, sketch and event routes: mounts
- Scan, sketch and event routes → Authorization dependencies: authorizes via
- Authorization dependencies → Domain services: permits service calls
- Domain services → PostgreSQL: persists records
- Domain services → Graph service and repository: manages investigation graph
- Graph service and repository → Typed entity vocabulary: serializes
- Graph service and repository → Neo4j: reads and writes
- Domain services → Encrypted key vault: manages keys
- Domain services → Execution orchestrator: starts executions
- Execution orchestrator → Celery enricher and flow tasks: dispatches
- Celery enricher and flow tasks → Redis: uses broker
- Celery enricher and flow tasks → Enricher registry and tools: runs enrichers
- Enricher registry and tools → Typed entity vocabulary: consumes and produces
- Celery enricher and flow tasks → Graph service and repository: mutates graph state
- Celery enricher and flow tasks → PostgreSQL: stores scan and log results
- Enricher registry and tools → OSINT and LLM providers: invokes
- Scan, sketch and event routes → API and SSE clients: emits events