reconurge/flowsint

Purpose and deployment

Flowsint is a self-hosted OSINT investigation system: users create investigations, model evidence as entity/relationship graphs, run enrichment workflows, and retain analyses, activity, and encrypted third-party keys locally. Docker Compose is the operational entry point: docker-compose.prod.yml exposes only the frontend/proxy on port 5173; flowsint-app/nginx.conf proxies backend traffic and applies Host-header protection.

Application entry points

The browser application starts at flowsint-app/src/main.tsx, with TanStack-style generated routing rooted in flowsint-app/src/routes/__root.tsx. Authenticated dashboard routes lead to investigations, sketches, flows, enrichers, vault, and analyses.

The Python HTTP boundary is FastAPI at flowsint-api/app/main.py. Route modules expose domain APIs—especially flowsint-api/app/api/routes/scan.py, flowsint-api/app/api/routes/sketches.py, and flowsint-api/app/api/routes/events.py—while schemas define request/response contracts.

Core domain and persistence boundary

flowsint-core centralizes business logic behind services and repositories. Services coordinate permissions, validation, persistence, and execution for investigations, flows, scans, sketches, keys, chat, and analysis; repositories isolate PostgreSQL persistence.

Graph-shaped investigation data is separately handled through Neo4j-oriented graph abstractions: flowsint-core/src/flowsint_core/core/graph/service.py, flowsint-core/src/flowsint_core/core/graph/repository.py, and flowsint-core/src/flowsint_core/core/graph/serializer.py. Neo4j migrations reside in neo4j-migrations/001_indexes.cypher and related migration files; relational schema evolution is Alembic under flowsint-api/alembic/versions.

Type system and graph semantics

flowsint-types provides the Pydantic entity vocabulary used as graph nodes and enricher inputs/outputs, including flowsint-types/src/flowsint_types/flowsint_base.py, flowsint-types/src/flowsint_types/domain.py, and flowsint-types/src/flowsint_types/ip.py. Core graph types and serialization mediate this typed model from enrichers through persistence to frontend graph payloads.

Custom types extend the built-in vocabulary through flowsint-core/src/flowsint_core/core/services/custom_type_service.py and API/UI surfaces rather than requiring a package-level entity class.

Enrichment and asynchronous execution

An investigation graph supplies entities to enrichers or visual flows. The orchestrator and task layer dispatch execution, persist scan/log results, mutate graph state, and emit events: flowsint-core/src/flowsint_core/core/orchestrator.py, flowsint-core/src/flowsint_core/tasks/enricher.py, and flowsint-core/src/flowsint_core/tasks/flow.py. Celery configuration in flowsint-core/src/flowsint_core/core/celery.py makes Redis a material asynchronous-work dependency.

Enrichers are modular transformations registered in flowsint-enrichers/src/flowsint_enrichers/registry.py, implemented per input domain (for example flowsint-enrichers/src/flowsint_enrichers/domain/to_dns.py), and built on the common contract in flowsint-core/src/flowsint_core/core/enricher_base.py. Tool adapters may invoke network utilities or containerized tools, e.g. flowsint-enrichers/src/tools/dockertool.py.

Visual investigation workspace

The React frontend separates remote API access from interactive state. Typed service clients under flowsint-app/src/api/api.ts and flowsint-app/src/api/scan-service.ts call FastAPI; server-sent events in flowsint-app/src/api/sse.ts update scans/logs in real time.

Sketches render and edit the persisted graph through flowsint-app/src/components/sketches/graph/index.tsx, transformation logic in flowsint-app/src/components/sketches/graph/utils/graph-data-transformer.ts, and graph state in flowsint-app/src/stores/graph-store.ts. Layout and pathfinding run off the UI thread in flowsint-app/src/workers/layout.worker.ts and flowsint-app/src/workers/pathfinder.worker.ts. The flow editor builds visual enrichment pipelines at flowsint-app/src/components/flows/editor.tsx.

Security and external integrations

Authentication/authorization is enforced at API dependencies and core services, with permissions in flowsint-api/app/api/deps.py and flowsint-api/app/security/permissions.py. User API keys are encrypted through flowsint-core/src/flowsint_core/core/vault.py and managed by flowsint-core/src/flowsint_core/core/services/vault_service.py.

Material infrastructure is PostgreSQL for application records, Neo4j for investigation graphs, Redis/Celery for background execution, and Nginx for frontend serving/proxying. Optional external services are invoked by enricher tools (DNS, WHOIS, breach, crawler, social, blockchain providers) and chat/template generation can select OpenAI or Mistral providers through flowsint-core/src/flowsint_core/core/llm/factory.py.