janavrana/osint

Purpose and Deployment Shape

OSINT is an intelligence-aggregation system centered on investigations of identifiers, collecting registry data, normalizing it into facts, and deriving correlation, graph, timeline, and report views. It has a Python backend, a separate React client, and an optional graph-service package. Container deployment is represented by docker-compose.yml, backend/Dockerfile, and graph/docker-compose-additions.yml.

Entry Points and API Boundary

The backend application starts at backend/app/main.py. Its HTTP surface is assembled through backend/app/api/routes.py, with health checking at backend/app/api/health.py and investigation-oriented versioned routes at backend/app/api/v1/investigations.py. API schemas isolate transport contracts from domain persistence in backend/app/schemas/investigation.py and backend/app/schemas/timeline.py.

The frontend enters through frontend/src/main.tsx and frontend/src/App.tsx; its backend boundary is centralized in frontend/src/services/api.ts. Existing pages include investigation, graph, report, and timeline screens, though the README identifies backend integration and these visualizations as incomplete.

Investigation Execution Pipeline

backend/app/services/investigation_service.py coordinates investigation lifecycle operations backed by backend/app/repositories/investigation_repository.py and the persisted backend/app/models/investigation.py model.

Connector execution is separated into backend/app/services/connector_execution_service.py. It selects registered implementations through backend/app/connectors/manager.py and backend/app/connectors/registry.py, using a common connector contract in backend/app/connectors/base.py. Current external intelligence sources are WHOIS (backend/app/connectors/whois/connector.py) and RDAP (backend/app/connectors/rdap/connector.py); raw outcomes are represented and stored as connector results via backend/app/models/connector_result.py and backend/app/repositories/connector_result_repository.py.

Normalized Fact Boundary

Source-specific responses are converted into a source-independent fact layer by backend/app/normalizers/manager.py and backend/app/normalizers/base.py. WHOIS and RDAP transformations live in backend/app/normalizers/whois/normalizer.py and backend/app/normalizers/rdap/normalizer.py.

Normalized facts are a material persistence boundary: backend/app/models/normalized_fact.py, backend/app/repositories/normalized_fact_repository.py, and backend/app/services/normalized_fact_service.py. The Alembic migration backend/alembic/versions/708401118290_add_normalized_facts_table.py indicates this store is intended to support downstream analysis independent of raw connector payloads.

Analysis Products

Correlation derives relationships or findings from normalized data through backend/app/correlation/engine.py and its typed contracts in backend/app/correlation/types.py.

Graph generation/query functionality is layered under backend/app/graph/: generation and mapping (generator.py, mapper.py), domain models (models.py), and query-facing service logic (query_service.py, service.py). A distinct graph runtime integration is suggested by graph/service.py, graph/connection.py, and graph/config.py, keeping graph infrastructure separate from the primary backend.

Timeline construction is handled by backend/app/timeline/service.py over backend/app/timeline/models.py. Reporting composes durable report records and generated sections via backend/app/reporting/report_generator.py, backend/app/reporting/sections.py, and backend/app/services/report_service.py.

Persistence, Runtime, and Observability

The backend uses a database session boundary at backend/app/db/session.py, SQL-model-style domain entities under backend/app/models/, repositories for persistence access, and Alembic under backend/alembic/ for schema evolution. Configuration, logging, and metrics are centralized in backend/app/core/config.py, backend/app/core/logging.py, and backend/app/core/metrics.py.

Material external dependencies are WHOIS and RDAP services. The frontend is TypeScript/React, while backend and graph packages are Python-based; backend dependencies are declared in backend/requirements.txt, with graph-specific additions in graph/requirements-additions.txt.