# dchinthan/iam-anomaly-detector architecture

> Architecture diagram of the GitHub repository dchinthan/iam-anomaly-detector, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dchinthan/iam-anomaly-detector
- Repository: https://github.com/dchinthan/iam-anomaly-detector

## Overview

IAM Anomaly Detector ingests AWS or GCP audit events, or synthetic/offline data, then aggregates per-user behavior into features and scores it with an unsupervised ensemble. The CLI owns batch orchestration; the Streamlit dashboard offers analysis and role-gated views, while a separate streaming path incrementally resc scores incoming events. Results can be persisted through cloud-specific stores, with optional cached Claude summaries and drift monitoring. The graph follows documented workflows and shown call sites; some unsampled subsystem wiring is omitted.

Main entry points: [`main.py`](https://github.com/dchinthan/iam-anomaly-detector/blob/main/main.py) (CLI Orchestrator), [`dashboard/app.py`](https://github.com/dchinthan/iam-anomaly-detector/blob/main/dashboard/app.py) (Dashboard), [`streaming/stream_processor.py`](https://github.com/dchinthan/iam-anomaly-detector/blob/main/streaming/stream_processor.py) (Stream Processor), [`models/detector.py`](https://github.com/dchinthan/iam-anomaly-detector/blob/main/models/detector.py) (Ensemble Detector)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ingestion["Event Ingestion"]
  node_synthetic["Synthetic Logs<br/>[log_generator.py]"]
  node_lanl["LANL Adapter<br/>[lanl_adapter.py]"]
  node_aws_ingest["AWS Ingestion"]
  node_gcp_ingest["GCP Ingestion"]
end

subgraph group_analysis["Detection Core"]
  node_features["Behavior Features<br/>[extractor.py]"]
  node_detector["Ensemble Detector<br/>[detector.py]"]
  node_autoencoder["Autoencoder<br/>[autoencoder.py]"]
  node_drift["Drift Monitor<br/>[drift.py]"]
  node_insights["Incident Insights<br/>[insights.py]"]
  node_cache[("Insight Cache<br/>[cache.py]")]
end

subgraph group_streaming["Live Scoring"]
  node_event_stream["Stream Broker Adapter<br/>[event_stream.py]"]
  node_pubsub["Pub/Sub Backend<br/>[pubsub_backend.py]"]
  node_stream_processor["Stream Processor"]
end

subgraph group_persistence["Persistence"]
  node_log_db[("IAM Log Store<br/>[db.py]")]
  node_aws_store[("DynamoDB Store<br/>[dynamodb_store.py]")]
  node_gcp_store[("Firestore Store<br/>[firestore_store.py]")]
end

subgraph group_experience["Analyst Experience"]
  node_main["CLI Orchestrator<br/>[main.py]"]
  node_dashboard["Dashboard<br/>[app.py]"]
  node_auth["Role Authorization<br/>[auth.py]"]
end

node_operator(("Operator"))
node_claude{{"Claude API"}}
node_analyst(("Analyst"))

node_operator -->|"runs commands"| node_main
node_main -->|"generates logs"| node_synthetic
node_main -->|"loads LANL data"| node_lanl
node_main -->|"selects ingestion"| node_aws_ingest
node_main -->|"selects ingestion"| node_gcp_ingest
node_synthetic -->|"inserts events"| node_log_db
node_aws_ingest -->|"inserts events"| node_log_db
node_gcp_ingest -->|"writes events"| node_log_db
node_main -->|"extracts features"| node_features
node_features -->|"loads events"| node_log_db
node_main -->|"trains and scores"| node_detector
node_detector -->|"scores with"| node_autoencoder
node_detector -->|"saves baseline"| node_drift
node_main -->|"analyzes alerts"| node_insights
node_insights -->|"reads and writes"| node_cache
node_insights -.->|"requests summaries"| node_claude
node_main -->|"persists scores"| node_aws_store
node_main -->|"selects persistence"| node_gcp_store
node_operator -->|"starts streaming"| node_stream_processor
node_stream_processor -->|"consumes events"| node_event_stream
node_event_stream -->|"dispatches Pub/Sub"| node_pubsub
node_stream_processor -->|"inserts events"| node_log_db
node_stream_processor -->|"extracts window features"| node_features
node_stream_processor -->|"scores windows"| node_detector
node_stream_processor -->|"persists scores"| node_aws_store
node_analyst -->|"views analysis"| node_dashboard
node_dashboard -->|"checks permissions"| node_auth
node_dashboard -->|"extracts features"| node_features
node_dashboard -->|"scores users"| node_detector
node_dashboard -->|"selects store"| node_aws_store
node_dashboard -->|"selects store"| node_gcp_store
node_dashboard -->|"generates summaries"| node_insights

click node_main "https://github.com/dchinthan/iam-anomaly-detector/blob/main/main.py"
click node_synthetic "https://github.com/dchinthan/iam-anomaly-detector/blob/main/data/log_generator.py"
click node_lanl "https://github.com/dchinthan/iam-anomaly-detector/blob/main/data/lanl_adapter.py"
click node_aws_ingest "https://github.com/dchinthan/iam-anomaly-detector/blob/main/aws/cloudwatch_client.py"
click node_gcp_ingest "https://github.com/dchinthan/iam-anomaly-detector/blob/main/gcp/cloud_logging_client.py"
click node_log_db "https://github.com/dchinthan/iam-anomaly-detector/blob/main/data/db.py"
click node_features "https://github.com/dchinthan/iam-anomaly-detector/blob/main/features/extractor.py"
click node_detector "https://github.com/dchinthan/iam-anomaly-detector/blob/main/models/detector.py"
click node_autoencoder "https://github.com/dchinthan/iam-anomaly-detector/blob/main/models/autoencoder.py"
click node_drift "https://github.com/dchinthan/iam-anomaly-detector/blob/main/models/drift.py"
click node_insights "https://github.com/dchinthan/iam-anomaly-detector/blob/main/genai/insights.py"
click node_cache "https://github.com/dchinthan/iam-anomaly-detector/blob/main/genai/cache.py"
click node_aws_store "https://github.com/dchinthan/iam-anomaly-detector/blob/main/aws/dynamodb_store.py"
click node_gcp_store "https://github.com/dchinthan/iam-anomaly-detector/blob/main/gcp/firestore_store.py"
click node_event_stream "https://github.com/dchinthan/iam-anomaly-detector/blob/main/streaming/event_stream.py"
click node_pubsub "https://github.com/dchinthan/iam-anomaly-detector/blob/main/streaming/pubsub_backend.py"
click node_stream_processor "https://github.com/dchinthan/iam-anomaly-detector/blob/main/streaming/stream_processor.py"
click node_dashboard "https://github.com/dchinthan/iam-anomaly-detector/blob/main/dashboard/app.py"
click node_auth "https://github.com/dchinthan/iam-anomaly-detector/blob/main/dashboard/auth.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_synthetic,node_lanl,node_aws_ingest,node_gcp_ingest toneBlue
class node_features,node_detector,node_autoencoder,node_drift,node_insights,node_cache toneAmber
class node_event_stream,node_pubsub,node_stream_processor,node_claude toneMint
class node_log_db,node_aws_store,node_gcp_store toneRose
class node_main,node_dashboard,node_auth,node_operator,node_analyst toneIndigo
```

## Components

### Event Ingestion

- [Synthetic Logs](https://github.com/dchinthan/iam-anomaly-detector/blob/main/data/log_generator.py): `data/log_generator.py`
- [LANL Adapter](https://github.com/dchinthan/iam-anomaly-detector/blob/main/data/lanl_adapter.py): `data/lanl_adapter.py`
- [AWS Ingestion](https://github.com/dchinthan/iam-anomaly-detector/blob/main/aws/cloudwatch_client.py): `aws/cloudwatch_client.py`
- [GCP Ingestion](https://github.com/dchinthan/iam-anomaly-detector/blob/main/gcp/cloud_logging_client.py): `gcp/cloud_logging_client.py`

### Detection Core

- [Behavior Features](https://github.com/dchinthan/iam-anomaly-detector/blob/main/features/extractor.py): `features/extractor.py`
- [Ensemble Detector](https://github.com/dchinthan/iam-anomaly-detector/blob/main/models/detector.py): `models/detector.py`
- [Autoencoder](https://github.com/dchinthan/iam-anomaly-detector/blob/main/models/autoencoder.py): `models/autoencoder.py`
- [Drift Monitor](https://github.com/dchinthan/iam-anomaly-detector/blob/main/models/drift.py): `models/drift.py`
- [Incident Insights](https://github.com/dchinthan/iam-anomaly-detector/blob/main/genai/insights.py): `genai/insights.py`
- [Insight Cache](https://github.com/dchinthan/iam-anomaly-detector/blob/main/genai/cache.py): `genai/cache.py`

### Live Scoring

- [Stream Broker Adapter](https://github.com/dchinthan/iam-anomaly-detector/blob/main/streaming/event_stream.py): `streaming/event_stream.py`
- [Pub/Sub Backend](https://github.com/dchinthan/iam-anomaly-detector/blob/main/streaming/pubsub_backend.py): `streaming/pubsub_backend.py`
- [Stream Processor](https://github.com/dchinthan/iam-anomaly-detector/blob/main/streaming/stream_processor.py): `streaming/stream_processor.py`

### Persistence

- [IAM Log Store](https://github.com/dchinthan/iam-anomaly-detector/blob/main/data/db.py): `data/db.py`
- [DynamoDB Store](https://github.com/dchinthan/iam-anomaly-detector/blob/main/aws/dynamodb_store.py): `aws/dynamodb_store.py`
- [Firestore Store](https://github.com/dchinthan/iam-anomaly-detector/blob/main/gcp/firestore_store.py): `gcp/firestore_store.py`

### Analyst Experience

- [CLI Orchestrator](https://github.com/dchinthan/iam-anomaly-detector/blob/main/main.py): `main.py`
- [Dashboard](https://github.com/dchinthan/iam-anomaly-detector/blob/main/dashboard/app.py): `dashboard/app.py`
- [Role Authorization](https://github.com/dchinthan/iam-anomaly-detector/blob/main/dashboard/auth.py): `dashboard/auth.py`

### Other components

- **Operator**
- **Claude API**
- **Analyst**

## Connections

- Operator → CLI Orchestrator: runs commands (evidence: `README.md`)
- CLI Orchestrator → Synthetic Logs: generates logs (evidence: `main.py`)
- CLI Orchestrator → LANL Adapter: loads LANL data (evidence: `main.py`)
- CLI Orchestrator → AWS Ingestion: selects ingestion (evidence: `main.py`)
- CLI Orchestrator → GCP Ingestion: selects ingestion (evidence: `main.py`)
- Synthetic Logs → IAM Log Store: inserts events (evidence: `data/log_generator.py`)
- AWS Ingestion → IAM Log Store: inserts events (evidence: `aws/cloudwatch_client.py`)
- GCP Ingestion → IAM Log Store: writes events (evidence: `gcp/cloud_logging_client.py`)
- CLI Orchestrator → Behavior Features: extracts features (evidence: `main.py`)
- Behavior Features → IAM Log Store: loads events (evidence: `features/extractor.py`)
- CLI Orchestrator → Ensemble Detector: trains and scores (evidence: `main.py`)
- Ensemble Detector → Autoencoder: scores with (evidence: `models/detector.py`)
- Ensemble Detector → Drift Monitor: saves baseline (evidence: `models/detector.py`)
- CLI Orchestrator → Incident Insights: analyzes alerts (evidence: `main.py`)
- Incident Insights → Insight Cache: reads and writes (evidence: `genai/insights.py`)
- Incident Insights → Claude API: requests summaries (evidence: `genai/insights.py`)
- CLI Orchestrator → DynamoDB Store: persists scores (evidence: `main.py`)
- CLI Orchestrator → Firestore Store: selects persistence (evidence: `main.py`)
- Operator → Stream Processor: starts streaming
- Stream Processor → Stream Broker Adapter: consumes events (evidence: `streaming/stream_processor.py`)
- Stream Broker Adapter → Pub/Sub Backend: dispatches Pub/Sub (evidence: `streaming/event_stream.py`)
- Stream Processor → IAM Log Store: inserts events (evidence: `streaming/stream_processor.py`)
- Stream Processor → Behavior Features: extracts window features (evidence: `streaming/stream_processor.py`)
- Stream Processor → Ensemble Detector: scores windows (evidence: `streaming/stream_processor.py`)
- Stream Processor → DynamoDB Store: persists scores (evidence: `streaming/stream_processor.py`)
- Analyst → Dashboard: views analysis (evidence: `README.md`)
- Dashboard → Role Authorization: checks permissions (evidence: `dashboard/app.py`)
- Dashboard → Behavior Features: extracts features (evidence: `dashboard/app.py`)
- Dashboard → Ensemble Detector: scores users (evidence: `dashboard/app.py`)
- Dashboard → DynamoDB Store: selects store (evidence: `dashboard/app.py`)
- Dashboard → Firestore Store: selects store (evidence: `dashboard/app.py`)
- Dashboard → Incident Insights: generates summaries (evidence: `README.md`)