How miztiik/azure-fraud-detection-with-stream-analytics works
Project Type & Purpose
• This is a cloud-native, event-driven streaming pipeline built on Azure. Its purpose is to ingest high-volume “sale_event” and “inventory_event” messages, persist them for audit, filter for potential fraud (high discount + priority shipping), and route suspect events for further processing.
File Structure Highlights
• app/: Python Azure Functions (HTTP-triggered producers & Service-Bus-triggered consumers).
• app/function_code/: Local function projects with their function.json, host.json, requirements.txt.
• deployment_scripts/: Bash scripts to deploy/destroy the ARM/Bicep stack and generate test data.
• main.bicep + modules/: Bicep ARM templates that define Resource Group, VNet, Storage, Event Hub, Stream Analytics, Service Bus, Cosmos DB, Functions, Key Vault, Monitor, etc.
• README.MD: Describes end-to-end flow, sample events, Stream Analytics query, and Azure service interactions.
Core Components & Flow (to be represented
a. Azure Functions – Event Producer
• HTTP Trigger receives “count” param → generates simulated store events → publishes to Azure Event Hub
• Runs under a User-Assigned Managed Identity
b. Azure Event Hub
• Ingests events, partitions by event_type (even=inventory, odd=sale)
• Capture enabled: writes raw streams to Blob Storage (all-events) and Cosmos DB (audit)
c. Azure Stream Analytics Job
• Input: Event Hub Stream
• Query: T-SQL tumbling window filter where discount>50 AND priority_shipping=true AND event_type=‘sale_event’
• Outputs:
• Service Bus Queue (fraud-detective-output-008)
• Blob Storage container (suspicious-events)
d. Azure Functions – Fraud Analyzer
• Service Bus Queue Trigger picks up batched suspicious IDs → writes full events to Blob Storage (fraud-events)
e. Data Stores
• Blob Storage: raw capture (all-events), suspicious events, fraud-events
• Cosmos DB: raw audit store for queries and reconciliation
f. Supporting Infrastructure
• Managed Identity grants least-privilege access to Event Hub, Storage, Service Bus
• Key Vault holds secrets/config
• Monitor (Log Analytics, Alerts) watches metrics
• Optional VM/VNet modules (unused in this demo)
Relationships & Data Flow
1. HTTP Trigger → Event Hub
2. Event Hub → Capture → Blob & Cosmos
3. Event Hub → Stream Analytics → Service Bus Queue & suspicious-events blob
4. Service Bus Queue → Fraud Analyzer Function → fraud-events blob
• Managed Identity being used by Functions and Stream Analytics
Architectural Patterns & Principles
• Event-Driven Architecture: decoupled producers/consumers via Event Hub and Service Bus
• Serverless & PaaS: Functions + Stream Analytics minimize infrastructure management
• Publish-Subscribe: multiple consumers (capture, analytics) on the same event stream
• Data Lake + Schema-On-Read: raw events in Blob Storage & Cosmos DB for audit