How cisagov/malcolm works
Malcolm is a deployable network-security monitoring suite: users submit PCAPs, Zeek logs, or Suricata alerts—or forward/capture traffic—and analyze normalized, enriched results in OpenSearch Dashboards and Arkime. The graph preserves the documented ingestion, processing, enrichment, storage, and analysis stages, plus file analysis and deployment configuration. README descriptions support the broad workflow; sampled source confirms upload handling, queued PCAP processing, API access, and kiosk service control, but does not establish every container-to-container connection.
Main entry points: pcap_processor.py PCAP Processor · dashboards OpenSearch · logstash Normalization Enrichment · zeek Zeek Analysis
Components 21
Traffic Ingestion
- File Upload
file-upload/php/submit.php - Upload Watchers
filebeat/scripts/filebeat-watch-zeeklogs-uploads-folder.py - PCAP Watcher
shared/bin/pcap_watcher.py
Processing Enrichment
- PCAP Processor
shared/bin/pcap_processor.py - Zeek Analysis
zeek - Suricata Analysis
suricata - File Scanning
filescan/python-filescan/filescan - Normalization Enrichment
logstash - Asset Inventory
netbox
Data Services
- OpenSearch
dashboards
Analysis Interfaces
- Arkime Sessions
arkime - WISE Fields
arkime/wise/source.zeeklogs.js - Dashboards
dashboards - Malcolm API
api/project/__init__.py
Deployment Operations
- Installer Configuration
scripts/installer - Kiosk Controls
malcolm-iso/kiosk/interface/routes.py - Service Operations
malcolm-iso/kiosk/interface/sysquery/sys_service.py
Other components
- Analyst
- Remote Forwarder
- Network Traffic
- Browser User
Connections 23
- Browser User → File Upload: uploads artifacts
- File Upload → Upload Watchers: saves uploads
- Remote Forwarder → Zeek Analysis: forwards logs
- Network Traffic → Zeek Analysis: captures traffic
- Network Traffic → Suricata Analysis: captures traffic
- PCAP Watcher → PCAP Processor: queues captures
- PCAP Processor → Zeek Analysis: dispatches analysis
- PCAP Processor → Suricata Analysis: dispatches analysis
- Zeek Analysis → Normalization Enrichment: emits logs
- Suricata Analysis → Normalization Enrichment: emits alerts
- Normalization Enrichment → Asset Inventory: looks up assets
- Normalization Enrichment → OpenSearch: writes events
- PCAP Processor → OpenSearch: stores session data
- PCAP Processor → File Scanning: scans extracted files
- OpenSearch → Malcolm API: serves searches
- Malcolm API → Dashboards: provides data
- OpenSearch → Dashboards: serves visualizations
- OpenSearch → Arkime Sessions: serves session data
- WISE Fields → Arkime Sessions: registers fields
- Analyst → Dashboards: explores traffic
- Analyst → Arkime Sessions: searches sessions
- Browser User → Kiosk Controls: requests controls
- Kiosk Controls → Service Operations: dispatches command