How ait-aecid/rootkit-detection-ebpf-time-trace works
Project Type and Purpose
• This is an open-source security tool comprising two phases:
– Data collection via eBPF probes and a custom rootkit (CARAXES)
– Semi-supervised anomaly detection on the collected timing data
• It is not a monolithic web app but a command-line driven, experimental pipeline integrating user-space scripts, kernel modules, and machine-learning components.
File Structure
• Single‐level layout: no nested “frontend”/“backend” folders; instead, Python scripts, C sources, and shell wrappers sit at the top.
• Key groups:
– Probing & kernel integration: probing.py, linux.py, hooks.h, kernel.c
– Rootkit source override: hooks.h (to redirect filldir)
– Data handling & stats: data_classes.py, process_data.py, get_statistics.py, count_events.py
– Anomaly detection: evaluate.py
– Orchestration scripts: demo.sh, experiment.sh, repeat_seq.sh, repeat_sim.sh, run.py
– Helper utilities: ftrace_test.sh, repeat scripts, ls-basic.c
• Configuration/build artifacts: requirements.txt, Makefiles invoked in CARAXES.
Extract README Insights
• Phase 1 (Data Collection):
– Clone CARAXES rootkit → replace hooks.h → compile → load kernel module
– Inject eBPF probes via BCC (probing.py) for selected getdents callbacks
– Drive system calls (ls variants) in loops, collect time measurements → store JSON.gz under events/
• Phase 2 (Anomaly Detection):
– Load JSON files → split into train/test → compute statistical tests → output metrics & confusion matrix
– Controlled via evaluate.py with CLI flags (-d, -t, -m, -g)
• External dependencies: Linux kernel headers, Python3-BCC, stress-ng (optional), scikit-learn (or similar) via requirements.txt.
1. External Rootkit (CARAXES Kernel Module)
2. Kernel Space
• Hooked filldir via CARAXES
• eBPF Probe Programs (loaded by BCC)
3. User-Space Controller (probing.py + linux.py)
4. Experiment Orchestrator (demo.sh, experiment.sh, run.py)
5. Data Storage (events/*.json.gz)
6. Data Processing & Statistics (process_data.py, get_statistics.py, count_events.py)
7. Anomaly Detection Engine (evaluate.py + ML libraries)
8. CLI Interfaces & Shell Scripts
– Orchestrator → invokes probing.py
– probing.py → compiles & attaches eBPF probes → loads rootkit module → triggers ls commands
– Kernel Space (hooks + probes) → emits timing events → writes to user-space via perf buffers
– probing.py → serializes events → writes JSON.gz to events/
– Data Processing scripts → read JSON.gz → produce aggregated statistics
– evaluate.py → consumes stats/data → trains/tests model → outputs metrics/plots
c. Architectural Patterns & Principles:
– Modular Pipeline: Clear separation between collection, processing, and detection
– Kernel/User-Space Boundary: eBPF in kernel, orchestration and ML in user space
– Extensibility Points: probe_points list in probing.py for adding/removing functions
– Reuse of external rootkit (CARAXES) as pluggable component
d. Technologies, Frameworks, Libraries:
– Kernel side: C, Linux eBPF (BCC), CARAXES rootkit
– User side: Python 3.10, BCC Python API, JSON, gzip, scikit-learn (or SciPy/stats), shell scripting (bash)