How veeranidhish-7/anamoly-detection-in-syslog-streams works
This repository is a research implementation for log-anomaly detection, comparing a paper-reproduction PCA/RPCA baseline with contributed models and thresholding methods. The main workflow evaluates precomputed labeled log features: split normal and anomalous samples, fit a detector on normal data, score held-out samples, apply a decision rule, and report cross-validation metrics. BGL preprocessing and sequence-based experiments are additional branches. The README is empty, and several modules are unsampled; links are limited to relationships shown in excerpts or the source index.
Main entry points: run_improvements.py Improvement runner · run_baseline.py Baseline runner · run_improvements_leak_fixed.py Leak-free runner · run_lstm_autoencoder.py LSTM experiment
Components 19
Data preparation
- BGL download
src/our_contributions/features/bgl_pipeline.py - Drain parsing
src/our_contributions/features/bgl_pipeline.py - BGL windows
src/our_contributions/features/bgl_pipeline.py - Sequence extraction
src/our_contributions/features/extract_sequences.py - Embedding extraction
src/our_contributions/features/extract_embeddings.py
Baseline evaluation
- Baseline runner
src/baseline_replication/run_baseline.py - PCA / RPCA
src/baseline_replication/models/pca_rpca.py - Saved baseline models
saved_models
Improved detectors
- Improvement runner
src/our_contributions/run_improvements.py - Leak-free runner
src/our_contributions/run_improvements_leak_fixed.py - ML detectors
src/our_contributions/models/deep_learning.py - Adaptive thresholds
src/our_contributions/models/thresholding.py - Saved contribution models
saved_models_our_contributions
Sequence experiments
- LSTM experiment
src/our_contributions/run_lstm_autoencoder.py - LSTM autoencoder
src/our_contributions/run_lstm_autoencoder.py
Research analysis
- PCA/RPCA evaluation
src/archive/attempt_1_rank_collapse/evaluate_pca_rpca.py - Benchmarking
src/our_contributions/benchmark.py
Other components
- Researcher
- Zenodo dataset
Connections 17
- Researcher → Baseline runner: runs evaluation
src/baseline_replication/run_baseline.py - Researcher → Improvement runner: runs experiments
src/our_contributions/run_improvements.py - Researcher → Leak-free runner: runs experiment
src/our_contributions/run_improvements_leak_fixed.py - Researcher → LSTM experiment: runs experiment
src/our_contributions/run_lstm_autoencoder.py - BGL download → Zenodo dataset: downloads archive
src/our_contributions/features/bgl_pipeline.py - BGL download → Drain parsing: extracts for parsing
src/our_contributions/features/bgl_pipeline.py - Drain parsing → BGL windows: structures logs
src/our_contributions/features/bgl_pipeline.py - Baseline runner → PCA / RPCA: fits and scores
src/baseline_replication/run_baseline.py - Baseline runner → Saved baseline models: loads or saves
src/baseline_replication/run_baseline.py - Improvement runner → PCA / RPCA: uses PCA
src/our_contributions/run_improvements.py - Improvement runner → ML detectors: fits and scores
src/our_contributions/run_improvements.py - Improvement runner → Adaptive thresholds: computes cutoff
src/our_contributions/run_improvements.py - Improvement runner → Saved contribution models: loads or saves
src/our_contributions/run_improvements.py - Leak-free runner → ML detectors: fits and scores
src/our_contributions/run_improvements_leak_fixed.py - Leak-free runner → Adaptive thresholds: computes cutoff
src/our_contributions/run_improvements_leak_fixed.py - LSTM experiment → LSTM autoencoder: trains and scores
src/our_contributions/run_lstm_autoencoder.py - LSTM experiment → Adaptive thresholds: computes cutoff
src/our_contributions/run_lstm_autoencoder.py