How abdulrehman25/msc-finals works
This repository detects anomalies in web access logs with two reconstruction-based branches: per-request content scoring and IP/user-agent session-window scoring. The principal runtime flow is a submitted log line, parsing and feature extraction, branch scores, weighted fusion, and a stored event that the dashboard can display. Separate training commands produce branch models and thresholds; a calibration command sets fusion weights. The README documents these capabilities; evaluation is also exposed by the API. Some API implementation is omitted, so only documented or excerpt-supported wiring is shown.
Main entry points: main.py Scoring API · train_session_lstm_vae.py Session trainer · train_content_ae.py Content trainer · features.py Request features
Components 17
Log input
- Access log
- Log parser
src/featurization/parse.py - Request features
src/featurization/features.py - Log vector loader
src/featurization/loaders.py
Branch training
- Content trainer
train_content_ae.py - Content AE
src/models/content_autoencoder.py - Session trainer
train_session_lstm_vae.py - Session windows
src/pipeline/sessionize.py - Session LSTM-VAE
src/models/session_lstm_vae.py
Detection runtime
- Fusion calibration
src/pipeline/fuse_calibrate.py - Scoring API
src/api/main.py
Review and evaluation
- Event database
src/storage/db.py - Dashboard
src/api/main.py - File evaluation
scripts/evaluate_file.py - Operating metrics
src/pipeline/metrics_extra.py
Other components
- Log operator
- API client
Connections 22
- Log operator → Access log: provides
- Access log → Content trainer: trains from
- Access log → Session trainer: trains from
- Content trainer → Log vector loader: loads vectors
- Session trainer → Log vector loader: loads vectors
- Content trainer → Content AE: trains
- Session trainer → Session windows: builds windows
- Session trainer → Session LSTM-VAE: trains
- Content trainer → Fusion calibration: provides scores
- Session trainer → Fusion calibration: provides scores
- API client → Scoring API: submits line
- Scoring API → Log parser: parses line
- Scoring API → Request features: extracts vector
- Request features → Log parser: parses line
- Scoring API → Content AE: scores branch
- Scoring API → Session LSTM-VAE: scores branch
- Scoring API → Event database: writes event
- Scoring API → File evaluation: runs evaluation
- File evaluation → Operating metrics: computes metrics
- API client → Dashboard: views results
- Dashboard → Event database: reads events
- Dashboard → Scoring API: requests dashboard data