How aicoe/prometheus-data-science works
This repository is a data-science toolkit for analyzing Prometheus metrics, not a continuously running service. Its main workflow reformats metric JSON packets into per-series Pandas data, then applies forecasting and anomaly-detection techniques. Separate metadata tools visualize label behavior and reduce metadata dimensions; histogram and summary analysis is also a documented capability. The formatter and several analysis scripts are excerpted; links between preprocessing and forecasting follow the README, while unsampled implementations are represented only where their paths are known.
Main entry points: format_to_pandas.py Packet formatter · graph_metadata.py Metadata timelines · graph_specific_ts.py Specific series plots · t_sne_for_metadata.py Metadata t-SNE
Components 16
Preprocessing
- Packet formatter
format_to_pandas.py - Series pickle
Metadata Analysis
- Metadata timelines
metadata_analysis/graph_metadata.py - Specific series plots
metadata_analysis/graph_specific_ts.py - Metadata t-SNE
metadata_analysis/t_sne_for_metadata.py - Label histograms
metadata_analysis/plot_metadata_labels.py - Single-series extraction
metadata_analysis/get_single_ts_all.py
Forecasting
- ARIMA training
arima_train.py - Exponential smoothing
exp_smoothing_train.py - Holt-Winters training
holt_winters_train.py - Fourier training
fourier_train.py - Prophet training
prophet_train.py - Fourier-Prophet comparison
compare_fourier_prophet.py
Anomaly Analysis
- Series graphing
graphing_ts.py - Anomaly decision rules
anomaly_detector.py
Other components
- Metric packets
Connections 14
- Metric packets → Packet formatter: provides packets
README.md - Packet formatter → Series pickle: writes series
README.md - Series pickle → ARIMA training: supplies series
README.md - Series pickle → Exponential smoothing: supplies series
README.md - Series pickle → Holt-Winters training: supplies series
README.md - Series pickle → Fourier training: supplies series
README.md - Series pickle → Prophet training: supplies series
README.md - Series pickle → Fourier-Prophet comparison: supplies series
README.md - Metric packets → Metadata timelines: reads packets
metadata_analysis/graph_metadata.py - Metric packets → Specific series plots: reads packets
metadata_analysis/graph_specific_ts.py - Metric packets → Metadata t-SNE: reads packets
metadata_analysis/t_sne_for_metadata.py - Metric packets → Label histograms: reads packets
metadata_analysis/plot_metadata_labels.py - Metric packets → Single-series extraction: reads packets
metadata_analysis/get_single_ts_all.py - Series pickle → Series graphing: loads series
graphing_ts.py