How akshayetukuri/ai-ids-capstone works
AI-IDS is a CICIDS2017 intrusion-detection research pipeline comparing a leakage-conscious Random Forest baseline with a chunk-sequence transformer, then analyzing transformer attention. Its documented workflow runs from dataset preparation through training and interpretation; predict.py also provides binary inference using the saved Random Forest. Some stages are documented but their source was not sampled, so their exact internal behavior and wiring are not inferred.
Main entry points: predict.py Predict flow labels · baseline_dedup.py Deduplicate and train RF · merge_data.py Merge daily flows · clean_data.py Clean flow data
Components 17
Flow data preparation
- Merge daily flows
merge_data.py - Clean flow data
clean_data.py - Drop infinities
fix_infinity.py - Deduplicated flows
- Stratified sample
make_sample.py - 150K flow sample
Model training
- Deduplicate and train RF
baseline_dedup.py - Train sequence transformer
train_transformer.py - Transformer weights
- Random Forest model
rf_binary_baseline_dedup.joblib
Interpretation
- Analyze attention
attention_analysis.py - Attention by label
Flow inference
- Predict flow labels
predict.py - Predictions CSV
Other components
- CICIDS2017 CSVs
- Analyst
- New flow records
Connections 16
- CICIDS2017 CSVs → Merge daily flows: loads CSVs
- Merge daily flows → Clean flow data: combined data
- Clean flow data → Drop infinities: cleaned data
- Drop infinities → Deduplicate and train RF: clean flows
- Deduplicate and train RF → Deduplicated flows: writes dataset
- Deduplicate and train RF → Random Forest model: saves model
- Deduplicated flows → Stratified sample: samples flows
- Stratified sample → 150K flow sample: writes sample
- 150K flow sample → Train sequence transformer: loads sample
- Train sequence transformer → Transformer weights: saves weights
- 150K flow sample → Analyze attention: loads sample
- Analyze attention → Attention by label: writes analysis
- Analyst → Predict flow labels: requests inference
- New flow records → Predict flow labels: supplies records
- Random Forest model → Predict flow labels: loads model
- Predict flow labels → Predictions CSV: writes results