How imperial-qore/tranad works
Main Components
• Data Repository: The “data” directory contains multiple datasets (e.g., SMAP_MSL, SMD, SWaT, etc.) organized by source and experiments. These datasets are the raw inputs for anomaly detection.
• Preprocessing Module: The preprocess.py script is responsible for taking these datasets and preparing them (format conversion, normalization, splitting, etc.) for model training and evaluation.
• Model Training/Execution Engine: The main.py is the primary entry point that accepts command-line parameters like “--model” and “--dataset”. It drives training and testing for different anomaly detection algorithms (TranAD, GDN, MAD_GAN, etc.).
• Model Components & Utilities: The “src” folder contains core modules:
• models.py for model implementations,
• diagnosis.py for evaluating and diagnosing results,
• dlutils.py, parser.py, plotting.py, and others that support the training, logging, and visualization process.
There’s also a configuration component (params.json in src and constants.py) that sets experiment parameters.
• Results & Visualization: The “results” directory stores output metrics while the “plots” directory holds generated visualizations (e.g., training graphs and anomaly detection outputs).
Relationships & Data Flow
• Input Flow: Raw datasets from the “data” folder are fed into the preprocessing module.
• Preprocessing Flow: Preprocess.py processes the datasets, ensuring that the formats align with what the training module expects.
• Training Pipeline: main.py initiates the training routine by accepting model and dataset inputs. It loads configurations from src/params.json and constants.
• Model Execution: Within main.py, the chosen model (e.g., TranAD or one of its ablated versions) is instantiated (via src/models.py) and then trained on the preprocessed data. Auxiliary modules in src (like diagnosis.py and utils.py) help with logging, evaluating, and diagnosing the anomaly detection performance.
• Output Flow: Once training and testing are complete, result metrics are written into the “results” folder and visualized via graphs stored in the “plots” folder. These outputs can also be used to conduct further ablation studies and comparisons.
Architectural Patterns & Design Principles
• Modular design: Separation between data preprocessing, model training, and evaluation.
• Configurability: Centralized parameters (params.json and constants.py) allow for reproducibility and varied experimental setups.
• Extensibility: The system supports multiple anomaly detection models (as well as ablation variants) making it easy to add future techniques.
• Pipeline orchestration: A clear pipeline from raw input to processing to training/testing to output generation.
Relevant Technologies & Frameworks
• Python (>=3.7) as the main language.
• PyTorch framework for deep learning (as indicated by installation instructions).
• Standard Python libraries for file handling, plotting (likely matplotlib or similar), and data manipulation.