# kihy/anidsc architecture

> Architecture diagram of the GitHub repository kihy/anidsc, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kihy/anidsc
- Repository: https://github.com/kihy/anidsc

## Overview

##### Project type and purpose
- ANIDSC is an open-source Python toolkit for building and evaluating network intrusion detection pipelines, with support for adversarial attacks, streaming or batch data ingestion from pcap files, feature extraction, normalization, model training/inference (classical and deep-learning), drift detection, and result summarization/visualization.
- It is not a web app but a library plus reference scripts (adv_gen.py, live_detection.py, summarize_results.py) that runs in Docker or local Python environments.

##### File structure
- Top-level directories and scripts:
- docker_root (build files)
- src/ANIDSC (core library)
- features (BDD tests for chain behavior)
- test_data (example pcap files)
- deploy_commands, gen_uml.sh, git-delete (utility scripts)
- src/ANIDSC subdivides by function:
- data_source: offline_sources, live_sources
- component: pipeline primitives (pipeline_source, pipeline_component, processors, feature_buffer, feature_extractor, normalizer)
- pipeline: orchestration (Pipeline, multi_splitter)
- model: classical (boxplot, kitsune, gnnids), torch_model variants, node_encoder
- adversarial_attack: attack algorithms
- cdd_framework: drift detectors (arcus, drift_sense)
- evaluator: metrics, summarizer
- save_mixin: serialization backends
- utils, graph_rep, scaler, templates

##### Main components and relationships
a. Data Ingestion Layer
- offline_sources.live_sources → raw packet streams (pcap)
- offline_sources.offline_sources → historical batches
b. Processing & Feature Layer
- PipelineSource provides raw data to Pipeline
- PipelineComponent chain:
- FeatureBuffer (accumulates windows)
- FeatureExtractor modules (after_image, frequency, graph, synthetic, etc.)
- Normalizer / Scaler (t_digest)
c. Modeling Layer
- Models consume processed feature vectors:
- Classical detectors (boxplot, kitsune, gnnids)
- Deep-learning (PyTorch autoencoder, GOAD, ICL, SLAD, kitsune_torch)
- NodeEncoder (graph neural net input)
d. Drift Detection Layer
- CDD_Framework modules monitor concept drift on feature streams
e. Adversarial Attack Module
- Hooks into pipeline input or model gradient flow (liuer_mihou)
f. Evaluation & Reporting Layer
- Evaluator computes metrics (OD‐metrics), summarizer plots anomaly scores
- Summarize_results.py and graph_vis utilities for visual output
g. Persistence & Extensibility
- SaveMixin backends (json, yaml, pickle, torch, null) for model or pipeline checkpoints
- Templates and helper utilities for code generation

##### Architectural patterns and design principles
- Pipeline pattern: modular, pluggable components that adhere to a common interface
- Strategy pattern for feature extractors, models, save mixins
- Layered architecture: ingestion → processing → modeling → evaluation
- Separation of concerns: data sources, compute components, persistence, evaluation

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Data Ingestion Layer
    subgraph "Data Ingestion Layer"
        direction TB
        Offline["Offline Sources"]:::datasrc
        Live["Live Sources"]:::datasrc
        PipelineSrc["Pipeline Source"]:::pipeline
    end

    Offline -->|"pcap stream"| PipelineSrc
    Live -->|"live packets"| PipelineSrc
    click Offline "https://github.com/kihy/anidsc/blob/main/src/ANIDSC/data_source/offline_sources.py"
    click Live "https://github.com/kihy/anidsc/blob/main/src/ANIDSC/data_source/live_sources.py"
    click PipelineSrc "https://github.com/kihy/anidsc/blob/main/src/ANIDSC/component/pipeline_source.py"

    %% Processing & Feature Layer
    subgraph "Processing & Feature Layer"
        direction TB
        PipeOrch["Pipeline Orchestration"]:::pipeline
        MultiSplit["Multi-Splitter"]:::pipeline
        FeatureBuf["Feature Buffer"]:::pipeline
        TabularBuf["Tabular Feature Buffer"]:::pipeline
        ProcBase["Processor Base"]:::pipeline
        Norm["Normalizer"]:::pipeline
        Scaler["t-digest Scaler"]:::pipeline
        FEBase["Feature Extractor Base"]:::pipeline
        AI["After-Image"]:::pipeline
        Freq["Frequency"]:::pipeline
        PsiAt["PS-IAT"]:::pipeline
        PyFlow["PyFlowMeter"]:::pipeline
        Synth["Synthetic Graph Features"]:::pipeline
    end

    PipelineSrc -->|add to| PipeOrch
    PipeOrch -->|"split batch"| MultiSplit
    MultiSplit -->|"window frames"| FeatureBuf
    FeatureBuf --> TabularBuf
    FeatureBuf -->|"buffer windows"| TabularBuf
    FeatureBuf -->|"process"| ProcBase
    ProcBase --> Norm
    Norm --> Scaler
    TabularBuf --> FEBase
    FEBase --> AI
    FEBase --> Freq
    FEBase --> PsiAt
    FEBase --> PyFlow
    FEBase --> Synth

    click PipeOrch "https://github.com/kihy/anidsc/blob/main/src/ANIDSC/pipeline/pipeline.py"
    click MultiSplit "https://github.com/kihy/anidsc/blob/main/src/ANIDSC/pipeline/multi_splitter.py"
    click FeatureBuf "https://github.com/kihy/anidsc/blob/main/src/ANIDSC/component/feature_buffer.py"
    click TabularBuf "https://github.com/kihy/anidsc/blob/main/src/ANIDSC/feature_buffer/tabular.py"
    click FEBase "https://github.com/kihy/anidsc/blob/main/src/ANIDSC/component/feature_extractor.py"
    click AI "https://github.com/kihy/anidsc/blob/main/src/ANIDSC/feature_extractor/after_image.py"
    click Freq "https://github.com/kihy/anidsc/blob/main/src/ANIDSC/feature_extractor/frequency.py"
    click PsiAt "https://github.com/kihy/anidsc/blob/main/src/ANIDSC/feature_extractor/psiat.py"
    click PyFlow "https://github.com/kihy/anidsc/blob/main/src/ANIDSC/feature_extractor/pyflowmeter.py"
    click Synth "https://github.com/kihy/anidsc/blob/main/src/ANIDSC/feature_extractor/synthetic_graph_features.py"
    click ProcBase "https://github.com/kihy/anidsc/blob/main/src/ANIDSC/component/processor.py"
    click Norm "https://github.com/kihy/anidsc/blob/main/src/ANIDSC/component/normalizer.py"
    click Scaler "https://github.com/kihy/anidsc/blob/main/src/ANIDSC/scaler/t_digest.py"

    %% Modeling & Adversarial Attack Layer
    subgraph "Modeling & Attack Layer"
        direction TB
        subgraph "Classical Models"
            Box["Boxplot"]:::model
            Kits["Kitsune"]:::model
            GNN["GNNIDS"]:::model
        end
        NodeEnc["Node Encoder"]:::model
        subgraph "Deep-Learning Models"
            TorchBase["Base Torch Model"]:::model
            BaseNet["Base Networks"]:::model
            AE["Autoencoder"]:::model
            GOAD["GOAD"]:::model
            ICL["ICL"]:::model
            SLAD["SLAD"]:::model
            KT["Kitsune-Torch"]:::model
        end
        Attack["Adversarial Attack"]:::attack
    end

    Scaler -->|"feature vector"| Box
    Scaler --> Kits
    Scaler --> GNN
    Scaler --> NodeEnc
    Scaler --> TorchBase
    TorchBase --> BaseNet
    BaseNet --> AE
    BaseNet --> GOAD
    BaseNet --> ICL
    BaseNet --> SLAD
    BaseNet --> KT

    Scaler --> Attack
    Attack -->|"perturbed vector"| Box
    Attack --> Kits
    Attack --> AE

    click Box "https://github.com/kihy/anidsc/blob/main/src/ANIDSC/model/boxplot.py"
    click Kits "https://github.com/kihy/anidsc/blob/main/src/ANIDSC/model/kitsune.py"
    click GNN "https://github.com/kihy/anidsc/blob/main/src/ANIDSC/model/gnnids.py"
    click NodeEnc "https://github.com/kihy/anidsc/blob/main/src/ANIDSC/model/node_encoder/encoder.py"
    click TorchBase "https://github.com/kihy/anidsc/blob/main/src/ANIDSC/model/torch_model/base_torch_model.py"
    click BaseNet "https://github.com/kihy/anidsc/blob/main/src/ANIDSC/model/torch_model/base_networks.py"
    click AE "https://github.com/kihy/anidsc/blob/main/src/ANIDSC/model/torch_model/autoencoder.py"
    click GOAD "https://github.com/kihy/anidsc/blob/main/src/ANIDSC/model/torch_model/goad.py"
    click ICL "https://github.com/kihy/anidsc/blob/main/src/ANIDSC/model/torch_model/icl.py"
    click SLAD "https://github.com/kihy/anidsc/blob/main/src/ANIDSC/model/torch_model/slad.py"
    click KT "https://github.com/kihy/anidsc/blob/main/src/ANIDSC/model/torch_model/kitsune_torch.py"
    click Attack "https://github.com/kihy/anidsc/blob/main/src/ANIDSC/adversarial_attack/liuer_mihou.py"

    %% Drift Detection Layer
    subgraph "Drift Detection Layer"
        direction TB
        Arcus["ARCUS Detector"]:::pipeline
        DriftSense["DriftSense Detector"]:::pipeline
    end

    Scaler --> Arcus
    Scaler --> DriftSense

    click Arcus "https://github.com/kihy/anidsc/blob/main/src/ANIDSC/cdd_framework/arcus.py"
    click DriftSense "https://github.com/kihy/anidsc/blob/main/src/ANIDSC/cdd_framework/drift_sense.py"

    %% Evaluation & Reporting Layer
    subgraph "Evaluation & Reporting Layer"
        direction TB
        EvalEng["Evaluator Engine"]:::eval
        ODMet["OD Metrics"]:::eval
        Sum["Summarizer/Reporter"]:::eval
        Vis["Graph Visualizer"]:::eval
    end

    Box --> EvalEng
    Kits --> EvalEng
    AE --> EvalEng
    EvalEng --> ODMet
    EvalEng --> Sum
    Sum --> Vis

    click EvalEng "https://github.com/kihy/anidsc/blob/main/src/ANIDSC/evaluator/evaluator.py"
    click ODMet "https://github.com/kihy/anidsc/blob/main/src/ANIDSC/evaluator/od_metrics.py"
    click Sum "https://github.com/kihy/anidsc/blob/main/src/ANIDSC/evaluator/summarizer.py"
    click Vis "https://github.com/kihy/anidsc/blob/main/src/ANIDSC/utils/graph_vis.py"

    %% Persistence Layer
    subgraph "Persistence & Extensibility"
        direction TB
        JSON["JSON Backend"]:::persist
        YAML["YAML Backend"]:::persist
        Pickle["Pickle Backend"]:::persist
        TorchSave["Torch Backend"]:::persist
        Null["Null Backend"]:::persist
        Templates["Template Helpers"]:::persist
    end

    PipeOrch -- save --> JSON
    PipeOrch -- save --> YAML
    PipeOrch -- save --> Pickle
    PipeOrch -- save --> TorchSave
    PipeOrch -- save --> Null
    Box -- save --> JSON
    NodeEnc -- save --> JSON
    TorchBase -- save --> TorchSave
    JSON -.-> Templates

    click JSON "https://github.com/kihy/anidsc/blob/main/src/ANIDSC/save_mixin/json.py"
    click YAML "https://github.com/kihy/anidsc/blob/main/src/ANIDSC/save_mixin/yaml.py"
    click Pickle "https://github.com/kihy/anidsc/blob/main/src/ANIDSC/save_mixin/pickle.py"
    click TorchSave "https://github.com/kihy/anidsc/blob/main/src/ANIDSC/save_mixin/torch.py"
    click Null "https://github.com/kihy/anidsc/blob/main/src/ANIDSC/save_mixin/null.py"
    click Templates "https://github.com/kihy/anidsc/blob/main/src/ANIDSC/templates.py"

    %% Styles
    classDef datasrc fill:#D2B4DE,stroke:#7D3C98,color:#fff
    classDef pipeline fill:#AED6F1,stroke:#2874A6,color:#000
    classDef model fill:#A9DFBF,stroke:#196F3D,color:#000
    classDef attack fill:#F5B7B1,stroke:#C0392B,color:#000
    classDef eval fill:#F9E79F,stroke:#B7950B,color:#000
    classDef persist stroke-dasharray: 5 5,fill:#E8DAEF,stroke:#6C3483,color:#000
```
