# tudelft-cda-lab/encode architecture

> Architecture diagram of the GitHub repository tudelft-cda-lab/encode, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tudelft-cda-lab/encode
- Repository: https://github.com/tudelft-cda-lab/encode

## Overview

ENCODE learns feature-value cluster assignments from NetFlow data. The main workflow accepts a CSV or DataFrame, orders records by the chosen connection/host/time level, computes contextual frequencies for neighboring values, builds a context matrix, and selects the best of repeated K-Means runs by silhouette score. It writes the matrix and resulting encoding to CSV and returns the encoding mapping. The graph separates data preparation, context construction, and clustering/output. Edges reflect the sampled source and README; unsampled behavior is not inferred.

Main entry points: [`encode/encode.py`](https://github.com/tudelft-cda-lab/encode/blob/main/encode/encode.py) (Encode API), [`encode/matrix_operations.py`](https://github.com/tudelft-cda-lab/encode/blob/main/encode/matrix_operations.py) (Context matrix), [`encode/frequency_utils.py`](https://github.com/tudelft-cda-lab/encode/blob/main/encode/frequency_utils.py) (Neighbor counts), [`encode/utils.py`](https://github.com/tudelft-cda-lab/encode/blob/main/encode/utils.py) (CSV input)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_workflow["Data preparation"]
  node_api["Encode API<br/>[encode.py]"]
  node_csv_input["CSV input<br/>[utils.py]"]
  node_preparation["Sort and prepare<br/>[encode.py]"]
  node_utilities["Data utilities<br/>[utils.py]"]
end

subgraph group_context["Context modeling"]
  node_counts["Neighbor counts<br/>[frequency_utils.py]"]
  node_frequency_bins["Frequency bins<br/>[frequency_utils.py]"]
  node_matrix["Context matrix"]
  node_matrix_file["Matrix CSV"]
end

subgraph group_encoding["Encoding"]
  node_cluster["K-Means selection<br/>[encode.py]"]
  node_mapping["Feature mapping<br/>[encode.py]"]
  node_encoding_file["Encoding CSV"]
  node_result["Encoding result"]
end

node_user(("ENCODE user"))
node_netflow["NetFlow data"]

node_user -->|"invokes"| node_api
node_netflow -->|"supplies data"| node_api
node_api -->|"reads CSV"| node_csv_input
node_api -->|"prepares data"| node_preparation
node_preparation -->|"sorts and indexes"| node_utilities
node_preparation -->|"computes neighbors"| node_counts
node_counts -->|"bins counts"| node_frequency_bins
node_api -->|"builds matrix"| node_matrix
node_matrix -->|"writes matrix"| node_matrix_file
node_api -->|"loads precomputed"| node_matrix
node_api -->|"clusters rows"| node_cluster
node_cluster -->|"assigns clusters"| node_mapping
node_api -->|"maps values"| node_mapping
node_api -->|"writes mapping"| node_encoding_file
node_api -->|"returns mapping"| node_result

click node_api "https://github.com/tudelft-cda-lab/encode/blob/main/encode/encode.py"
click node_csv_input "https://github.com/tudelft-cda-lab/encode/blob/main/encode/utils.py"
click node_preparation "https://github.com/tudelft-cda-lab/encode/blob/main/encode/encode.py"
click node_utilities "https://github.com/tudelft-cda-lab/encode/blob/main/encode/utils.py"
click node_counts "https://github.com/tudelft-cda-lab/encode/blob/main/encode/frequency_utils.py"
click node_frequency_bins "https://github.com/tudelft-cda-lab/encode/blob/main/encode/frequency_utils.py"
click node_matrix "https://github.com/tudelft-cda-lab/encode/blob/main/encode/matrix_operations.py"
click node_cluster "https://github.com/tudelft-cda-lab/encode/blob/main/encode/encode.py"
click node_mapping "https://github.com/tudelft-cda-lab/encode/blob/main/encode/encode.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_api,node_csv_input,node_preparation,node_utilities,node_user toneBlue
class node_counts,node_frequency_bins,node_matrix,node_matrix_file toneAmber
class node_cluster,node_mapping,node_encoding_file,node_result toneMint
class node_netflow toneIndigo
```

## Components

### Data preparation

- [Encode API](https://github.com/tudelft-cda-lab/encode/blob/main/encode/encode.py): `encode/encode.py`
- [CSV input](https://github.com/tudelft-cda-lab/encode/blob/main/encode/utils.py): `encode/utils.py`
- [Sort and prepare](https://github.com/tudelft-cda-lab/encode/blob/main/encode/encode.py): `encode/encode.py`
- [Data utilities](https://github.com/tudelft-cda-lab/encode/blob/main/encode/utils.py): `encode/utils.py`

### Context modeling

- [Neighbor counts](https://github.com/tudelft-cda-lab/encode/blob/main/encode/frequency_utils.py): `encode/frequency_utils.py`
- [Frequency bins](https://github.com/tudelft-cda-lab/encode/blob/main/encode/frequency_utils.py): `encode/frequency_utils.py`
- [Context matrix](https://github.com/tudelft-cda-lab/encode/blob/main/encode/matrix_operations.py): `encode/matrix_operations.py`
- **Matrix CSV**

### Encoding

- [K-Means selection](https://github.com/tudelft-cda-lab/encode/blob/main/encode/encode.py): `encode/encode.py`
- [Feature mapping](https://github.com/tudelft-cda-lab/encode/blob/main/encode/encode.py): `encode/encode.py`
- **Encoding CSV**
- **Encoding result**

### Other components

- **ENCODE user**
- **NetFlow data**

## Connections

- ENCODE user → Encode API: invokes (evidence: `README.md`)
- NetFlow data → Encode API: supplies data (evidence: `README.md`)
- Encode API → CSV input: reads CSV (evidence: `encode/encode.py`)
- Encode API → Sort and prepare: prepares data (evidence: `encode/encode.py`)
- Sort and prepare → Data utilities: sorts and indexes (evidence: `encode/encode.py`)
- Sort and prepare → Neighbor counts: computes neighbors (evidence: `encode/encode.py`)
- Neighbor counts → Frequency bins: bins counts (evidence: `encode/encode.py`)
- Encode API → Context matrix: builds matrix (evidence: `encode/encode.py`)
- Context matrix → Matrix CSV: writes matrix
- Encode API → Context matrix: loads precomputed (evidence: `encode/encode.py`)
- Encode API → K-Means selection: clusters rows (evidence: `encode/encode.py`)
- K-Means selection → Feature mapping: assigns clusters (evidence: `encode/encode.py`)
- Encode API → Feature mapping: maps values (evidence: `encode/encode.py`)
- Encode API → Encoding CSV: writes mapping (evidence: `encode/encode.py`)
- Encode API → Encoding result: returns mapping (evidence: `encode/encode.py`)
