# r-niemans/cds_ternarysearchtree architecture

> Architecture diagram of the GitHub repository r-niemans/cds_ternarysearchtree, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/r-niemans/cds_ternarysearchtree
- Repository: https://github.com/r-niemans/cds_ternarysearchtree

## Overview

##### Project type and purpose
- This is a library and benchmarking framework for various Ternary Search Tree (TST) implementations in Python.
- It explores multiple TST variants (object-oriented, iterative, sparse, recursive), compares them to a binary tree, and measures performance on small datasets locally and large datasets on HPC infrastructure.

##### File structure
- Top-level
- trees/ – all tree implementations: btree.py, ternary_tree.py, ternary_tree_B.py, ternary_tree_minimalistic.py, ternary_tree_recursive.py
- data/ – raw datasets (small and sample words files)
- test_tst.py – unit tests for correctness
- tst_implementations.ipynb – Jupyter notebook for local benchmarking
- HPC/ – scripts, job definitions, and outputs for large-scale tests
- HPC/ substructure
- jobscript.slurm, jobscript2.slurm – SLURM batch definitions
- *.py scripts – wrappers to load data, perform insert/search tests on HPC
- Outputs/slurm-out.zip – collected logs and metrics
- README files describe high-level flow and how to run tests.

##### Extract insights from the README
- Four distinct TST variants plus a modified binary tree
- Workflow: implement → unit test → local performance tests → large-scale HPC performance tests → documentation
- Data flow: ingest dataset → build tree → run searches → collect metrics

##### Main components
a. Code modules
- Tree library layer (trees/)
- Unit test harness (test_tst.py)
- Local performance bench (tst_implementations.ipynb)
- HPC performance bench scripts (HPC/*.py)
b. Data stores
- Local datasets (data/*.txt)
- External datasets (wikipedia, frequency words)
- HPC output storage (HPC/Outputs)
c. Execution environments
- Local workstation (running Python + Jupyter)
- HPC cluster (SLURM scheduler)
d. External services
- HPC scheduler (SLURM)
- Storage for big datasets (Hugging Face URLs)

##### Relationships and data flows
- Local workflow:
data/ → Python TST modules → unit tests → Jupyter notebook → metrics
- HPC workflow:
data/ (transferred) → HPC/*.py → SLURM jobscript → HPC compute nodes → Outputs
- The tree library is the core component invoked by both local bench and HPC scripts.

##### Architectural patterns and design principles
- Modular codebase: separate modules for each TST variant, clear API for insert/search.
- Test-driven approach: unit tests enforce correctness before performance tests.
- Benchmark layering: local quick tests vs. large-scale HPC runs share the same core library.
- Separation of concerns: data ingestion, algorithm implementation, test harness, result collection.

##### Technologies and frameworks
- Python 3.x
- Jupyter Notebook for local benchmarking
- SLURM for HPC batch scheduling
- Standard text files for datasets

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Stores
    subgraph "Data Stores"
        DSLocal["Local Datasets<br/>(small)"]:::data
        ExternalDS["External Datasets<br/>(large)"]:::data
    end

    %% Core Library
    TL["Tree Library<br/>(TST & BTree Variants)"]:::core

    %% Local Environment
    subgraph "Local Workstation"
        UT["Unit Test Harness"]:::test
        TR["Test Results"]:::test
        LB["Local Benchmark Notebook"]:::test
        LM["Local Metrics"]:::test
    end

    %% HPC Environment
    subgraph "HPC Cluster"
        HC["HPC Client Scripts"]:::test
        SL["SLURM Scheduler"]:::service
        CN["Compute Nodes"]:::env
        OS["HPC Output Storage"]:::data
    end

    %% Workflow Arrows
    DSLocal -->|"ingest (insert/search)"| TL
    ExternalDS -->|"ingest (insert/search)"| HC

    TL -->|"run tests"| UT
    UT -->|"generate results"| TR

    TL -->|"benchmark with small data"| LB
    LB -->|"collect metrics"| LM

    HC -->|"submit jobs"| SL
    SL -->|"execute on"| CN
    CN -->|"store logs & metrics"| OS

    %% Interconnections
    TL --> HC

    %% Click Events
    click DSLocal "https://github.com/r-niemans/cds_ternarysearchtree/tree/main/data/"
    click DSLocal "https://github.com/r-niemans/cds_ternarysearchtree/blob/main/data/corncob_lowercase.txt"
    click DSLocal "https://github.com/r-niemans/cds_ternarysearchtree/blob/main/data/insert_words.txt"
    click DSLocal "https://github.com/r-niemans/cds_ternarysearchtree/blob/main/data/not_insert_words.txt"
    click DSLocal "https://github.com/r-niemans/cds_ternarysearchtree/blob/main/data/words_alpha.txt"
    click DSLocal "https://github.com/r-niemans/cds_ternarysearchtree/blob/main/data/README.md"
    click ExternalDS "https://github.com/r-niemans/cds_ternarysearchtree/blob/main/HPC/dataset_frequency.py"
    click ExternalDS "https://github.com/r-niemans/cds_ternarysearchtree/blob/main/HPC/dataset_frequency_pkl.py"
    click ExternalDS "https://github.com/r-niemans/cds_ternarysearchtree/blob/main/HPC/dataset_wiki_pkl.py"
    click TL "https://github.com/r-niemans/cds_ternarysearchtree/tree/main/trees/"
    click TL "https://github.com/r-niemans/cds_ternarysearchtree/blob/main/trees/btree.py"
    click TL "https://github.com/r-niemans/cds_ternarysearchtree/blob/main/trees/ternary_tree.py"
    click TL "https://github.com/r-niemans/cds_ternarysearchtree/blob/main/trees/ternary_tree_B.py"
    click TL "https://github.com/r-niemans/cds_ternarysearchtree/blob/main/trees/ternary_tree_minimalistic.py"
    click TL "https://github.com/r-niemans/cds_ternarysearchtree/blob/main/trees/ternary_tree_recursive.py"
    click UT "https://github.com/r-niemans/cds_ternarysearchtree/blob/main/test_tst.py"
    click LB "https://github.com/r-niemans/cds_ternarysearchtree/blob/main/tst_implementations.ipynb"
    click HC "https://github.com/r-niemans/cds_ternarysearchtree/blob/main/HPC/insert_test.py"
    click HC "https://github.com/r-niemans/cds_ternarysearchtree/blob/main/HPC/insert_test2.py"
    click HC "https://github.com/r-niemans/cds_ternarysearchtree/blob/main/HPC/search_test.py"
    click HC "https://github.com/r-niemans/cds_ternarysearchtree/blob/main/HPC/search_test2.py"
    click HC "https://github.com/r-niemans/cds_ternarysearchtree/blob/main/HPC/search_exact_test.py"
    click HC "https://github.com/r-niemans/cds_ternarysearchtree/blob/main/HPC/search_exact_test2.py"
    click HC "https://github.com/r-niemans/cds_ternarysearchtree/blob/main/HPC/tst_full_execution.py"
    click HC "https://github.com/r-niemans/cds_ternarysearchtree/blob/main/HPC/vsc_py_file.py"
    click SL "https://github.com/r-niemans/cds_ternarysearchtree/blob/main/HPC/jobscript.slurm"
    click SL "https://github.com/r-niemans/cds_ternarysearchtree/blob/main/HPC/jobscript2.slurm"
    click OS "https://github.com/r-niemans/cds_ternarysearchtree/blob/main/HPC/Outputs/slurm-out.zip"

    %% Styles
    classDef core fill:#D6EAF8,stroke:#1B4F72,color:#1B2631;
    classDef test fill:#D5F5E3,stroke:#196F3D,color:#0B5345;
    classDef data fill:#FCF3CF,stroke:#B7950B,color:#7D6608,shape:cylinder;
    classDef env fill:#FADBD8,stroke:#C0392B,color:#641E16;
    classDef service fill:#E8DAEF,stroke:#7D3C98,color:#4A235A,shape:roundrect;
```
