# smartlord7/qidlearninglib architecture

> Architecture diagram of the GitHub repository smartlord7/qidlearninglib, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/smartlord7/qidlearninglib
- Repository: https://github.com/smartlord7/qidlearninglib

## Overview

##### Project Type and Purpose
- QIDLearningLib is an open-source Python library (not a web or full-stack app) whose purpose is to automate the detection, evaluation, and optimization of quasi-identifiers (QIDs) in tabular datasets. It provides a suite of privacy, utility, causality, and performance metrics, redundancy analysis, multiple optimization strategies, CSV export, and visualization utilities.

##### File-Structure Analysis
- Top-level:
- src/QIDLearningLib/…: the main package
- src/QIDLearningLib/datasets: built-in sample datasets
- src/QIDLearningLib/doc: generated HTML docs
- src/QIDLearningLib/main.py: CLI/entry point
- .github/workflows: CI (unit tests, lint)
- Core modules under src/QIDLearningLib/QIDLearningLib:
- metrics/ (causality, correlation, data_privacy, data_utility, performance, qid_specific)
- optimizer/ (evolutionary_algorithm, greedy_search, simulated_annealing, tabu_search, util, graph, metric)
- structure/ (defines data structures, grouped_metric)
- util/ (data I/O, stats, time)
- test/ (unit tests)
- Patterns: layered modules (data I/O → metrics → redundancy analysis → optimization → export/visualization), strategy pattern for pluggable optimizers and metrics.

##### README Insights
- Describes features: metric suite, redundancy analysis, flexible optimization, extensibility, CSV export, visualization
- Highlights interoperability with ARX and Amnesia via CSV output
- No UI—interaction via CLI or Python API

1. CLI / Python API (main.py)
2. Data Loader (util/data.py)
3. Metrics Engine (metrics/*)
4. Redundancy Analyzer (metrics/correlation + metric-redundancy utilities)
5. Optimization Framework (optimizer/*)
– Evolutionary Algorithm
– Simulated Annealing
– Greedy Search
– Tabu Search
6. Result Exporter (CSV writer)
7. Visualization Tools (plotting utilities in util and optional graph module)
8. Built-in Datasets (datasets/*)
9. External Tools (ARX, Amnesia)
10. Test Harness (test/*)
11. Documentation Generator (doc/)

- CLI/API → Data Loader → Metrics Engine → Redundancy Analyzer → Optimization Framework → Result Exporter → Visualization → CLI/API
- Datasets feed into Data Loader
- Optimization Framework calls back into Metrics Engine for evaluating candidate QID sets
- Test Harness exercises Metrics Engine and Optimization Framework in isolation
- Documentation Generator reads docstrings and outputs HTML under doc/

c. Architectural Patterns & Principles:
- Layered architecture: separation between data I/O, computation, optimization, and presentation
- Strategy pattern: interchangeable optimizer modules implementing a common interface
- Modular metric plug-ins: each metric lives in its own file/folder for extensibility
- Dependency inversion: high‐level workflow code depends on abstract metric and optimizer interfaces, not concrete implementations

d. Key Technologies & Frameworks:
- Python 3.x
- pandas, numpy, scipy for data handling and computations
- matplotlib or seaborn (implied) for plotting
- pytest or unittest for test/
- setuptools / pip for packaging
- CI via GitHub Actions (.github/workflows/workflow.yml)

##### Details to Capture
- The feedback loop where the optimizer invokes metrics to score candidate QID sets
- Test harness and documentation generator as side components that verify and describe the system

## Diagram (Mermaid)

```mermaid
graph TB
    %% Entry Layer
    subgraph "Entry Layer"
        CLI["CLI / Python API"]:::infra
        EntryScript["Entry Point Script"]:::infra
    end

    %% Data I/O
    DataLoader["Data Loader"]:::io

    %% Metrics Engine
    subgraph "Metrics Engine" 
        CoreMetrics["Metrics Engine"]:::core
        Causality["Causality"]:::core
        Correlation["Redundancy Analyzer"]:::core
        Privacy["Data Privacy"]:::core
        Utility["Data Utility"]:::core
        Performance["Performance"]:::core
        QIDSpecific["QID Specific"]:::core
    end

    %% Optimization Framework
    subgraph "Optimization Framework"
        OptimizerCore["Optimizer Core"]:::core
        EvoAlg["Evolutionary Algorithm"]:::core
        Greedy["Greedy Search"]:::core
        SimAnneal["Simulated Annealing"]:::core
        Tabu["Tabu Search"]:::core
        OptMetric["Optimizer Metric Interface"]:::core
        OptUtil["Optimizer Utilities"]:::core
        OptGraph["Optimizer Graph Utils"]:::core
    end

    %% Result and Visualization
    ResultExporter["Result Exporter (CSV)"]:::io
    Visualization["Visualization Tools"]:::io

    %% Datasets and Integration
    Datasets["Built-in Datasets"]:::io
    ARX["ARX"]:::external
    Amnesia["Amnesia"]:::external

    %% Testing and Documentation
    TestHarness["Test Harness"]:::infra
    DocGen["Documentation Generator"]:::infra
    DocOutput["Documentation Output"]:::io

    %% Data Structures & Util modules
    GroupedMetric["Grouped Metrics"]:::core
    StatsUtil["Stats Utility"]:::infra
    TimeUtil["Time Utility"]:::infra

    %% Flows
    CLI --> EntryScript
    EntryScript --> DataLoader
    Datasets --> DataLoader
    DataLoader --> CoreMetrics
    CoreMetrics --> Correlation
    Correlation --> OptimizerCore
    CoreMetrics --> Privacy
    CoreMetrics --> Utility
    CoreMetrics --> Performance
    CoreMetrics --> QIDSpecific
    OptimizerCore --> EvoAlg
    OptimizerCore --> Greedy
    OptimizerCore --> SimAnneal
    OptimizerCore --> Tabu
    EvoAlg --> CoreMetrics
    Greedy --> CoreMetrics
    SimAnneal --> CoreMetrics
    Tabu --> CoreMetrics
    OptimizerCore --> OptMetric
    OptimizerCore --> OptUtil
    OptimizerCore --> OptGraph
    OptimizerCore --> ResultExporter
    ResultExporter --> Visualization
    Visualization --> CLI

    TestHarness --> CoreMetrics
    TestHarness --> OptimizerCore

    DocGen --> CoreMetrics
    DocGen --> OptimizerCore
    DocGen --> DataLoader
    DocGen --> GroupedMetric
    DocGen --> StatsUtil
    DocGen --> TimeUtil
    DocGen --> DocOutput

    ResultExporter --> ARX
    ResultExporter --> Amnesia

    %% Click Events
    click CLI "https://github.com/smartlord7/qidlearninglib/blob/master/src/QIDLearningLib/QIDLearningLib/QIDLearningLib/main.py"
    click EntryScript "https://github.com/smartlord7/qidlearninglib/blob/master/src/QIDLearningLib/main.py"
    click DataLoader "https://github.com/smartlord7/qidlearninglib/blob/master/src/QIDLearningLib/QIDLearningLib/QIDLearningLib/util/data.py"
    click CoreMetrics "https://github.com/smartlord7/qidlearninglib/tree/master/src/QIDLearningLib/QIDLearningLib/QIDLearningLib/metrics/"
    click Correlation "https://github.com/smartlord7/qidlearninglib/blob/master/src/QIDLearningLib/QIDLearningLib/QIDLearningLib/metrics/correlation.py"
    click OptimizerCore "https://github.com/smartlord7/qidlearninglib/tree/master/src/QIDLearningLib/QIDLearningLib/QIDLearningLib/optimizer/"
    click EvoAlg "https://github.com/smartlord7/qidlearninglib/blob/master/src/QIDLearningLib/QIDLearningLib/QIDLearningLib/optimizer/evolutionary_algorithm/evolutionary_algorithm.py"
    click Greedy "https://github.com/smartlord7/qidlearninglib/blob/master/src/QIDLearningLib/QIDLearningLib/QIDLearningLib/optimizer/greedy_search/greedy_search.py"
    click SimAnneal "https://github.com/smartlord7/qidlearninglib/blob/master/src/QIDLearningLib/QIDLearningLib/QIDLearningLib/optimizer/simulated_annealing/simulated_annealing.py"
    click Tabu "https://github.com/smartlord7/qidlearninglib/blob/master/src/QIDLearningLib/QIDLearningLib/QIDLearningLib/optimizer/tabu_search/tabu_search.py"
    click OptMetric "https://github.com/smartlord7/qidlearninglib/blob/master/src/QIDLearningLib/QIDLearningLib/QIDLearningLib/optimizer/metric.py"
    click OptUtil "https://github.com/smartlord7/qidlearninglib/blob/master/src/QIDLearningLib/QIDLearningLib/QIDLearningLib/optimizer/util.py"
    click OptGraph "https://github.com/smartlord7/qidlearninglib/blob/master/src/QIDLearningLib/QIDLearningLib/QIDLearningLib/optimizer/graph.py"
    click ResultExporter "https://github.com/smartlord7/qidlearninglib/blob/master/src/QIDLearningLib/QIDLearningLib/QIDLearningLib/util/data.py"
    click Visualization "https://github.com/smartlord7/qidlearninglib/blob/master/src/QIDLearningLib/QIDLearningLib/QIDLearningLib/optimizer/graph.py"
    click Datasets "https://github.com/smartlord7/qidlearninglib/tree/master/src/QIDLearningLib/datasets/"
    click TestHarness "https://github.com/smartlord7/qidlearninglib/tree/master/src/QIDLearningLib/QIDLearningLib/QIDLearningLib/test/"
    click DocOutput "https://github.com/smartlord7/qidlearninglib/tree/master/src/QIDLearningLib/doc/"
    click GroupedMetric "https://github.com/smartlord7/qidlearninglib/blob/master/src/QIDLearningLib/QIDLearningLib/QIDLearningLib/structure/grouped_metric.py"
    click StatsUtil "https://github.com/smartlord7/qidlearninglib/blob/master/src/QIDLearningLib/QIDLearningLib/QIDLearningLib/util/stats.py"
    click TimeUtil "https://github.com/smartlord7/qidlearninglib/blob/master/src/QIDLearningLib/QIDLearningLib/QIDLearningLib/util/time.py"

    %% Styles
    classDef core fill:#ADD8E6,stroke:#000,stroke-width:1px
    classDef infra fill:#90EE90,stroke:#000,stroke-width:1px
    classDef io fill:#FFA500,stroke:#000,stroke-width:1px
    classDef external fill:#D3D3D3,stroke:#000,stroke-width:1px
```
