# tolga-b/debiaswe architecture

> Architecture diagram of the GitHub repository tolga-b/debiaswe, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-23). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tolga-b/debiaswe
- Repository: https://github.com/tolga-b/debiaswe

## Overview

This project is a Python-based tool for debiasing word embeddings—specifically, removing gender bias from pre-trained embeddings such as Google News’ Word2Vec. The main goal is to process a given embedding, utilize seed lists and predefined word pairs, and produce a debiased version of the embedding, following the techniques described in the related NIPS paper.

##### Main Components
- Input Embedding: Represent the original word embedding file (e.g., “GoogleNews-vectors-negative300.bin”) that serves as the system’s input.
- Data Files: Include the JSON files under the “data” folder: definitional_pairs.json, equalize_pairs.json, gender_specific_seed.json, and gender_specific_full.json. These files encapsulate the gender-specific words and pairing information used in the debiasing process.
- Processing Modules:
- Learn Gender Specific Module: (debiaswe/learn_gender_specific.py) which takes the embedding and the seed set (gender_specific_seed.json) to learn/extend a full list of gender-specific words (outputting gender_specific_full.json).
- Debiasing Module: (debiaswe/debias.py) that applies the debiasing algorithm. It uses the definitional pairs, the expanded gender-specific words set (gender_specific_full.json), and equalize pairs to produce the hard debiased embedding.
- Embedding I/O Module: (debiaswe/we.py) likely responsible for reading and writing the word2vec binary format.
- Data Utilities: (debiaswe/data.py) for common data handling, possibly abstracting JSON file interactions.
- Output Embedding: The final debiased embedding binary file produced by the debias.py script.

##### Relationships and Data Flow
- Parallel to that, represent the static data files (definitional_pairs.json and equalize_pairs.json) linking to the Debiasing Module.
- Optionally, include a branch for the notebook (tutorial_example1.ipynb) as an illustrative usage example, indicating how a user might interact with the components or run sample experiments.

##### Important Architectural Patterns and Design Principles
- Modularity: The processing is split into clear modules (one for learning gender-specific terms and one for applying debiasing). Each module is organized within the “debiaswe” package.
- Data-Driven Flow: The system uses external JSON files to parameterize processing, allowing adjustments without changing code.
- Separation of Concerns: Input/output handling (in debiaswe/we.py and the script wrappers) is kept separate from the algorithm’s core logic.
- Command-line Script Integration: The project is designed to be executed via Python scripts, which form the operational entry points.

##### Instructing the Principal Software Engineer
- Then, combine the outputs from the “Learn Gender-Specific Module” and the static data files into a module labeled “Debiasing Module (debias.py)”. This module also references the “Embedding I/O Module (we.py)” for reading/writing operations.
- Optionally, indicate a peripheral component for “Tutorial Example (tutorial_example1.ipynb)” that might interface with these modules for demonstration purposes.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Layer
    subgraph "Data Layer"
        D1["data/definitional_pairs.json"]:::data
        D2["data/equalize_pairs.json"]:::data
        D3["data/gender_specific_seed.json"]:::data
        D4["data/gender_specific_full.json"]:::data
    end

    %% Processing Layer
    subgraph "Processing Layer"
        P1["Learn Gender-Specific Module (learn_gender_specific.py)"]:::processing
        P2["Debiasing Module (debias.py)"]:::processing
        P3["Embedding I/O Module (we.py)"]:::processing
        P4["Data Utilities (data.py)"]:::processing
    end

    %% Output Layer
    subgraph "Output Layer"
        O1["Debiased Embedding File"]:::inputOutput
    end

    %% Input and Tutorial
    I1["Input Embedding File"]:::inputOutput
    T1["Tutorial Example (tutorial_example1.ipynb)"]:::notebook

    %% Data Flow
    I1 -->|"processes"| P1
    D3 -->|"provides_seed"| P1
    P1 -->|"outputs_gender_list"| D4

    I1 -->|"reads_embedding"| P2
    D1 -->|"provides_definitional_pairs"| P2
    D2 -->|"provides_equalize_pairs"| P2
    D4 -->|"provides_gender_list"| P2

    P2 -->|"invokes_IO"| P3
    P1 -->|"utilizes"| P4
    P2 -->|"utilizes"| P4

    P2 -->|"generates"| O1

    T1 ---|"demonstrates"| P1
    T1 ---|"demonstrates"| P2

    %% Click Events
    click I1 "https://github.com/tolga-b/debiaswe/tree/master/embeddings"
    click D1 "https://github.com/tolga-b/debiaswe/blob/master/data/definitional_pairs.json"
    click D2 "https://github.com/tolga-b/debiaswe/blob/master/data/equalize_pairs.json"
    click D3 "https://github.com/tolga-b/debiaswe/blob/master/data/gender_specific_seed.json"
    click D4 "https://github.com/tolga-b/debiaswe/blob/master/data/gender_specific_full.json"
    click P1 "https://github.com/tolga-b/debiaswe/blob/master/debiaswe/learn_gender_specific.py"
    click P2 "https://github.com/tolga-b/debiaswe/blob/master/debiaswe/debias.py"
    click P3 "https://github.com/tolga-b/debiaswe/blob/master/debiaswe/we.py"
    click P4 "https://github.com/tolga-b/debiaswe/blob/master/debiaswe/data.py"
    click T1 "https://github.com/tolga-b/debiaswe/blob/master/tutorial_example1.ipynb"

    %% Styles
    classDef data fill:#cce5ff,stroke:#007bff,stroke-width:2px;
    classDef processing fill:#d4edda,stroke:#28a745,stroke-width:2px;
    classDef inputOutput fill:#ffeeba,stroke:#fd7e14,stroke-width:2px;
    classDef notebook fill:#f8d7da,stroke:#dc3545,stroke-width:2px;
```
