# aditya-grover/node2vec architecture

> Architecture diagram of the GitHub repository aditya-grover/node2vec, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/aditya-grover/node2vec
- Repository: https://github.com/aditya-grover/node2vec

## Overview

##### Project type and purpose
- This is an *open-source machine-learning tool* that implements the node2vec algorithm in two variants:
a. A Python CLI version (src/main.py + node2vec.py)
b. A distributed Scala/Spark version (node2vec_spark/)
- Purpose: consume an edgelist graph, generate biased random walks, train a skip-gram (Word2Vec) model, and emit node embeddings.

##### Main components
- **CLI/Front-end**
- Python entry point: src/main.py parses flags, invokes node2vec.py APIs
- Scala entry point: Main.scala reads arguments (via AbstractParams), sets up SparkContext
- **Graph Loader & Processor**
- Python: node2vec.py loads edgelist, builds in-memory graph
- Spark: GraphOps.scala loads edges into RDDs, constructs distributed graph representations
- **Random Walk Generator**
- Node2vec.scala (Scala) / functions in node2vec.py: implements biased second-order random walks
- **Skip-Gram Trainer**
- Word2vec.scala (Scala) / Word2vec module in Python: trains continuous embeddings on sequences
- **Data Layer**
- Input: graph/karate.edgelist
- Output: emb/karate.emb
- Configuration files: log4j2.properties, requirements.txt, pom.xml
- **Build & Deployment**
- Python: pip installs via requirements.txt
- Scala: Maven project (pom.xml) produces a JAR runnable on Spark cluster
- **External Dependencies**
- Python libraries: numpy, networkx (optional), gensim or pure Python
- Spark Core, Spark RDD APIs, log4j for logging

##### Relationships & Data Flows
- User invokes CLI → Parser → Graph Loader
- Graph Loader → Random Walk Generator (produces sequences of node IDs)
- Random Walks → Skip-Gram Trainer → Embedding vectors
- Trainer → persisting vectors to Output file
- In Spark path: data flows over RDD transformations between GraphOps → Node2vec → Word2vec

##### Architectural patterns & principles
- **Modular design**: clear separation between graph handling, sampling, and training
- **Distributed data processing** (Scala path): leverages RDD lineage, transformations, and actions
- **CLI‐driven configurability**: both implementations read flags/params for walk length, p/q, dimensions, etc.
- **Extensibility points**:
- Adding new sampling strategies in Node2vec.scala/node2vec.py
- Swapping in a different embedding trainer

##### Essential details
- Optionally number the steps 1–5 along the flow

## Diagram (Mermaid)

```mermaid
flowchart TD
    classDef python fill:#9f6,stroke:#333,stroke-width:1px
    classDef scala fill:#6cf,stroke:#333,stroke-width:1px
    classDef data fill:#ccc,stroke:#333,stroke-width:1px
    classDef external fill:#f96,stroke:#333,stroke-width:1px

    User["User CLI"]:::external

    subgraph "Data Layer"
        DInput["graph/karate.edgelist"]:::data
        DOutput["emb/karate.emb"]:::data
    end

    subgraph "Python Path"
        PMain["src/main.py"]:::python
        PyNode["src/node2vec.py"]:::python
        PMain -->|"parse flags"| PyNode
        DInput -->|"load edges"| PyNode
        PyNode -->|"save embeddings"| DOutput
    end

    subgraph "Scala/Spark Path"
        SMain["Main.scala"]:::scala
        SParams["AbstractParams.scala"]:::scala
        SGraph["GraphOps.scala"]:::scala
        SNode["Node2vec.scala"]:::scala
        SWord["Word2vec.scala"]:::scala
        SMain -->|"parse args"| SParams
        SMain -->|"setup SparkContext"| SGraph
        DInput -->|"edgelist RDD"| SGraph
        SGraph -->|"edges→walks"| SNode
        SNode -->|"walk sequences"| SWord
        SWord -->|"save embeddings"| DOutput
    end

    subgraph "Build & Config"
        Req["requirements.txt"]:::external
        Pom["pom.xml"]:::external
        LogConf["log4j2.properties"]:::external
        Spark["Spark Cluster"]:::external
        Req -->|"pip install"| User
        Pom -->|"mvn package"| User
        Pom -->|"deploy JAR"| Spark
        LogConf --> SMain
    end

    User --> PMain
    User --> SMain

    click PMain "https://github.com/aditya-grover/node2vec/blob/master/src/main.py"
    click PyNode "https://github.com/aditya-grover/node2vec/blob/master/src/node2vec.py"
    click SMain "https://github.com/aditya-grover/node2vec/blob/master/node2vec_spark/src/main/scala/com/navercorp/Main.scala"
    click SParams "https://github.com/aditya-grover/node2vec/blob/master/node2vec_spark/src/main/scala/com/navercorp/lib/AbstractParams.scala"
    click SGraph "https://github.com/aditya-grover/node2vec/blob/master/node2vec_spark/src/main/scala/com/navercorp/graph/GraphOps.scala"
    click SNode "https://github.com/aditya-grover/node2vec/blob/master/node2vec_spark/src/main/scala/com/navercorp/Node2vec.scala"
    click SWord "https://github.com/aditya-grover/node2vec/blob/master/node2vec_spark/src/main/scala/com/navercorp/Word2vec.scala"
    click DInput "https://github.com/aditya-grover/node2vec/blob/master/graph/karate.edgelist"
    click DOutput "https://github.com/aditya-grover/node2vec/blob/master/emb/karate.emb"
    click Req "https://github.com/aditya-grover/node2vec/blob/master/requirements.txt"
    click Pom "https://github.com/aditya-grover/node2vec/blob/master/node2vec_spark/pom.xml"
    click LogConf "https://github.com/aditya-grover/node2vec/blob/master/node2vec_spark/src/main/resources/log4j2.properties"
```
