# us107/github-repos-3d-graph-representation architecture

> Architecture diagram of the GitHub repository us107/github-repos-3d-graph-representation, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/us107/github-repos-3d-graph-representation
- Repository: https://github.com/us107/github-repos-3d-graph-representation

## Overview

##### Project Type and Purpose
- This is a data-processing + visualization pipeline.
- It fetches GitHub metadata, generates embeddings, computes cos-similarity, builds a graph, and renders it in-browser via Sigma.js.

##### Main Components
a. External Services
- GitHub API (metadata)
- OpenAI API (text embeddings)
b. ETL Scripts (Python)
- fetch_github_data.py – retrieves repo metadata → repos_data.json
- .py (or equivalent) – generates embeddings → similarity_matrix.npy
- build_graph.py – builds NetworkX graph → graph.json
- sigma.py – converts graph.json → sigma_graph.json / graph-data.js
c. Data Storage (Local Files)
- repos_data.json, repo_names.json
- similarity_matrix.npy
- graph.json, sigma_graph.json / graph-data.js
d. Visualization Frontend (Web)
- index.html + graph-data.js → Sigma.js renderer in browser

##### Architectural Patterns & Principles
- Linear ETL pipeline: sequential data transformations
- Separation of concerns: each script has one responsibility
- Client-server split: heavy compute in Python scripts, static assets served to browser
- Data-at-rest: intermediate JSON / NumPy files for reproducibility

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% External Services
    GHAPI["GitHub API"]:::external
    OAI["OpenAI API"]:::external

    %% ETL Pipeline
    subgraph "ETL Pipeline"
        FG["fetch_github_data.py"]:::script
        AN["analyze.py"]:::script
        BG["build_graph.py"]:::script
        SG["sigma.py"]:::script
    end

    %% Data Storage
    subgraph "Data Storage"
        RD["repos_data.json"]:::data
        RN["repo_names.json"]:::data
        SM["similarity_matrix.npy"]:::data
        GJ["graph.json"]:::data
        SJ["sigma_graph.json"]:::data
        GD["graph-data.js"]:::data
    end

    %% Frontend
    subgraph "Frontend"
        IH["index.html"]:::frontend
        BZ["Browser (Sigma.js)"]:::frontend
    end

    %% Data Flow
    GHAPI -->|HTTP GET| FG
    FG -->|writes repos_data.json| RD
    FG -->|writes repo_names.json| RN

    RD -->|reads repos_data.json| AN
    RN -->|reads repo_names.json| AN
    AN -->|writes similarity_matrix.npy| SM
    AN -->|calls OpenAI API| OAI

    RD -->|reads repos_data.json| BG
    SM -->|reads similarity_matrix.npy| BG
    BG -->|writes graph.json| GJ

    GJ -->|reads graph.json| SG
    SG -->|writes sigma_graph.json| SJ
    SG -->|writes graph-data.js| GD

    GD -->|loaded by| IH
    IH -->|renders graph| BZ

    %% Click Events
    click FG "https://github.com/us107/github-repos-3d-graph-representation/blob/main/fetch_github_data.py"
    click AN "https://github.com/us107/github-repos-3d-graph-representation/blob/main/analyze.py"
    click BG "https://github.com/us107/github-repos-3d-graph-representation/blob/main/build_graph.py"
    click SG "https://github.com/us107/github-repos-3d-graph-representation/blob/main/sigma.py"
    click RD "https://github.com/us107/github-repos-3d-graph-representation/blob/main/repos_data.json"
    click RN "https://github.com/us107/github-repos-3d-graph-representation/blob/main/repo_names.json"
    click SM "https://github.com/us107/github-repos-3d-graph-representation/blob/main/similarity_matrix.npy"
    click GJ "https://github.com/us107/github-repos-3d-graph-representation/blob/main/graph.json"
    click SJ "https://github.com/us107/github-repos-3d-graph-representation/blob/main/sigma_graph.json"
    click GD "https://github.com/us107/github-repos-3d-graph-representation/blob/main/graph-data.js"
    click IH "https://github.com/us107/github-repos-3d-graph-representation/blob/main/index.html"

    %% Styles
    classDef external fill:#d5f5e3,stroke:#27ae60,stroke-width:2px
    classDef script fill:#d6eaf8,stroke:#2e86c1,stroke-width:2px
    classDef data fill:#fcf3cf,stroke:#f1c40f,stroke-width:2px
    classDef frontend fill:#fdebd0,stroke:#d35400,stroke-width:2px
```
