How us107/github-repos-3d-graph-representation works
Project Type and Purpose
• This is a data-processing + visualization pipeline.
• It fetches GitHub metadata, generates embeddings, computes cos-similarity, builds a graph, and renders it in-browser via Sigma.js.
Main Components
a. External Services
• GitHub API (metadata)
• OpenAI API (text embeddings)
b. ETL Scripts (Python)
• fetch_github_data.py – retrieves repo metadata → repos_data.json
• .py (or equivalent) – generates embeddings → similarity_matrix.npy
• build_graph.py – builds NetworkX graph → graph.json
• sigma.py – converts graph.json → sigma_graph.json / graph-data.js
c. Data Storage (Local Files)
• repos_data.json, repo_names.json
• similarity_matrix.npy
• graph.json, sigma_graph.json / graph-data.js
d. Visualization Frontend (Web)
• index.html + graph-data.js → Sigma.js renderer in browser
Architectural Patterns & Principles
• Linear ETL pipeline: sequential data transformations
• Separation of concerns: each script has one responsibility
• Client-server split: heavy compute in Python scripts, static assets served to browser
• Data-at-rest: intermediate JSON / NumPy files for reproducibility