kamwithk/gitstarred

How kamwithk/gitstarred works

GitStarred’s implemented workflow collects GitHub repository data, stores it in PostgreSQL, engineers features with Spark, and trains/evaluates an H2O AutoML model. The README also describes neural-network and website capabilities, but the supplied tree shows no implementation paths for those, so they are not represented as runtime components. Edges are limited to relationships visible in excerpts or documented in the README.

Main entry points: DateBreakdown.py Date breakdown · CollectData.py Collection runner · ProcessTrain.py Process and train · Processor.py Spark processor

Components 16

Data collection

Collection runner src/CollectData.py
Date breakdown src/DataCollection/DateBreakdown.py
GraphQL client src/DataCollection/GraphQL.py
Repository query src/Queries/Collection.graphql
Date query src/Queries/InnerDateTester.graphql
Repository parser src/DataCollection/Parser.py

Feature engineering

Process and train src/ProcessTrain.py
Spark processor src/Pipeline/Processor.py
Feature transforms src/Pipeline/Components.py
Train/test datasets

Model training

H2O AutoML src/Modelling/H2OSparkAutoML.py
Model results

Other components

Analyst
GitHub API
PostgreSQL
Apache Spark

Connections 20

Analyst → Collection runner: starts collection src/CollectData.py
Collection runner → Date breakdown: collects periods src/CollectData.py
Collection runner → Repository parser: creates parser src/CollectData.py
Collection runner → Repository query: loads query src/CollectData.py
Date breakdown → Date query: loads date query src/DataCollection/DateBreakdown.py
Date breakdown → GraphQL client: runs searches src/DataCollection/DateBreakdown.py
Date breakdown → Repository query: pages through
GraphQL client → GitHub API: sends requests src/DataCollection/GraphQL.py
Date breakdown → Repository parser: processes results src/DataCollection/DateBreakdown.py
Repository parser → PostgreSQL: upserts repositories src/DataCollection/Parser.py
Date breakdown → PostgreSQL: tracks progress src/DataCollection/DateBreakdown.py
Analyst → Process and train: starts training src/ProcessTrain.py
Process and train → Spark processor: processes data src/ProcessTrain.py
Spark processor → PostgreSQL: reads repositories src/Pipeline/Processor.py
Spark processor → Apache Spark: runs transformations README.md
Process and train → Feature transforms: configures stages src/ProcessTrain.py
Spark processor → Train/test datasets: saves datasets src/Pipeline/Processor.py
Process and train → H2O AutoML: trains models src/ProcessTrain.py
H2O AutoML → Apache Spark: fits AutoML README.md
H2O AutoML → Model results: returns results src/Modelling/H2OSparkAutoML.py

Drawn by GitDiagram on November 12, 2026 from the repository's file tree, README and source files. Read as Markdown ·