How ayhamjo7/iquantum works
Project type and purpose
• This is a static “Awesome Lists” aggregator: a build‐time data‐processing pipeline plus a client‐side web application. It automatically pulls in multiple GitHub “awesome list” repositories, extracts and validates their links, outputs JSON, and hosts a searchable static site on GitHub Pages.
Main components
a. Source Repositories
• External GitHub “awesome‑list” repos (vinta/awesome‑python, awesome‑nodejs, etc.)
b. CI/CD / Automation Layer
• GitHub Actions workflow (daily-update.yml)
• Triggers: cron schedule → jobs
c. Data Extraction & Processing
• Python scripts in scripts/
– extract_data.py: clones each repo, parses README.md, builds raw JSON
– filter_valid_urls.py: checks each URL for 200 OK
– update_awesome_lists.py & main.py: orchestrate extraction + filtering + writing to data/
• Input folders: none (scripts clone directly)
• Output files: data/awesome-lists.json, data/metadata.json
d. Data Storage
• Local JSON files in data/ committed back into the repo
e. Hosting
• GitHub Pages serves static site from the main branch
f. Web Interface (Client)
• index.html
• CSS: css/styles.css, css/aurora-title.css
• JavaScript:
– js/data-processor.js (loads JSON, builds DOM)
– js/main.js (search orchestration via Fuse.js)
– js/theme.js, js/particles.js, js/stars.js (UI animations)
g. End Users
• Browser clients (mobile & desktop) that fetch the static assets and JSON data
Relationships and data flows
• GitHub Actions → invokes Python scripts → clones external repos → outputs JSON → commits JSON back to repo → GitHub Pages rebuilds site
• Browser clients → HTTP GET index.html, CSS, JS, JSON data → front-end renders searchable UI
• Development: local dev uses python scripts/main.py or npx serve to mimic the same pipeline and hosting
Architectural patterns & design principles
• ETL pipeline for data ingestion: Extract (clone & parse), Transform (filter/validate), Load (write JSON)
• CI/CD automation: scheduled jobs in GitHub Actions
• Static site generation: precomputed JSON + client rendering for zero‑backend runtime
• Separation of concerns: build (Python scripts + CI) vs. runtime (HTML/CSS/JS)