How abhishek-b-r/eresults.kle-scraper works
Project type and purpose
• This is a CLI-based, TypeScript web-scraping tool whose sole purpose is to automate fetching student results from the KLE Tech “eResults” portal and emit them in an Excel workbook.
File structure
• Top-level files:
• data.json – user-supplied configuration (branch code, year, start/end USNs)
• results.xlsx – the output artifact produced after a run
• package.json / package-lock.json – Node.js dependency manifests
• tsconfig.json / tsconfig.tsbuildinfo – TypeScript compiler settings
• src/index.ts: the single entry point. It will:
a. Load data.json
b. Drive the scraping workflow (HTTP requests, parsing)
c. Generate the .xlsx file
README for additional insights
• Technologies: Node.js (v14+), npm, TypeScript
• Workflow: user populates data.json → npm run dev → code scrapes portal → writes results.xlsx
Main components & their interactions
a. Configuration Loader
• Reads data.json from disk → passes settings to the Scraper Engine
b. Scraper Engine
• HTTP Client (Axios / node-fetch)
• Rate limiting / retry logic (if implemented)
• Targets the KLE Tech eResults endpoint (External Service)
c. HTML Parser
• Cheerio (or built-in DOM parsing)
• Transforms HTML pages → structured JavaScript objects (StudentRecord[])
d. Data Model / Aggregator
• Collects StudentRecord objects across multiple USNs
• Performs in-memory aggregation / validation
e. Excel Generator
• Uses an XLSX library to map StudentRecord[] → .xlsx workbook
f. File System
• Writes results.xlsx to disk
External Service:
• https://eresults.kletech.ac.in (or equivalent)
Architectural patterns & design principles
• “Pipeline” / “ETL” style: Extract (HTTP) → Transform (Parser) → Load (Excel)
• Single-process, linear workflow (no microservices)
• Clear separation of concerns: config | scrape | parse | export