# abhishek-b-r/eresults.kle-scraper architecture

> Architecture diagram of the GitHub repository abhishek-b-r/eresults.kle-scraper, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/abhishek-b-r/eresults.kle-scraper
- Repository: https://github.com/abhishek-b-r/eresults.kle-scraper

## Overview

##### Project type and purpose
- This is a CLI-based, TypeScript web-scraping tool whose sole purpose is to automate fetching student results from the KLE Tech “eResults” portal and emit them in an Excel workbook.

##### File structure
- Top-level files:
- data.json – user-supplied configuration (branch code, year, start/end USNs)
- results.xlsx – the output artifact produced after a run
- package.json / package-lock.json – Node.js dependency manifests
- tsconfig.json / tsconfig.tsbuildinfo – TypeScript compiler settings
- src/index.ts: the single entry point. It will:
a. Load data.json
b. Drive the scraping workflow (HTTP requests, parsing)
c. Generate the .xlsx file

##### README for additional insights
- Technologies: Node.js (v14+), npm, TypeScript
- Workflow: user populates data.json → npm run dev → code scrapes portal → writes results.xlsx

##### Main components & their interactions
a. Configuration Loader
- Reads data.json from disk → passes settings to the Scraper Engine
b. Scraper Engine
- HTTP Client (Axios / node-fetch)
- Rate limiting / retry logic (if implemented)
- Targets the KLE Tech eResults endpoint (External Service)
c. HTML Parser
- Cheerio (or built-in DOM parsing)
- Transforms HTML pages → structured JavaScript objects (StudentRecord[])
d. Data Model / Aggregator
- Collects StudentRecord objects across multiple USNs
- Performs in-memory aggregation / validation
e. Excel Generator
- Uses an XLSX library to map StudentRecord[] → .xlsx workbook
f. File System
- Writes results.xlsx to disk

External Service:
- `https://eresults.kletech.ac.in` (or equivalent)

##### Architectural patterns & design principles
- “Pipeline” / “ETL” style: Extract (HTTP) → Transform (Parser) → Load (Excel)
- Single-process, linear workflow (no microservices)
- Clear separation of concerns: config | scrape | parse | export

## Diagram (Mermaid)

```mermaid
flowchart TB
    ConfigFile["data.json"]:::io
    UserCLI["User runs npm script\n/ CLI flags"]:::runtime
    Runtime["Node.js Runtime\n(TypeScript)"]:::runtime
    ConfigLoader["Config Loader\n(TypeScript)"]:::proc
    ScraperEngine["Scraper Engine\n(Axios, Throttler)"]:::proc
    ExternalService["KLE Tech eResults\n(https://eresults.kletech.ac.in)"]:::external
    HTMLParser["HTML Parser\n(Cheerio)"]:::proc
    DataAggregator["Data Aggregator\n(StudentRecord[])"]:::proc
    ExcelGenerator["Excel Generator\n(xlsx)"]:::proc
    FileSystem["File System\n(results.xlsx)"]:::io
    ProjectConfigs["Project Configs\npackage.json, package-lock.json,\ntsconfig.json, tsconfig.tsbuildinfo"]:::io

    ConfigFile --> ConfigLoader
    UserCLI --> Runtime
    Runtime --> ConfigLoader
    ConfigLoader --> ScraperEngine
    ScraperEngine --> ExternalService
    ScraperEngine --> HTMLParser
    HTMLParser --> DataAggregator
    DataAggregator --> ExcelGenerator
    ExcelGenerator --> FileSystem
    Runtime --> ProjectConfigs

    click ConfigFile "https://github.com/abhishek-b-r/eresults.kle-scraper/blob/main/data.json"
    click ConfigLoader "https://github.com/abhishek-b-r/eresults.kle-scraper/blob/main/src/index.ts"
    click ScraperEngine "https://github.com/abhishek-b-r/eresults.kle-scraper/blob/main/src/index.ts"
    click HTMLParser "https://github.com/abhishek-b-r/eresults.kle-scraper/blob/main/src/index.ts"
    click DataAggregator "https://github.com/abhishek-b-r/eresults.kle-scraper/blob/main/src/index.ts"
    click ExcelGenerator "https://github.com/abhishek-b-r/eresults.kle-scraper/blob/main/src/index.ts"
    click FileSystem "https://github.com/abhishek-b-r/eresults.kle-scraper/blob/main/src/index.ts"
    click FileSystem "https://github.com/abhishek-b-r/eresults.kle-scraper/blob/main/results.xlsx"
    click ProjectConfigs "https://github.com/abhishek-b-r/eresults.kle-scraper/blob/main/package.json"
    click ProjectConfigs "https://github.com/abhishek-b-r/eresults.kle-scraper/blob/main/package-lock.json"
    click ProjectConfigs "https://github.com/abhishek-b-r/eresults.kle-scraper/blob/main/tsconfig.json"
    click ProjectConfigs "https://github.com/abhishek-b-r/eresults.kle-scraper/blob/main/tsconfig.tsbuildinfo"

    classDef io fill:#D0E4F5,stroke:#036,stroke-width:1px
    classDef runtime fill:#E0E0E0,stroke:#666,stroke-width:1px
    classDef proc fill:#C8F0C8,stroke:#080,stroke-width:1px
    classDef external fill:#FAD7A0,stroke:#FA0,stroke-width:1px
```
