# goutham7675/internshala-automation architecture

> Architecture diagram of the GitHub repository goutham7675/internshala-automation, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-24). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/goutham7675/internshala-automation
- Repository: https://github.com/goutham7675/internshala-automation

## Overview

##### Project Type and Purpose
- This is a **backend automation tool** written in Node.js. It uses Puppeteer to drive a headless browser, log in to Internshala, run searches, scrape internship/job data, and save results. There is no frontend or UI—just scripted automation.

##### File Structure
- Top-level scripts:
- index.js – the main orchestrator
- internship.js – a module for scraping internship listings
- cover.js – a module for composing cover letters or related tasks
- Configuration:
- package.json / package-lock.json – lists dependencies (Puppeteer, etc.)
- .gitattributes – repo settings
- .env (implicit) – environment variables for credentials
- Dependencies: node_modules folder holds Puppeteer and other npm packages.

##### README for Additional Insights
- Key features: automated login, parameterized searches, data extraction, configurable output (JSON/CSV).
- Tech stack: Node.js + Puppeteer.
- Extensibility: a `scripts/` folder can house additional task modules.

##### Main Components
a. Configuration Layer
– Environment Variables (.env)
– package.json (dependency manifest)
b. Orchestrator
– index.js (initializes config, selects tasks)
c. Task Modules
– internship.js, cover.js, any scripts in `scripts/`
d. Puppeteer Engine
– Headless Chrome instance driven by the Puppeteer library
e. External Service
– Internshala website (login, search, data pages)
f. Storage Layer
– File System output (JSON/CSV)

##### Relationships and Data Flow
1. **Startup**: Node.js loads index.js → reads .env → loads config.
2. **Task Invocation**: index.js calls specific task module (e.g., internship.js).
3. **Browser Automation**: task module imports Puppeteer → launches headless browser → navigates to Internshala.
4. **Data Exchange**:
- Orchestrator → Task Module: “run internship scrape with filters X”
- Task Module → Puppeteer: “navigate to URL, login, extract data”
- Puppeteer ↔ External Site: HTTP requests, DOM interactions
- Task Module → Orchestrator: returns scraped data
- Orchestrator → Storage Layer: writes JSON/CSV
5. **Shutdown**: Puppeteer browser closes, process ends.

##### Architectural Patterns and Design Principles
- **Modularity**: clear separation between orchestrator and task scripts
- **Stateless Execution**: each run is fresh; no persistent server or DB
- **Configuration-Driven**: environment variables for credentials; script parameters for search filters
- **Extensibility**: add new scripts in a `scripts/` folder to support additional tasks

##### Essential Details to Capture
- Environment variable injection at startup.
- That node_modules is a dependency bundle, not runtime logic.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Configuration Layer
    subgraph "Configuration Layer"
        ENV[".env"]:::config
        PKG["package.json"]:::config
        PKGLK["package-lock.json"]:::config
    end

    %% Orchestrator
    ORCH["index.js (Orchestrator)"]:::internal

    %% Automation Scripts
    subgraph "Automation Scripts"
        SCR1["internship.js (Scraper)"]:::internal
        SCR2["cover.js (Cover Generator)"]:::internal
    end

    %% Puppeteer Engine
    PE["Puppeteer Engine"]:::puppeteer

    %% External Service
    subgraph "External Service"
        IS["Internshala Website"]:::external
    end

    %% Storage Layer
    FS["File System (JSON/CSV)"]:::storage

    %% Data Flow
    ENV -->|loads env vars| ORCH
    PKG -->|loads dependencies| ORCH
    PKGLK -->|locks dependencies| ORCH

    ORCH -->|invoke scraper| SCR1
    ORCH -->|invoke cover generator| SCR2

    SCR1 -->|launch browser| PE
    SCR2 -->|launch browser| PE

    PE -->|HTTP requests/DOM| IS
    IS -->|HTTP responses/DOM| PE

    SCR1 -->|return data| ORCH
    SCR2 -->|return data| ORCH

    ORCH -->|write JSON/CSV| FS

    %% Legend
    subgraph "Legend"
        L1["Blue rectangle: Node.js components"]:::legend
        L2["Orange rectangle: Puppeteer engine"]:::legend
        L3["Green cylinder: Storage"]:::legend
        L4["Grey cloud: External Service"]:::legend
    end

    %% Click Events
    click PKG "https://github.com/goutham7675/internshala-automation/blob/main/package.json"
    click PKGLK "https://github.com/goutham7675/internshala-automation/blob/main/package-lock.json"
    click ORCH "https://github.com/goutham7675/internshala-automation/blob/main/index.js"
    click SCR1 "https://github.com/goutham7675/internshala-automation/blob/main/internship.js"
    click SCR2 "https://github.com/goutham7675/internshala-automation/blob/main/cover.js"
    click PE "https://github.com/goutham7675/internshala-automation/tree/main/node_modules/puppeteer"

    %% Styles
    classDef config fill:#e0f7fa,stroke:#006064,color:#006064;
    classDef internal fill:#bbdefb,stroke:#0d47a1,color:#0d47a1;
    classDef puppeteer fill:#ffe0b2,stroke:#e65100,color:#e65100;
    classDef storage fill:#c8e6c9,stroke:#1b5e20,color:#1b5e20;
    classDef external fill:#eeeeee,stroke:#616161,color:#616161;
    classDef legend fill:#ffffff,stroke:#000000,stroke-dasharray:2 2,color:#000000;
```
