# naman14113114/practice_paper_generator architecture

> Architecture diagram of the GitHub repository naman14113114/practice_paper_generator, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/naman14113114/practice_paper_generator
- Repository: https://github.com/naman14113114/practice_paper_generator

## Overview

1. **Overall shape**
   - This is a **Python, offline-first question-bank pipeline**.
   - The repository is split into two main concerns: **ingestion** of questions from aptitude sites and **local storage / future paper generation** from normalized JSON banks.

2. **Primary entry point for ingestion**
   - `scrape.py` is the current operational entry point.
   - It drives the scraping workflow end-to-end: choose a scraper, fetch questions from a source URL, normalize them, and merge them into the local bank.

3. **Scraper subsystem**
   - `scrapers/base.py` defines the common scraper interface.
   - `scrapers/indiabix.py` is the concrete implementation currently in use.
   - `scrapers/examveda.py` is a placeholder for another source, showing the intended extension pattern.
   - This boundary isolates website-specific parsing from the rest of the system.

4. **Question normalization contract**
   - Scrapers emit a shared question shape with fields like `question`, `options`, `answer`, `source`, and `topic_url`.
   - That common schema is the key integration point: different websites can feed the same storage layer without downstream code caring where a question came from.

5. **Local persistence layer**
   - `storage/question_bank.py` manages the JSON-backed question repository.
   - The system stores questions by topic under `data/banks/*.json`, such as `probability.json`.
   - This is the durable system of record; there is no database or remote backend in the current design.

6. **Merge and deduplication boundary**
   - The bank layer is responsible for merging new scraped questions into an existing topic bank and removing duplicates.
   - This keeps ingestion idempotent and prevents repeated scraping from polluting the bank.

7. **Data organization**
   - Each topic is isolated into its own JSON file, which makes the storage model simple and file-based.
   - This means the architecture is topic-centric rather than source-centric: one bank can accumulate questions from multiple sites over time.

8. **Generation subsystem**
   - `generator/generate.py` and `generator/paper.py` represent the practice-paper assembly layer.
   - The README describes this as planned/future-facing, but the code layout already reserves a separate boundary for it.
   - Its job is to consume banks, select questions, randomize them, and produce a paper artifact.

9. **Planned paper-generation flow**
   - The intended downstream flow is: **question banks → selection/filtering → randomization → paper output**.
   - The README calls out future features like multi-topic papers, difficulty balancing, answer keys, and printable templates.

10. **Extensibility model**
   - New source support is added by implementing the base scraper interface and wiring it into the scraping entry point.
   - New content is added by topic through additional JSON files, without changing the storage model.
   - This makes the architecture modular at the source boundary and lightweight at the persistence boundary.

11. **Operational characteristics**
   - The project is designed to be **local, deterministic, and easy to maintain**.
   - It does not depend on a web service, queue, or database; the filesystem is the main integration surface.
   - `requirements.txt` suggests a small Python dependency set, consistent with a simple scraping-and-storage toolchain.

12. **How to draw the architecture**
   - Draw three layers: **website scrapers** on the left, **question bank storage** in the middle, and **paper generation** on the right.
   - Show `scrape.py` as the current orchestrator feeding normalized questions into `storage/question_bank.py`.
   - Show `data/banks/*.json` as the persistent output, and `generator/*` as a separate consumer that will read those banks to build practice papers.

Main entry points: [`storage/question_bank.py`](https://github.com/naman14113114/practice_paper_generator/blob/main/storage/question_bank.py) (Question bank), [`scrape.py`](https://github.com/naman14113114/practice_paper_generator/blob/main/scrape.py) (scrape.py), [`scrapers/base.py`](https://github.com/naman14113114/practice_paper_generator/blob/main/scrapers/base.py) (Base scraper), [`generator/generate.py`](https://github.com/naman14113114/practice_paper_generator/blob/main/generator/generate.py) (Paper builder)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ingestion["Ingestion"]
  node_scrape_py["scrape.py<br/>orchestrator<br/>[scrape.py]"]
  node_scrapers_base_py["Base scraper<br/>interface<br/>[base.py]"]
  node_scrapers_indiabix_py["Indiabix<br/>scraper<br/>[indiabix.py]"]
  node_scrapers_examveda_py["ExamVeda<br/>scraper stub<br/>[examveda.py]"]
  node_normalized_question{{"Normalized question<br/>data contract"}}
end

subgraph group_storage["Storage"]
  node_storage_question_bank_py[("Question bank<br/>bank store<br/>[question_bank.py]")]
  node_data_banks_probability_json["Probability bank<br/>topic json<br/>[probability.json]"]
end

subgraph group_generation["Generation"]
  node_generator_generate_py["Paper builder<br/>assembler<br/>[generate.py]"]
  node_generator_paper_py["Paper model<br/>paper format<br/>[paper.py]"]
  node_generate_py["generate.py<br/>entrypoint<br/>[generate.py]"]
end

node_scrape_py -->|"uses"| node_scrapers_base_py
node_scrape_py -->|"runs"| node_scrapers_indiabix_py
node_scrape_py -->|"merges into"| node_storage_question_bank_py
node_scrapers_base_py -->|"implements"| node_scrapers_indiabix_py
node_scrapers_base_py -->|"extends"| node_scrapers_examveda_py
node_scrapers_indiabix_py -->|"emits"| node_normalized_question
node_scrapers_examveda_py -.->|"emits"| node_normalized_question
node_normalized_question -->|"ingested by"| node_storage_question_bank_py
node_storage_question_bank_py -->|"persists"| node_data_banks_probability_json
node_generator_generate_py -->|"reads"| node_storage_question_bank_py
node_generator_generate_py -->|"builds"| node_generator_paper_py
node_generate_py -->|"delegates"| node_generator_generate_py

click node_scrape_py "https://github.com/naman14113114/practice_paper_generator/blob/main/scrape.py"
click node_scrapers_base_py "https://github.com/naman14113114/practice_paper_generator/blob/main/scrapers/base.py"
click node_scrapers_indiabix_py "https://github.com/naman14113114/practice_paper_generator/blob/main/scrapers/indiabix.py"
click node_scrapers_examveda_py "https://github.com/naman14113114/practice_paper_generator/blob/main/scrapers/examveda.py"
click node_storage_question_bank_py "https://github.com/naman14113114/practice_paper_generator/blob/main/storage/question_bank.py"
click node_data_banks_probability_json "https://github.com/naman14113114/practice_paper_generator/blob/main/data/banks/probability.json"
click node_generator_generate_py "https://github.com/naman14113114/practice_paper_generator/blob/main/generator/generate.py"
click node_generator_paper_py "https://github.com/naman14113114/practice_paper_generator/blob/main/generator/paper.py"
click node_generate_py "https://github.com/naman14113114/practice_paper_generator/blob/main/generate.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_scrape_py,node_scrapers_base_py,node_scrapers_indiabix_py,node_scrapers_examveda_py,node_normalized_question toneBlue
class node_storage_question_bank_py,node_data_banks_probability_json toneAmber
class node_generator_generate_py,node_generator_paper_py,node_generate_py toneMint
```

## Components

### Ingestion

Scraping and normalization of questions from external sites.

- [scrape.py](https://github.com/naman14113114/practice_paper_generator/blob/main/scrape.py): orchestrator · Current end-to-end scraping entry point. · `scrape.py`
- [Base scraper](https://github.com/naman14113114/practice_paper_generator/blob/main/scrapers/base.py): interface · Common scraper contract for source-specific parsers. · `scrapers/base.py`
- [Indiabix](https://github.com/naman14113114/practice_paper_generator/blob/main/scrapers/indiabix.py): scraper · Concrete scraper currently used for question ingestion. · `scrapers/indiabix.py`
- [ExamVeda](https://github.com/naman14113114/practice_paper_generator/blob/main/scrapers/examveda.py): scraper stub · Placeholder for adding another question source. · `scrapers/examveda.py`
- **Normalized question**: data contract · Shared question shape passed from scrapers into storage.

### Storage

Local topic banks and merge/dedup persistence.

- [Question bank](https://github.com/naman14113114/practice_paper_generator/blob/main/storage/question_bank.py): bank store · JSON-backed topic bank with merge and dedup logic. · `storage/question_bank.py`
- [Probability bank](https://github.com/naman14113114/practice_paper_generator/blob/main/data/banks/probability.json): topic json · Representative persisted topic bank file. · `data/banks/probability.json`

### Generation

Paper assembly from stored question banks.

- [Paper builder](https://github.com/naman14113114/practice_paper_generator/blob/main/generator/generate.py): assembler · Selects and prepares questions for paper output. · `generator/generate.py`
- [Paper model](https://github.com/naman14113114/practice_paper_generator/blob/main/generator/paper.py): paper format · Paper artifact structure and rendering support. · `generator/paper.py`
- [generate.py](https://github.com/naman14113114/practice_paper_generator/blob/main/generate.py): entrypoint · Root-level wrapper for generation workflows. · `generate.py`

## Connections

- scrape.py → Base scraper: uses · Loads the shared scraper interface to drive ingestion.
- scrape.py → Indiabix: runs · Current operational path for fetching questions.
- scrape.py → Question bank: merges into · Writes normalized questions into the local bank.
- Base scraper → Indiabix: implements · Concrete scraper follows the common contract.
- Base scraper → ExamVeda: extends · Placeholder source also fits the same interface.
- Indiabix → Normalized question: emits · Produces the shared question schema.
- ExamVeda → Normalized question: emits · Would produce the same normalized schema.
- Normalized question → Question bank: ingested by · Storage layer accepts the shared question shape.
- Question bank → Probability bank: persists · Topic bank is stored as JSON on disk.
- Paper builder → Question bank: reads · Consumes banks as the source corpus.
- Paper builder → Paper model: builds · Assembles a paper artifact from selected questions.
- generate.py → Paper builder: delegates · Top-level generation entrypoint forwards to the generator package.