# bulktornado/cbse-rdms-project architecture

> Architecture diagram of the GitHub repository bulktornado/cbse-rdms-project, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-05-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bulktornado/cbse-rdms-project
- Repository: https://github.com/bulktornado/cbse-rdms-project

## Overview

1. **Project intent**
   This repository is a Python data-preparation and database project for CBSE school/curriculum information. It ingests curriculum PDFs and raw school datasets, converts them into structured records, and stores them in local data files and a relational database.

2. **Primary pipeline**
   The main flow is: **reference documents and raw text data** in `docs/` and `data/raw_datasets/` → **parsing/cleanup scripts** in `src/` → **tabular/serialized outputs** in `data/` and `backup/` → **database schema and connection layer** for query/use by the application.

3. **Source data boundary**
   `docs/` contains the curriculum PDFs and the project report; these are the human/reference inputs. `data/raw_datasets/` holds extracted or intermediate source material, including reference notes about filtered courses and pickled data snapshots. These are not runtime logic, but they define the input domain.

4. **Parsing and normalization**
   `src/parse_affiliated_school_data.py` and `src/parse_courses.py` are the key transformation scripts. They appear to convert raw school/course sources into structured datasets, likely handling filtering, normalization, and mapping into consistent records for downstream storage.

5. **Setup and configuration boundary**
   `src/setup.py` and `src/set_configuration.py` suggest a one-time or repeatable initialization phase that prepares paths, configuration, and probably database/data generation. This is separate from the parsing logic and from the runtime module layer.

6. **Database schema**
   `schemas/database_schema.sql` defines the relational structure for the project. The tracked dataset names strongly indicate at least two main entities: **affiliated schools** and **courses**, with the schema serving as the contract between parsed data and the database.

7. **Shared runtime modules**
   `modules/connection_module.py` centralizes database connectivity, while `modules/functions.py` contains reusable helpers used across scripts. This is the main internal utility layer that separates low-level I/O/DB handling from parsing/setup logic.

8. **Entry point**
   `main.py` is the top-level entry point for running the project. It likely coordinates setup, data loading, and/or query operations by invoking the modules and scripts above rather than containing the business logic itself.

9. **Derived data and snapshots**
   `data/` and `backup/` contain generated artifacts such as `.dat` and `.txt` files for `affiliated_schools` and `courses`. These look like persistence snapshots and interchange formats, useful for reruns, debugging, or restoring state without re-parsing the original sources.

10. **One-off/manual correction path**
   `src/forbidden_scripts/manually_update_affiliated_schools_from_tabular_data.py` stands out as an exceptional maintenance script. It likely exists for manual data correction or migration and should be treated as outside the normal pipeline.

11. **Execution environment**
   The repository is a Python project managed with `pyproject.toml`, `requirements.txt`, `uv.lock`, and `.python-version`. The architecture is local/offline and file-driven; there are no obvious external network services or web framework boundaries in the repo.

12. **What to draw in an architecture diagram**
   A clear diagram would show:
   - **Reference inputs**: PDFs/text/raw datasets
   - **Transform layer**: parsing and setup scripts
   - **Shared services**: connection/utilities
   - **Storage layer**: schema + generated data files/backups
   - **Entry point**: `main.py`
   The most important boundary is between **raw source material** and **normalized structured data**.

Main entry points: [`src/parse_affiliated_school_data.py`](https://github.com/bulktornado/cbse-rdms-project/blob/main/src/parse_affiliated_school_data.py) (Parse schools), [`src/parse_courses.py`](https://github.com/bulktornado/cbse-rdms-project/blob/main/src/parse_courses.py) (Parse courses), [`src/setup.py`](https://github.com/bulktornado/cbse-rdms-project/blob/main/src/setup.py) (Setup), [`main.py`](https://github.com/bulktornado/cbse-rdms-project/blob/main/main.py) (Main)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_source["Source inputs"]
  node_docs["Docs<br/>reference-inputs"]
  node_raw_data["Raw data<br/>source-datasets"]
end

subgraph group_processing["Processing layer"]
  node_parse_affiliated["Parse schools<br/>parser"]
  node_parse_courses["Parse courses<br/>parser<br/>[parse_courses.py]"]
  node_setup["Setup<br/>initializer<br/>[setup.py]"]
  node_config["Config"]
  node_manual_fix["Manual fix<br/>maintenance-script"]
end

subgraph group_runtime["Shared runtime"]
  node_conn[("DB connection<br/>database-access")]
  node_functions["Helpers<br/>utilities<br/>[functions.py]"]
end

subgraph group_storage["Storage layer"]
  node_schema["Schema"]
  node_schools_data[("Schools data<br/>generated-dataset")]
  node_courses_data[("Courses data<br/>generated-dataset<br/>[courses.dat]")]
  node_tabular_schools["Schools table<br/>tabular-output"]
  node_tabular_courses["Courses table<br/>tabular-output<br/>[courses.txt]"]
  node_backup_store[("Backups<br/>snapshot-store")]
end

subgraph group_entry["Entry point"]
  node_main(("Main<br/>entrypoint<br/>[main.py]"))
end

node_docs -->|"curriculum source"| node_parse_courses
node_docs -->|"reference source"| node_parse_affiliated
node_raw_data -->|"raw input"| node_parse_affiliated
node_raw_data -->|"raw input"| node_parse_courses
node_config -->|"paths"| node_setup
node_config -->|"settings"| node_parse_affiliated
node_config -->|"settings"| node_parse_courses
node_functions -->|"helpers"| node_parse_affiliated
node_functions -->|"helpers"| node_parse_courses
node_functions -->|"helpers"| node_setup
node_conn -->|"db access"| node_setup
node_conn -->|"db access"| node_main
node_schema -->|"contract"| node_conn
node_parse_affiliated -->|"tabular output"| node_tabular_schools
node_parse_affiliated -->|"serialized output"| node_schools_data
node_parse_courses -->|"tabular output"| node_tabular_courses
node_parse_courses -->|"serialized output"| node_courses_data
node_setup -->|"generates"| node_schools_data
node_setup -->|"generates"| node_courses_data
node_schools_data -->|"snapshot"| node_backup_store
node_courses_data -->|"snapshot"| node_backup_store
node_manual_fix -.->|"correction"| node_tabular_schools
node_main -->|"orchestrates"| node_setup
node_main -->|"orchestrates"| node_parse_affiliated
node_main -->|"orchestrates"| node_parse_courses

click node_main "https://github.com/bulktornado/cbse-rdms-project/blob/main/main.py"
click node_parse_affiliated "https://github.com/bulktornado/cbse-rdms-project/blob/main/src/parse_affiliated_school_data.py"
click node_parse_courses "https://github.com/bulktornado/cbse-rdms-project/blob/main/src/parse_courses.py"
click node_setup "https://github.com/bulktornado/cbse-rdms-project/blob/main/src/setup.py"
click node_config "https://github.com/bulktornado/cbse-rdms-project/blob/main/src/set_configuration.py"
click node_manual_fix "https://github.com/bulktornado/cbse-rdms-project/blob/main/src/forbidden_scripts/manually_update_affiliated_schools_from_tabular_data.py"
click node_conn "https://github.com/bulktornado/cbse-rdms-project/blob/main/modules/connection_module.py"
click node_functions "https://github.com/bulktornado/cbse-rdms-project/blob/main/modules/functions.py"
click node_schema "https://github.com/bulktornado/cbse-rdms-project/blob/main/schemas/database_schema.sql"
click node_schools_data "https://github.com/bulktornado/cbse-rdms-project/blob/main/data/affiliated_schools.dat"
click node_courses_data "https://github.com/bulktornado/cbse-rdms-project/blob/main/data/courses.dat"
click node_tabular_schools "https://github.com/bulktornado/cbse-rdms-project/blob/main/data/tabular_data/affiliated_schools.txt"
click node_tabular_courses "https://github.com/bulktornado/cbse-rdms-project/blob/main/data/tabular_data/courses.txt"
click node_backup_store "https://github.com/bulktornado/cbse-rdms-project/blob/main/backup/affiliated_schools.dat"
click node_docs "https://github.com/bulktornado/cbse-rdms-project/blob/main/docs/Curriculum_Secondary_2025-26.pdf"
click node_raw_data "https://github.com/bulktornado/cbse-rdms-project/tree/main/data/raw_datasets/affiliated_schools_raw_data"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_docs,node_raw_data toneBlue
class node_parse_affiliated,node_parse_courses,node_setup,node_config,node_manual_fix toneAmber
class node_conn,node_functions toneMint
class node_schema,node_schools_data,node_courses_data,node_tabular_schools,node_tabular_courses,node_backup_store toneRose
class node_main toneIndigo
```

## Components

### Source inputs

Human/reference material and raw extracted datasets that feed the preparation pipeline.

- [Docs](https://github.com/bulktornado/cbse-rdms-project/blob/main/docs/Curriculum_Secondary_2025-26.pdf): reference-inputs · Curriculum PDFs and project notes that define the source domain. · `docs/Curriculum_Secondary_2025-26.pdf`
- [Raw data](https://github.com/bulktornado/cbse-rdms-project/tree/main/data/raw_datasets/affiliated_schools_raw_data): source-datasets · Extracted raw and intermediate datasets for setup and parsing. · `data/raw_datasets/affiliated_schools_raw_data`

### Processing layer

Scripts that parse, normalize, configure, and initialize project data.

- [Parse schools](https://github.com/bulktornado/cbse-rdms-project/blob/main/src/parse_affiliated_school_data.py): parser · Transforms raw affiliated-school data into structured records. · `src/parse_affiliated_school_data.py`
- [Parse courses](https://github.com/bulktornado/cbse-rdms-project/blob/main/src/parse_courses.py): parser · Transforms curriculum/course sources into normalized course data. · `src/parse_courses.py`
- [Setup](https://github.com/bulktornado/cbse-rdms-project/blob/main/src/setup.py): initializer · Prepares the project environment and data generation workflow. · `src/setup.py`
- [Config](https://github.com/bulktornado/cbse-rdms-project/blob/main/src/set_configuration.py): configuration · Centralizes path and runtime configuration. · `src/set_configuration.py`
- [Manual fix](https://github.com/bulktornado/cbse-rdms-project/blob/main/src/forbidden_scripts/manually_update_affiliated_schools_from_tabular_data.py): maintenance-script · Exceptional script for one-off correction of affiliated-school data. · `src/forbidden_scripts/manually_update_affiliated_schools_from_tabular_data.py`

### Shared runtime

Reusable modules for database access and common helper logic.

- [DB connection](https://github.com/bulktornado/cbse-rdms-project/blob/main/modules/connection_module.py): database-access · Shared database connectivity and connection management. · `modules/connection_module.py`
- [Helpers](https://github.com/bulktornado/cbse-rdms-project/blob/main/modules/functions.py): utilities · Reusable helper functions shared across scripts. · `modules/functions.py`

### Storage layer

Generated datasets, backups, and the relational schema contract.

- [Schema](https://github.com/bulktornado/cbse-rdms-project/blob/main/schemas/database_schema.sql): sql-schema · Relational schema defining the database contract. · `schemas/database_schema.sql`
- [Schools data](https://github.com/bulktornado/cbse-rdms-project/blob/main/data/affiliated_schools.dat): generated-dataset · Structured affiliated-school dataset stored for reuse. · `data/affiliated_schools.dat`
- [Courses data](https://github.com/bulktornado/cbse-rdms-project/blob/main/data/courses.dat): generated-dataset · Structured course dataset stored for reuse. · `data/courses.dat`
- [Schools table](https://github.com/bulktornado/cbse-rdms-project/blob/main/data/tabular_data/affiliated_schools.txt): tabular-output · Tabular affiliated-school output used for inspection or reloads. · `data/tabular_data/affiliated_schools.txt`
- [Courses table](https://github.com/bulktornado/cbse-rdms-project/blob/main/data/tabular_data/courses.txt): tabular-output · Tabular course output used for inspection or reloads. · `data/tabular_data/courses.txt`
- [Backups](https://github.com/bulktornado/cbse-rdms-project/blob/main/backup/affiliated_schools.dat): snapshot-store · Backup snapshots for datasets and restored state. · `backup/affiliated_schools.dat`

### Entry point

Top-level script that coordinates project execution.

- [Main](https://github.com/bulktornado/cbse-rdms-project/blob/main/main.py): entrypoint · Top-level program that drives the project workflow. · `main.py`

## Connections

- Docs → Parse courses: curriculum source · Curriculum PDFs and notes inform course extraction.
- Docs → Parse schools: reference source · Project documentation and notes guide school-data preparation.
- Raw data → Parse schools: raw input · Raw affiliated-school material is normalized into structured records.
- Raw data → Parse courses: raw input · Raw reference datasets support course filtering and mapping.
- Config → Setup: paths · Configuration is consumed during initialization.
- Config → Parse schools: settings · Parsing relies on shared configuration values.
- Config → Parse courses: settings · Parsing relies on shared configuration values.
- Helpers → Parse schools: helpers · Reusable helpers support parsing and cleanup.
- Helpers → Parse courses: helpers · Reusable helpers support parsing and cleanup.
- Helpers → Setup: helpers · Setup uses common utility functions.
- DB connection → Setup: db access · Initialization prepares or validates database connectivity.
- DB connection → Main: db access · The entry point coordinates runtime database use.
- Schema → DB connection: contract · Database access targets the defined relational schema.
- Parse schools → Schools table: tabular output · Parsed school data is written to tabular form.
- Parse schools → Schools data: serialized output · Parsed school data is stored as a reusable dataset.
- Parse courses → Courses table: tabular output · Parsed course data is written to tabular form.
- Parse courses → Courses data: serialized output · Parsed course data is stored as a reusable dataset.
- Setup → Schools data: generates · Initialization may generate or refresh school snapshots.
- Setup → Courses data: generates · Initialization may generate or refresh course snapshots.
- Schools data → Backups: snapshot · Generated school data is mirrored in backup artifacts.
- Courses data → Backups: snapshot · Generated course data is mirrored in backup artifacts.
- Manual fix → Schools table: correction · Manual maintenance updates the school tabular dataset.
- Main → Setup: orchestrates · Main coordinates initialization before data use.
- Main → Parse schools: orchestrates · Main can invoke school-data preparation.
- Main → Parse courses: orchestrates · Main can invoke course-data preparation.