# shravanshidruk16/te-dsbda-72311412k-shravan-shidruk architecture

> Architecture diagram of the GitHub repository shravanshidruk16/te-dsbda-72311412k-shravan-shidruk, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk
- Repository: https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk

## Overview

This repository is best understood as an educational monorepo, not a single application. It contains many independent lab exercises, each with its own notebook/script, inputs, and often a small write-up. There is very little shared code across folders, so the main architectural boundary is the experiment directory itself.

The top-level structure splits into three broad areas:
- **DSBDA laboratory experiments** (`Experiment_1` through `Experiment_13`)
- **AI and cloud computing practicals** under `AI_CC_PRACTICALS`
- **Mini projects**, especially the loan approval system and other notebook-based projects

For an architecture diagram, each experiment should be shown as a self-contained pipeline: **dataset or input file → notebook/script → output artifacts/plots/metrics**. Most of the Group A experiments follow this pattern in Jupyter notebooks, with CSV inputs such as Titanic, Iris, Housing, Netflix, Social Network Ads, and Students Performance datasets.

The **Group A notebooks** are focused on data science workflows:
- data loading and preprocessing
- missing value and outlier handling
- descriptive statistics
- regression and classification
- text analysis with TF/IDF
- data visualization with seaborn and matplotlib
These notebooks are interactive analysis units rather than reusable services.

The **Group B experiments** introduce distributed/data-processing systems:
- **Experiment 11** is a Hadoop MapReduce log analysis job using `LogMapper`, `LogReducer`, and a driver/processor class, with input log files and generated output partitions
- **Experiment 12** is another MapReduce pipeline for weather aggregation using `WeatherMapper`, `WeatherReducer`, and a driver, with sample weather input and reducer output
- **Experiment 13** is a Scala Spark word-count style program with a text input file
These should be diagrammed as batch-processing jobs with explicit mapper/reducer or Spark execution boundaries.

The **mini project: Loan Approval System** is the most complete application-like subsystem in the repo. It includes:
- `app.py`, which is likely the runtime entry point
- serialized ML artifacts in `models/` such as logistic regression, random forest, scaler, and label encoders
- `model_metrics.json` for evaluation results
- `assets/` for generated EDA outputs and Streamlit app resources
This looks like an inference-oriented ML app: user input or form data flows into preprocessing, then into a trained model, then to a prediction/result displayed by the app.

The loan project’s architecture is important because it separates **training artifacts** from **runtime inference**. The notebooks likely produced the models and charts, while `app.py` consumes the saved encoders/scaler/model files without retraining. That is the key boundary for diagramming the system.

The **AI practicals** folder contains classic AI algorithm demos as standalone Python scripts:
- BFS/DFS
- A* game search
- Dijkstra shortest path
- Kruskal’s MST
- selection sort
- N-Queen CSP
- chatbot
- hospital expert system
These are algorithm-centric scripts, each independent and likely console-based, so they should be represented as isolated examples rather than a single connected pipeline.

The **cloud computing practicals** area is minimal in the tree, but the presence of a PDF and README suggests it is documentation-driven rather than code-heavy. It appears to serve as course material or lab reference rather than a deployed service.

Across the repository, the dominant runtime technologies are:
- **Python/Jupyter Notebook** for analytics, visualization, and ML experiments
- **Java/Hadoop MapReduce** for batch distributed processing
- **Scala/Apache Spark** for distributed data processing
- **Streamlit-style Python app** for the loan approval mini project

Data flows are almost always local and file-based. Inputs are stored as CSV, TXT, or notebook-managed datasets; outputs are notebooks, plots, metrics, model files, or Hadoop output directories. There is no evidence of databases, APIs, message queues, or persistent backend services.

In practical diagram terms, the repo should be shown as a collection of independent learning modules with three major execution styles:
1. **Interactive analytics notebooks**
2. **Batch/distributed data-processing jobs**
3. **ML application with saved model artifacts and a UI layer**

The clearest architectural message is that this repository is a coursework portfolio: it demonstrates a progression from preprocessing and statistics, to ML and visualization, to distributed systems, to small applied ML deployment.

Main entry points: [`README.md`](https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/blob/main/README.md) (Repo overview), [`Experiment_7/main.ipynb`](https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/blob/main/Experiment_7/main.ipynb) (Exp 7-10 notebooks), [`Experiment_3/main.ipynb`](https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/blob/main/Experiment_3/main.ipynb) (Exp 3 notebook), [`MINI PROJECT - LOAN APPROVAL SYSTEM/app.py`](https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/blob/HEAD/MINI%20PROJECT%20-%20LOAN%20APPROVAL%20SYSTEM/app.py) (Loan app)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_g1["Interactive notebooks"]
  node_n2["Netflix data<br/>dataset<br/>[netflix_titles.csv]"]
  node_n3["Exp 1 notebook<br/>analysis notebook<br/>[main.ipynb]"]
  node_n4["Students data<br/>dataset"]
  node_n5["Exp 2 notebook<br/>analysis notebook<br/>[main.ipynb]"]
  node_n6["Iris data<br/>dataset<br/>[iris_data.csv]"]
  node_n7["Salary data<br/>dataset<br/>[salary_data.csv]"]
  node_n8["Exp 3 notebook<br/>analysis notebook<br/>[main.ipynb]"]
  node_n9["Housing data<br/>dataset<br/>[HousingData.csv]"]
  node_n10["Exp 4 notebook<br/>analysis notebook<br/>[main.ipynb]"]
  node_n11["Ads data<br/>dataset"]
  node_n12["Exp 5 notebook<br/>analysis notebook<br/>[main.ipynb]"]
  node_n13["Exp 6 notebook<br/>analysis notebook<br/>[main.ipynb]"]
  node_n14["Exp 7-10 notebooks<br/>analysis notebooks<br/>[main.ipynb]"]
end

subgraph group_g2["Batch jobs"]
  node_n15["Log analysis<br/>hadoop mapreduce job<br/>[LogProcessor.java]"]
  node_n16["Weather aggregation<br/>hadoop mapreduce job<br/>[WeatherDriver.java]"]
  node_n17["Spark word count<br/>spark job<br/>[WordCount.scala]"]
end

subgraph group_g3["ML mini projects"]
  node_n18["Loan app<br/>ml inference app<br/>[app.py]"]
  node_n19[("Loan models<br/>trained artifacts")]
  node_n20["Loan visuals<br/>eda assets"]
  node_n21["Tweet classifier<br/>notebook project<br/>[main.ipynb]"]
  node_n22["Movie recommender<br/>notebook project<br/>[main.ipynb]"]
end

subgraph group_g4["AI practicals"]
  node_n23["AI scripts<br/>algorithm demos"]
end

subgraph group_g5["Course docs"]
  node_n1["Repo overview<br/>repo root<br/>[README.md]"]
  node_n24["Cloud docs<br/>practical notes"]
end

node_n1 -->|"includes"| node_n3
node_n1 -->|"includes"| node_n5
node_n1 -->|"includes"| node_n8
node_n1 -->|"includes"| node_n10
node_n1 -->|"includes"| node_n12
node_n1 -->|"includes"| node_n13
node_n1 -->|"includes"| node_n14
node_n1 -->|"includes"| node_n15
node_n1 -->|"includes"| node_n16
node_n1 -->|"includes"| node_n17
node_n1 -->|"includes"| node_n18
node_n1 -->|"includes"| node_n21
node_n1 -->|"includes"| node_n22
node_n1 -->|"includes"| node_n23
node_n1 -->|"includes"| node_n24
node_n2 -->|"input"| node_n3
node_n4 -->|"input"| node_n5
node_n6 -->|"input"| node_n8
node_n7 -->|"input"| node_n8
node_n9 -->|"input"| node_n10
node_n11 -->|"input"| node_n12
node_n3 -.->|"pattern"| node_n14
node_n5 -.->|"pattern"| node_n14
node_n8 -.->|"pattern"| node_n14
node_n10 -.->|"pattern"| node_n14
node_n12 -.->|"pattern"| node_n14
node_n13 -.->|"pattern"| node_n14
node_n15 -->|"pipeline"| node_n15
node_n16 -->|"pipeline"| node_n16
node_n17 -->|"pipeline"| node_n17
node_n19 -->|"served by"| node_n18
node_n20 -.->|"assets for"| node_n18
node_n18 -->|"uses"| node_n19

click node_n1 "https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/blob/main/README.md"
click node_n2 "https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/blob/main/Experiment_1/netflix_titles.csv"
click node_n3 "https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/blob/main/Experiment_1/main.ipynb"
click node_n4 "https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/blob/main/Experiment_2/StudentsPerformance.csv"
click node_n5 "https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/blob/main/Experiment_2/main.ipynb"
click node_n6 "https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/blob/main/Experiment_3/iris_data.csv"
click node_n7 "https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/blob/main/Experiment_3/salary_data.csv"
click node_n8 "https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/blob/main/Experiment_3/main.ipynb"
click node_n9 "https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/blob/main/Experiment_4/HousingData.csv"
click node_n10 "https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/blob/main/Experiment_4/main.ipynb"
click node_n11 "https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/blob/main/Experiment_5/Social_Network_Ads.csv"
click node_n12 "https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/blob/main/Experiment_5/main.ipynb"
click node_n13 "https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/blob/main/Experiment_6/main.ipynb"
click node_n14 "https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/blob/main/Experiment_7/main.ipynb"
click node_n15 "https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/blob/main/Experiment_11/EXP_11_UBUNTU/DSBDA_11/LogProcessor.java"
click node_n16 "https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/blob/main/Experiment_12/EXP_12_UBUNTU/DSBDA_12/WeatherDriver.java"
click node_n17 "https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/blob/main/Experiment_13/WordCount.scala"
click node_n18 "https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/blob/main/MINI PROJECT - LOAN APPROVAL SYSTEM/app.py"
click node_n19 "https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/tree/main/MINI PROJECT - LOAN APPROVAL SYSTEM/models"
click node_n20 "https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/tree/main/MINI PROJECT - LOAN APPROVAL SYSTEM/assets"
click node_n21 "https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/blob/main/MINI PROJECT - CLASSIFY TWEETS/main.ipynb"
click node_n22 "https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/blob/main/MINI PROJECT - MOVIE RECOMMENDATION/main.ipynb"
click node_n23 "https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/tree/main/AI_CC_PRACTICALS/AI_PRACTICALS"
click node_n24 "https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/tree/main/AI_CC_PRACTICALS/CC_PRACTICALS"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_n2,node_n3,node_n4,node_n5,node_n6,node_n7,node_n8,node_n9,node_n10,node_n11,node_n12,node_n13,node_n14 toneBlue
class node_n15,node_n16,node_n17 toneAmber
class node_n18,node_n19,node_n20,node_n21,node_n22 toneMint
class node_n23 toneRose
class node_n1,node_n24 toneIndigo
```

## Components

### Interactive notebooks

DSBDA lab notebooks that read local datasets and produce analysis artifacts

- [Netflix data](https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/blob/main/Experiment_1/netflix_titles.csv): dataset · Input data for the first data-analysis notebook · `Experiment_1/netflix_titles.csv`
- [Exp 1 notebook](https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/blob/main/Experiment_1/main.ipynb): analysis notebook · Interactive EDA and visualization workflow over Netflix titles · `Experiment_1/main.ipynb`
- [Students data](https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/blob/main/Experiment_2/StudentsPerformance.csv): dataset · Tabular input for descriptive statistics and preprocessing exercises · `Experiment_2/StudentsPerformance.csv`
- [Exp 2 notebook](https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/blob/main/Experiment_2/main.ipynb): analysis notebook · Notebook for data cleaning, statistics, and exploratory analysis · `Experiment_2/main.ipynb`
- [Iris data](https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/blob/main/Experiment_3/iris_data.csv): dataset · Shared small dataset used by multiple classification and plotting labs · `Experiment_3/iris_data.csv`
- [Salary data](https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/blob/main/Experiment_3/salary_data.csv): dataset · Regression input used in the paired notebook exercise · `Experiment_3/salary_data.csv`
- [Exp 3 notebook](https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/blob/main/Experiment_3/main.ipynb): analysis notebook · Notebook for regression and classification style analysis · `Experiment_3/main.ipynb`
- [Housing data](https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/blob/main/Experiment_4/HousingData.csv): dataset · Input for regression and data-preprocessing exercises · `Experiment_4/HousingData.csv`
- [Exp 4 notebook](https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/blob/main/Experiment_4/main.ipynb): analysis notebook · Notebook centered on preprocessing, statistics, and modeling · `Experiment_4/main.ipynb`
- [Ads data](https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/blob/main/Experiment_5/Social_Network_Ads.csv): dataset · Binary classification input for social network ad analysis · `Experiment_5/Social_Network_Ads.csv`
- [Exp 5 notebook](https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/blob/main/Experiment_5/main.ipynb): analysis notebook · Notebook for classification and visualization workflows · `Experiment_5/main.ipynb`
- [Exp 6 notebook](https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/blob/main/Experiment_6/main.ipynb): analysis notebook · Notebook-based lab using iris data for data science practice · `Experiment_6/main.ipynb`
- [Exp 7-10 notebooks](https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/blob/main/Experiment_7/main.ipynb): analysis notebooks · Later notebook labs for visualization, text analysis, and related data science tasks · `Experiment_7/main.ipynb`

### Batch jobs

Hadoop and Spark practicals with mapper/reducer or text-processing boundaries

- [Log analysis](https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/blob/main/Experiment_11/EXP_11_UBUNTU/DSBDA_11/LogProcessor.java): hadoop mapreduce job · Experiment 11 log processing pipeline with mapper, reducer, driver, and generated output · `Experiment_11/EXP_11_UBUNTU/DSBDA_11/LogProcessor.java`
- [Weather aggregation](https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/blob/main/Experiment_12/EXP_12_UBUNTU/DSBDA_12/WeatherDriver.java): hadoop mapreduce job · Experiment 12 weather averaging pipeline with mapper, reducer, driver, and output directory · `Experiment_12/EXP_12_UBUNTU/DSBDA_12/WeatherDriver.java`
- [Spark word count](https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/blob/main/Experiment_13/WordCount.scala): spark job · Experiment 13 Scala text-processing job over a local input file · `Experiment_13/WordCount.scala`

### ML mini projects

Notebook-driven ML projects and the deployable loan approval app

- [Loan app](https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/blob/HEAD/MINI%20PROJECT%20-%20LOAN%20APPROVAL%20SYSTEM/app.py): ml inference app · Streamlit-style entry point that serves predictions from saved training artifacts · `MINI PROJECT - LOAN APPROVAL SYSTEM/app.py`
- [Loan models](https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/tree/HEAD/MINI%20PROJECT%20-%20LOAN%20APPROVAL%20SYSTEM/models): trained artifacts · Serialized preprocessing objects, classifiers, and evaluation metrics used at runtime · `MINI PROJECT - LOAN APPROVAL SYSTEM/models`
- [Loan visuals](https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/tree/HEAD/MINI%20PROJECT%20-%20LOAN%20APPROVAL%20SYSTEM/assets): eda assets · Generated analysis outputs and app resources used by the mini project · `MINI PROJECT - LOAN APPROVAL SYSTEM/assets`
- [Tweet classifier](https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/blob/HEAD/MINI%20PROJECT%20-%20CLASSIFY%20TWEETS/main.ipynb): notebook project · Separate mini project notebook for text classification · `MINI PROJECT - CLASSIFY TWEETS/main.ipynb`
- [Movie recommender](https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/blob/HEAD/MINI%20PROJECT%20-%20MOVIE%20RECOMMENDATION/main.ipynb): notebook project · Separate mini project notebook for recommendation workflows · `MINI PROJECT - MOVIE RECOMMENDATION/main.ipynb`

### AI practicals

Standalone algorithm demo scripts and console-style experiments

- [AI scripts](https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/tree/main/AI_CC_PRACTICALS/AI_PRACTICALS): algorithm demos · Standalone search, graph, sorting, CSP, chatbot, and expert-system scripts · `AI_CC_PRACTICALS/AI_PRACTICALS`

### Course docs

Documentation-only practicals and repository overview

- [Repo overview](https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/blob/main/README.md): repo root · Top-level coursework portfolio spanning notebooks, batch jobs, AI demos, and mini projects · `README.md`
- [Cloud docs](https://github.com/shravanshidruk16/te-dsbda-72311412k-shravan-shidruk/tree/main/AI_CC_PRACTICALS/CC_PRACTICALS): practical notes · Documentation-driven cloud computing practicals · `AI_CC_PRACTICALS/CC_PRACTICALS`

## Connections

- Repo overview → Exp 1 notebook: includes · Repository overview points to the first notebook lab
- Repo overview → Exp 2 notebook: includes · Repository overview points to the second notebook lab
- Repo overview → Exp 3 notebook: includes · Repository overview points to the third notebook lab
- Repo overview → Exp 4 notebook: includes · Repository overview points to the fourth notebook lab
- Repo overview → Exp 5 notebook: includes · Repository overview points to the fifth notebook lab
- Repo overview → Exp 6 notebook: includes · Repository overview points to the sixth notebook lab
- Repo overview → Exp 7-10 notebooks: includes · Repository overview points to the later notebook labs
- Repo overview → Log analysis: includes · Repository overview points to distributed batch processing labs
- Repo overview → Weather aggregation: includes · Repository overview points to distributed batch processing labs
- Repo overview → Spark word count: includes · Repository overview points to distributed batch processing labs
- Repo overview → Loan app: includes · Repository overview points to the deployable loan app
- Repo overview → Tweet classifier: includes · Repository overview points to the tweet classifier notebook
- Repo overview → Movie recommender: includes · Repository overview points to the movie recommender notebook
- Repo overview → AI scripts: includes · Repository overview points to algorithm practice scripts
- Repo overview → Cloud docs: includes · Repository overview points to cloud computing notes
- Netflix data → Exp 1 notebook: input · Netflix dataset is consumed by the notebook
- Students data → Exp 2 notebook: input · Students dataset is consumed by the notebook
- Iris data → Exp 3 notebook: input · Iris data feeds the notebook workflow
- Salary data → Exp 3 notebook: input · Salary data supplements the notebook workflow
- Housing data → Exp 4 notebook: input · Housing data feeds the notebook workflow
- Ads data → Exp 5 notebook: input · Ads dataset feeds the notebook workflow
- Exp 1 notebook → Exp 7-10 notebooks: pattern · Notebook labs share the same interactive analysis style
- Exp 2 notebook → Exp 7-10 notebooks: pattern · Notebook labs share the same interactive analysis style
- Exp 3 notebook → Exp 7-10 notebooks: pattern · Notebook labs share the same interactive analysis style
- Exp 4 notebook → Exp 7-10 notebooks: pattern · Notebook labs share the same interactive analysis style
- Exp 5 notebook → Exp 7-10 notebooks: pattern · Notebook labs share the same interactive analysis style
- Exp 6 notebook → Exp 7-10 notebooks: pattern · Notebook labs share the same interactive analysis style
- Log analysis → Log analysis: pipeline · Log input is mapped, reduced, and written to partitioned output
- Weather aggregation → Weather aggregation: pipeline · Weather input is mapped, reduced, and written to partitioned output
- Spark word count → Spark word count: pipeline · Text input is transformed by Spark into word-count style results
- Loan models → Loan app: served by · Saved encoders, scaler, and models are loaded by the app
- Loan visuals → Loan app: assets for · EDA outputs and UI resources support the application
- Loan app → Loan models: uses · Runtime inference depends on persisted artifacts rather than retraining