# anshuman1823/pca_and_lloyds architecture

> Architecture diagram of the GitHub repository anshuman1823/pca_and_lloyds, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/anshuman1823/pca_and_lloyds
- Repository: https://github.com/anshuman1823/pca_and_lloyds

## Overview

##### Project type and purpose
- This is a standalone Python data-analysis pipeline.
- It implements two core algorithms from scratch: PCA on MNIST and Lloyd’s k-means clustering on 2D data.

##### File structure
- Top-level scripts/modules:
– main_file.py: orchestrator that imports and calls all functions.
– PCA.py: PCA implementation (covariance computation, eigen decomposition, reconstruction).
– Lloyds.py: Lloyd’s k-means algorithm (assignment, update, convergence).
– Helpers.py: plotting and visualization utilities.
– datasets/: raw input data (train.csv, test.csv, MNIST parquet files).
– output_images/: directory for generated charts and reconstructed image grids.
- Configuration: No external build or deployment scripts—pure Python scripts executed locally.
- Artifacts: Report.pdf, output image files.

##### Extract architecture insights from the README
- Data preprocessing: flattening MNIST images to 784-dimensional vectors, subsampling 1000 images (100 per digit).
- Two pipeline branches:
– PCA branch: variance explained → dimensionality selection → reconstruction.
– Clustering branch: Lloyd’s algorithm runs for k={2,3,4,5}, random initializations and Voronoi plots.
- Downstream tasks: classification recommendation, algorithm limitations.

1. Data Sources
- datasets/train.csv, datasets/test.csv
- mnist_parquet/train-00000… .parquet, test-00000… .parquet
2. Data Loader & Preprocessor (in main_file.py + Helpers.py)
- Reads CSV/parquet, flattens images, subsamples data
3. PCA Module (PCA.py)
- Covariance matrix → eigenvalues/vectors → dimensionality reduction → image reconstruction
4. Clustering Module (Lloyds.py)
- Lloyd’s algorithm loop → cluster assignments → center updates → convergence
5. Visualization & Reporting (Helpers.py + main_file.py)
6. Outputs
- output_images/ (PNGs of each visualization)
- Report.pdf (final summary)

– Data Sources → Data Loader & Preprocessor
– Preprocessor → PCA Module
– Preprocessor → Clustering Module
– PCA Module → Visualization & Reporting
– Clustering Module → Visualization & Reporting
– Visualization & Reporting → Outputs (output_images folder + PDF)

c. Architectural patterns & design principles
- Modularity: each algorithm lives in its own module (separation of concerns).
- Reusability: Helpers.py contains common plotting functions used by both PCA and clustering branches.
- Single entrypoint: main_file.py orchestrates the end-to-end flow.

d. Key technologies & libraries
- Python 3.x
- NumPy (covariance, eigen decomposition, vector operations)
- Pandas (CSV/parquet I/O, DataFrame handling)
- Matplotlib / Seaborn (visualizations)
- Pyarrow or fastparquet (for parquet reading, if used)
- Standard library (os, argparse, etc.)

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph "Data Sources"
        DS1["train.csv"]:::data
        DS2["test.csv"]:::data
        DS3["train-00000-of-00001.parquet"]:::data
        DS4["test-00000-of-00001.parquet"]:::data
    end

    DS1 -->|"load data"| main
    DS2 -->|"load data"| main
    DS3 -->|"load data"| main
    DS4 -->|"load data"| main

    main["main_file.py\nOrchestrator"]:::processing

    main -->|"flatten & subsample"| PCA
    main -->|"flatten & subsample"| Lloyds
    main -->|"uses utilities"| utils

    subgraph "Shared Utilities"
        utils["Helpers.py\n(Plotting & Preprocessing)"]:::processing
    end

    subgraph "Algorithm Modules"
        PCA["PCA.py\nPrincipal Components & Reconstruction"]:::processing
        Lloyds["Lloyds.py\nk-means Clustering"]:::processing
    end

    PCA -->|"eigenvectors & reconstructed images"| viz
    Lloyds -->|"cluster labels & Voronoi data"| viz

    viz["Visualization & Reporting"]:::viz

    viz -->|"save PNGs"| out1
    viz -->|"compile PDF report"| out2

    subgraph "Outputs"
        out1["output_images/"]:::data
        out2["Report.pdf"]:::data
    end

    click DS1 "https://github.com/anshuman1823/pca_and_lloyds/blob/main/datasets/train.csv"
    click DS2 "https://github.com/anshuman1823/pca_and_lloyds/blob/main/datasets/test.csv"
    click DS3 "https://github.com/anshuman1823/pca_and_lloyds/blob/main/mnist_parquet/train-00000-of-00001.parquet"
    click DS4 "https://github.com/anshuman1823/pca_and_lloyds/blob/main/mnist_parquet/test-00000-of-00001.parquet"
    click main "https://github.com/anshuman1823/pca_and_lloyds/blob/main/main_file.py"
    click utils "https://github.com/anshuman1823/pca_and_lloyds/blob/main/Helpers.py"
    click PCA "https://github.com/anshuman1823/pca_and_lloyds/blob/main/PCA.py"
    click Lloyds "https://github.com/anshuman1823/pca_and_lloyds/blob/main/Lloyds.py"
    click out1 "https://github.com/anshuman1823/pca_and_lloyds/tree/main/output_images/"
    click out2 "https://github.com/anshuman1823/pca_and_lloyds/blob/main/Report.pdf"

    classDef data fill:#add8e6,stroke:#000,stroke-width:1px
    classDef processing fill:#90ee90,stroke:#000,stroke-width:1px
    classDef viz fill:#ffa500,stroke:#000,stroke-width:1px
```
