# janitabose13/personalised_skincare_recommendation_system architecture

> Architecture diagram of the GitHub repository janitabose13/personalised_skincare_recommendation_system, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-04-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/janitabose13/personalised_skincare_recommendation_system
- Repository: https://github.com/janitabose13/personalised_skincare_recommendation_system

## Overview

1. This repository is primarily an offline data pipeline for a personalized skincare recommendation system. The visible codebase is focused on collecting skincare data, cleaning it, and preparing a unified dataset for later recommendation logic.

2. The ingestion layer consists of three scraper scripts in `Code/`: `nykaa_full_scraper.py`, `incidecoder_scraper.py`, and `ewg_scraper.py`. These pull data from Nykaa, INCIDecoder, and EWG respectively, producing raw CSV extracts in `data/raw/`.

3. `data/raw/` is the repository’s source-data boundary. It contains the original datasets for:
   - `nykaa_products.csv`
   - `nykaa_reviews.csv`
   - `incidecoder_ingredients.csv`
   - `ewg_ingredients.csv`
   This suggests the system is designed to fuse product catalog data, reviews, ingredient metadata, and ingredient safety information.

4. The first transformation stage appears to be `Code/00_clean_nykaa.py`. Based on the naming and dataset layout, this script likely standardizes and cleans Nykaa product/review data before any cross-source integration happens.

5. The next stage is `Code/01_data_preparation.py`, which likely performs feature engineering and dataset merging across the three sources. This is the main preparation boundary where raw product, ingredient, and safety data become a model-ready dataset.

6. `data/processed/` is the curated output layer. It is meant to hold cleaned, transformed, and possibly joined datasets that downstream recommendation or analysis code can consume without needing to touch the raw scrapes.

7. The central domain model is ingredient-centric skincare analysis. The repository is set up to support recommendations based on product attributes, ingredient composition, ingredient safety, and user-review signals rather than only product popularity.

8. The architecture is batch-oriented rather than service-oriented. There is no visible API, UI, or database layer in the file tree; execution appears to be driven by standalone Python scripts run in sequence to populate the data folders.

9. Python is the main runtime, with data-processing dependencies implied by the README such as pandas and NumPy, and likely scikit-learn for later recommendation work. Those libraries matter architecturally because they indicate a tabular-ML pipeline rather than a real-time transactional system.

10. The key external boundaries are the three source websites and the local filesystem. Data flows one way: scrape external sources → land in `data/raw/` → clean and prepare in `Code/` → write curated artifacts to `data/processed/` for future modeling or recommendation generation.

11. From an architecture-diagram perspective, the important subsystems are: source scrapers, cleaning/preparation scripts, raw data storage, processed data storage, and the future recommendation engine implied by the README but not present in the visible tree.

Main entry points: [`Code/01_data_preparation.py`](https://github.com/janitabose13/personalised_skincare_recommendation_system/blob/main/Code/01_data_preparation.py) (Prepare data), [`Code/nykaa_full_scraper.py`](https://github.com/janitabose13/personalised_skincare_recommendation_system/blob/main/Code/nykaa_full_scraper.py) (Nykaa scraper), [`Code/00_clean_nykaa.py`](https://github.com/janitabose13/personalised_skincare_recommendation_system/blob/main/Code/00_clean_nykaa.py) (Clean Nykaa), [`Code/incidecoder_scraper.py`](https://github.com/janitabose13/personalised_skincare_recommendation_system/blob/main/Code/incidecoder_scraper.py) (INCIDecoder scraper)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_grp_code["Pipeline code"]
  node_n_nykaa_scraper["Nykaa scraper"]
  node_n_incidecoder_scraper["INCIDecoder scraper"]
  node_n_ewg_scraper["EWG scraper<br/>[ewg_scraper.py]"]
  node_n_clean_nykaa["Clean Nykaa<br/>transform<br/>[00_clean_nykaa.py]"]
  node_n_prepare_data["Prepare data<br/>pipeline"]
end

subgraph group_grp_raw["Raw data"]
  node_n_nykaa_products["Nykaa products<br/>raw csv<br/>[nykaa_products.csv]"]
  node_n_nykaa_reviews["Nykaa reviews<br/>raw csv<br/>[nykaa_reviews.csv]"]
  node_n_incidecoder_ingredients["INCIDecoder ingredients<br/>raw csv"]
  node_n_ewg_ingredients["EWG ingredients<br/>raw csv"]
end

subgraph group_grp_processed["Processed data"]
  node_n_processed_boundary[("Curated datasets<br/>processed data")]
end

subgraph group_grp_external["External sources"]
  node_n_nykaa_site(("Nykaa<br/>external site"))
  node_n_incidecoder_site(("INCIDecoder<br/>external site"))
  node_n_ewg_site(("EWG<br/>external site"))
end

subgraph group_grp_future["Downstream modeling"]
  node_n_future_rec_engine{{"Recommender<br/>future model"}}
end

node_n_nykaa_site -->|"scrape"| node_n_nykaa_scraper
node_n_incidecoder_site -->|"scrape"| node_n_incidecoder_scraper
node_n_ewg_site -->|"scrape"| node_n_ewg_scraper
node_n_nykaa_scraper -->|"land"| node_n_nykaa_products
node_n_nykaa_scraper -->|"land"| node_n_nykaa_reviews
node_n_incidecoder_scraper -->|"land"| node_n_incidecoder_ingredients
node_n_ewg_scraper -->|"land"| node_n_ewg_ingredients
node_n_nykaa_products -->|"input"| node_n_clean_nykaa
node_n_nykaa_reviews -->|"input"| node_n_clean_nykaa
node_n_clean_nykaa -->|"cleaned nykaa"| node_n_prepare_data
node_n_incidecoder_ingredients -->|"ingredient data"| node_n_prepare_data
node_n_ewg_ingredients -->|"safety data"| node_n_prepare_data
node_n_prepare_data -->|"write"| node_n_processed_boundary
node_n_processed_boundary -.->|"feeds"| node_n_future_rec_engine

click node_n_nykaa_scraper "https://github.com/janitabose13/personalised_skincare_recommendation_system/blob/main/Code/nykaa_full_scraper.py"
click node_n_incidecoder_scraper "https://github.com/janitabose13/personalised_skincare_recommendation_system/blob/main/Code/incidecoder_scraper.py"
click node_n_ewg_scraper "https://github.com/janitabose13/personalised_skincare_recommendation_system/blob/main/Code/ewg_scraper.py"
click node_n_clean_nykaa "https://github.com/janitabose13/personalised_skincare_recommendation_system/blob/main/Code/00_clean_nykaa.py"
click node_n_prepare_data "https://github.com/janitabose13/personalised_skincare_recommendation_system/blob/main/Code/01_data_preparation.py"
click node_n_nykaa_products "https://github.com/janitabose13/personalised_skincare_recommendation_system/blob/main/data/raw/nykaa_products.csv"
click node_n_nykaa_reviews "https://github.com/janitabose13/personalised_skincare_recommendation_system/blob/main/data/raw/nykaa_reviews.csv"
click node_n_incidecoder_ingredients "https://github.com/janitabose13/personalised_skincare_recommendation_system/blob/main/data/raw/incidecoder_ingredients.csv"
click node_n_ewg_ingredients "https://github.com/janitabose13/personalised_skincare_recommendation_system/blob/main/data/raw/ewg_ingredients.csv"
click node_n_processed_boundary "https://github.com/janitabose13/personalised_skincare_recommendation_system/tree/main/data/processed"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_n_nykaa_scraper,node_n_incidecoder_scraper,node_n_ewg_scraper,node_n_clean_nykaa,node_n_prepare_data toneBlue
class node_n_nykaa_products,node_n_nykaa_reviews,node_n_incidecoder_ingredients,node_n_ewg_ingredients toneAmber
class node_n_processed_boundary toneMint
class node_n_nykaa_site,node_n_incidecoder_site,node_n_ewg_site toneRose
class node_n_future_rec_engine toneIndigo
```

## Components

### Pipeline code

Standalone Python scripts that scrape, clean, and prepare skincare data.

- [Nykaa scraper](https://github.com/janitabose13/personalised_skincare_recommendation_system/blob/main/Code/nykaa_full_scraper.py): scraper · Collects product and review data from Nykaa. · `Code/nykaa_full_scraper.py`
- [INCIDecoder scraper](https://github.com/janitabose13/personalised_skincare_recommendation_system/blob/main/Code/incidecoder_scraper.py): scraper · Extracts ingredient data from INCIDecoder. · `Code/incidecoder_scraper.py`
- [EWG scraper](https://github.com/janitabose13/personalised_skincare_recommendation_system/blob/main/Code/ewg_scraper.py): scraper · Extracts ingredient safety data from EWG. · `Code/ewg_scraper.py`
- [Clean Nykaa](https://github.com/janitabose13/personalised_skincare_recommendation_system/blob/main/Code/00_clean_nykaa.py): transform · Standardizes and cleans scraped Nykaa data before integration. · `Code/00_clean_nykaa.py`
- [Prepare data](https://github.com/janitabose13/personalised_skincare_recommendation_system/blob/main/Code/01_data_preparation.py): pipeline · Merges and engineers features across sources into a unified dataset. · `Code/01_data_preparation.py`

### Raw data

Source-data boundary containing original scraped CSV extracts.

- [Nykaa products](https://github.com/janitabose13/personalised_skincare_recommendation_system/blob/main/data/raw/nykaa_products.csv): raw csv · Raw product catalog extracted from Nykaa. · `data/raw/nykaa_products.csv`
- [Nykaa reviews](https://github.com/janitabose13/personalised_skincare_recommendation_system/blob/main/data/raw/nykaa_reviews.csv): raw csv · Raw customer review data extracted from Nykaa. · `data/raw/nykaa_reviews.csv`
- [INCIDecoder ingredients](https://github.com/janitabose13/personalised_skincare_recommendation_system/blob/main/data/raw/incidecoder_ingredients.csv): raw csv · Raw ingredient metadata extracted from INCIDecoder. · `data/raw/incidecoder_ingredients.csv`
- [EWG ingredients](https://github.com/janitabose13/personalised_skincare_recommendation_system/blob/main/data/raw/ewg_ingredients.csv): raw csv · Raw ingredient safety data extracted from EWG. · `data/raw/ewg_ingredients.csv`

### Processed data

Curated outputs produced by the pipeline for downstream use.

- [Curated datasets](https://github.com/janitabose13/personalised_skincare_recommendation_system/tree/main/data/processed): processed data · Cleaned, transformed, and joined datasets ready for downstream consumption. · `data/processed`

### External sources

Upstream websites scraped by the ingestion layer.

- **Nykaa**: external site · Source website for products and reviews.
- **INCIDecoder**: external site · Source website for ingredient information.
- **EWG**: external site · Source website for ingredient safety information.

### Downstream modeling

Implied recommendation layer referenced by the repository purpose.

- **Recommender**: future model · Implied downstream recommendation logic using the prepared skincare dataset.

## Connections

- Nykaa → Nykaa scraper: scrape · Pull product and review data.
- INCIDecoder → INCIDecoder scraper: scrape · Pull ingredient metadata.
- EWG → EWG scraper: scrape · Pull ingredient safety data.
- Nykaa scraper → Nykaa products: land · Write raw product extracts.
- Nykaa scraper → Nykaa reviews: land · Write raw review extracts.
- INCIDecoder scraper → INCIDecoder ingredients: land · Write raw ingredient extracts.
- EWG scraper → EWG ingredients: land · Write raw safety extracts.
- Nykaa products → Clean Nykaa: input · Clean scraped product catalog data.
- Nykaa reviews → Clean Nykaa: input · Clean scraped review data.
- Clean Nykaa → Prepare data: cleaned nykaa · Provide standardized Nykaa tables to the integration stage.
- INCIDecoder ingredients → Prepare data: ingredient data · Provide ingredient metadata for joining and feature creation.
- EWG ingredients → Prepare data: safety data · Provide safety annotations for joining and feature creation.
- Prepare data → Curated datasets: write · Persist curated datasets for downstream consumption.
- Curated datasets → Recommender: feeds · Supply model-ready data to the implied recommendation layer.
