# otniel113/lungcanceridentification architecture

> Architecture diagram of the GitHub repository otniel113/lungcanceridentification, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/otniel113/lungcanceridentification
- Repository: https://github.com/otniel113/lungcanceridentification

## Overview

##### Project Type and Purpose
- This is a data-science/ML pipeline for “Identification of Lung Cancer in Smoker Person Using Ensemble Methods Based on Gene Expression Data.”
- It is not a web or full-stack app; instead it’s a sequence of Jupyter-based ETL, feature-engineering, training, hyperparameter tuning, validation, and inference steps.

##### Main Architectural Components
a. Data Sources
- Raw DNA microarray gene-expression files (not shown in file tree but implied)
- Sample labels/metadata
b. Preprocessing Stage
- Notebook “01_PersiapanData.ipynb”
- Cleans, normalizes, and merges raw data → outputs preprocessed tables
c. Feature Engineering Stage
- Notebook “02_Praproses_Fitur.ipynb”
- Selects and possibly reduces features → writes feature sets to features/*.pkl
d. Hyperparameter Tuning & Model Training
- “03A_HyperTuning_RF.ipynb” (Random Forest)
- “03B_HyperTuning_AB.ipynb” (AdaBoost)
- Each explores parameter grid via scikit-learn, outputs best estimators to models/*.pkl
e. Validation & Inference
- “04A_ValidasiModel_RF.ipynb” & “04B_ValidasiModel_AB.ipynb”
- Load each model, run predictions on test data → writes hasil_prediksi/*.csv
f. Artifacts & Storage
- features/       (feature pickles)
- models/         (trained model pickles)
- hasil_prediksi/ (prediction CSVs)

##### Relationships & Data Flow
- Raw Data → Preprocessing Notebook → Preprocessed Data
- Preprocessed Data → Feature Engineering Notebook → feature‐store (features/*.pkl)
- feature‐store + hyperparameter scripts → trained-model store (models/*.pkl)
- trained models + test data → validation notebooks → prediction results (CSV)

##### Architectural Patterns & Principles
- ETL pipeline: sequential notebook stages that each read from/write to disk.
- Modular separation: each logical step (prep, feature, train, validate) is in its own notebook.
- Artifact versioning: distinct directories for features, models, predictions.
- Reproducibility: all steps are Jupyter notebooks using Python + scikit-learn.

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph "Data Ingestion"
        RawData["Raw DNA microarray files"]:::storage
        Metadata["Sample metadata"]:::storage
        Prep["01_PersiapanData.ipynb\n(Clean, normalize)"]:::process
        click Prep "https://github.com/otniel113/lungcanceridentification/blob/main/01_PersiapanData.ipynb"
        Preprocessed["Preprocessed tables"]:::storage
        RawData --> Prep
        Metadata --> Prep
        Prep -->|"CSV/TSV"| Preprocessed
    end

    subgraph "Feature Preparation"
        FE["02_Praproses_Fitur.ipynb\n(Feature selection)"]:::process
        click FE "https://github.com/otniel113/lungcanceridentification/blob/main/02_Praproses_Fitur.ipynb"
        Features["features/\n(pickled .pkl)"]:::storage
        click Features "https://github.com/otniel113/lungcanceridentification/tree/main/features/"
        Preprocessed --> FE
        FE -->|"features/*.pkl"| Features
    end

    subgraph "Training & Tuning"
        RF["03A_HyperTuning_RF.ipynb\n(RandomForestClassifier)"]:::process
        click RF "https://github.com/otniel113/lungcanceridentification/blob/main/03A_HyperTuning_RF.ipynb"
        AB["03B_HyperTuning_AB.ipynb\n(AdaBoostClassifier)"]:::process
        click AB "https://github.com/otniel113/lungcanceridentification/blob/main/03B_HyperTuning_AB.ipynb"
        Models["models/\n(best-estimator.pkl)"]:::storage
        click Models "https://github.com/otniel113/lungcanceridentification/tree/main/models/"
        Features --> RF
        Features --> AB
        RF -->|"models/*.pkl"| Models
        AB -->|"models/*.pkl"| Models
    end

    subgraph "Inference & Validation"
        ValRF["04A_ValidasiModel_RF.ipynb\n(Predict RF)"]:::process
        click ValRF "https://github.com/otniel113/lungcanceridentification/blob/main/04A_ValidasiModel_RF.ipynb"
        ValAB["04B_ValidasiModel_AB.ipynb\n(Predict AB)"]:::process
        click ValAB "https://github.com/otniel113/lungcanceridentification/blob/main/04B_ValidasiModel_AB.ipynb"
        Results["hasil_prediksi/\n(.csv outputs)"]:::storage
        click Results "https://github.com/otniel113/lungcanceridentification/tree/main/hasil_prediksi/"
        Models --> ValRF
        Models --> ValAB
        ValRF -->|"csv"| Results
        ValAB -->|"csv"| Results
    end

    subgraph "Documentation"
        README["README.md"]:::doc
        click README "https://github.com/otniel113/lungcanceridentification/blob/main/README.md"
    end

    classDef storage fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef process fill:#90EE90,stroke:#333,stroke-width:1px
    classDef doc fill:#FFDAB9,stroke:#333,stroke-width:1px
```
