# shawongazi7/spamshield-ai architecture

> Architecture diagram of the GitHub repository shawongazi7/spamshield-ai, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/shawongazi7/spamshield-ai
- Repository: https://github.com/shawongazi7/spamshield-ai

## Overview

1. This repository is a notebook-driven machine learning project, not a deployed application. The core “system” lives in two Jupyter notebooks: one complete exploratory/educational version and one cleaned implementation version. All model training, evaluation, and interactive prediction happen inside the notebook environment.

2. The primary entry points are:
- `SpamShield_AI_Complete.ipynb`: the full end-to-end analysis, comparison of models, visualization, and the interactive prediction UI.
- `SpamShield_AI_Clean.ipynb`: a streamlined version of the same pipeline, likely meant for clearer reuse or presentation.
There is no separate Python package, service layer, or API layer in this repo.

3. The data layer is file-based and local:
- `spam.csv` is the main training corpus with 5,572 labeled emails.
- `spam_ham_test_dataset.csv` is a small external validation set with 79 emails from a different source.
The architecture depends heavily on comparing in-domain training results against out-of-domain test performance.

4. The processing pipeline is classic NLP classification:
raw email text is cleaned/preprocessed, then transformed into numeric features using TF-IDF. The README indicates unigram + bigram support and a cap of about 5,000 features. This feature-extraction step is the main boundary between text input and model-ready vectors.

5. Class imbalance is a first-class architectural concern. The training set is heavily skewed toward ham, so the notebooks evaluate balancing strategies such as `class_weight='balanced'` and SMOTE from `imblearn`. In the project narrative, class-weighting appears to be the preferred production-like choice over synthetic oversampling.

6. The modeling subsystem compares four supervised classifiers on the same feature matrix:
- Logistic Regression
- Naive Bayes
- Linear SVM
- Random Forest
This is a model-selection architecture, not a single-model pipeline; the notebooks are designed to benchmark speed, accuracy, F1, and generalization across candidates.

7. Evaluation is multi-stage. The notebooks use training metrics, cross-validation, and an external holdout dataset. That external dataset is architecturally important because it reveals the gap between internal performance and real-world generalization, and it drives the final model choice.

8. The project’s decision boundary is based on generalization, not just training scores. Logistic Regression emerges as the best overall model because it performs best on unseen data, even though several models show similarly high training metrics. That makes the evaluation loop a central part of the architecture rather than a reporting afterthought.

9. There is an interactive prediction layer inside the notebook using `ipywidgets`. This functions like a lightweight UI for entering email text and receiving a spam/ham prediction, but it is notebook-local rather than a deployed frontend.

10. The technology stack is standard Python ML tooling:
- `pandas` and `numpy` for data handling
- `scikit-learn` for vectorization, model training, validation, and metrics
- `imblearn` for SMOTE experiments
- `matplotlib` and `seaborn` for analysis/visualization
- `ipywidgets` for interactive notebook controls
This stack shapes the architecture into an offline analytics and modeling workflow.

11. The repo also contains documentation artifacts that support architecture understanding:
- `TECHNICAL_DOCUMENTATION.md` for deeper technical notes
- `DEVELOPMENT_LOG.md` for iteration history and decisions
- `LEARNING_JOURNEY.md` for narrative context
These files don’t execute any logic, but they explain why the pipeline and model choices look the way they do.

12. `requirements.txt` defines the execution boundary for reproducing the notebooks. Since the project is notebook-based and depends on scientific Python libraries, environment consistency matters more than application deployment plumbing.

13. The current architecture stops short of production deployment. The README explicitly mentions future API deployment, online learning, and deep learning experiments, but the repo itself is still an experiment-and-validation system centered on local CSV data, notebook execution, and comparative model analysis.

Main entry points: [`SpamShield_AI_Complete.ipynb`](https://github.com/shawongazi7/spamshield-ai/blob/main/SpamShield_AI_Complete.ipynb) (Complete notebook), [`TECHNICAL_DOCUMENTATION.md`](https://github.com/shawongazi7/spamshield-ai/blob/main/TECHNICAL_DOCUMENTATION.md) (Technical docs), [`SpamShield_AI_Clean.ipynb`](https://github.com/shawongazi7/spamshield-ai/blob/main/SpamShield_AI_Clean.ipynb) (Clean notebook), [`spam.csv`](https://github.com/shawongazi7/spamshield-ai/blob/main/spam.csv) (Training data)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_notebooks["Notebook workflow"]
  node_complete_nb["Complete notebook<br/>entry notebook"]
  node_clean_nb["Clean notebook<br/>entry notebook"]
  node_widgets_ui["Prediction UI<br/>notebook widget"]
end

subgraph group_ml["ML pipeline"]
  node_cleaning["Text cleaning<br/>preprocessing"]
  node_tfidf["TF-IDF vectors<br/>feature extraction"]
  node_imbalance["Imbalance handling<br/>sampling strategy"]
  node_logreg(("Logistic regression<br/>classifier"))
  node_nb(("Naive Bayes<br/>classifier"))
  node_svm(("Linear SVM<br/>classifier"))
  node_rf(("Random forest<br/>classifier"))
  node_evaluation["Evaluation loop<br/>metrics and validation"]
end

subgraph group_support["Support files"]
  node_data_train[("Training data<br/>labeled corpus<br/>[spam.csv]")]
  node_data_test[("External test set<br/>holdout corpus")]
  node_docs["Technical docs<br/>documentation"]
  node_log["Dev log<br/>documentation<br/>[DEVELOPMENT_LOG.md]"]
  node_journey["Learning notes<br/>documentation"]
  node_requirements["Requirements<br/>environment spec<br/>[requirements.txt]"]
end

node_complete_nb -->|"runs"| node_cleaning
node_clean_nb -->|"runs"| node_cleaning
node_data_train -->|"feeds"| node_cleaning
node_data_test -->|"feeds"| node_cleaning
node_cleaning -->|"transforms to"| node_tfidf
node_tfidf -->|"optionally balances"| node_imbalance
node_imbalance -->|"trains"| node_logreg
node_imbalance -->|"trains"| node_nb
node_imbalance -->|"trains"| node_svm
node_imbalance -->|"trains"| node_rf
node_tfidf -->|"features for"| node_logreg
node_tfidf -->|"features for"| node_nb
node_tfidf -->|"features for"| node_svm
node_tfidf -->|"features for"| node_rf
node_logreg -->|"measured by"| node_evaluation
node_nb -->|"measured by"| node_evaluation
node_svm -->|"measured by"| node_evaluation
node_rf -->|"measured by"| node_evaluation
node_evaluation -->|"reported in"| node_complete_nb
node_evaluation -->|"reported in"| node_clean_nb
node_evaluation -->|"powers"| node_widgets_ui
node_widgets_ui -->|"embedded in"| node_complete_nb
node_docs -.->|"explains"| node_complete_nb
node_log -.->|"supports"| node_docs
node_journey -.->|"supports"| node_docs
node_requirements -->|"reproduces"| node_complete_nb

click node_complete_nb "https://github.com/shawongazi7/spamshield-ai/blob/main/SpamShield_AI_Complete.ipynb"
click node_clean_nb "https://github.com/shawongazi7/spamshield-ai/blob/main/SpamShield_AI_Clean.ipynb"
click node_data_train "https://github.com/shawongazi7/spamshield-ai/blob/main/spam.csv"
click node_data_test "https://github.com/shawongazi7/spamshield-ai/blob/main/spam_ham_test_dataset.csv"
click node_docs "https://github.com/shawongazi7/spamshield-ai/blob/main/TECHNICAL_DOCUMENTATION.md"
click node_log "https://github.com/shawongazi7/spamshield-ai/blob/main/DEVELOPMENT_LOG.md"
click node_journey "https://github.com/shawongazi7/spamshield-ai/blob/main/LEARNING_JOURNEY.md"
click node_requirements "https://github.com/shawongazi7/spamshield-ai/blob/main/requirements.txt"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_complete_nb,node_clean_nb,node_widgets_ui toneBlue
class node_cleaning,node_tfidf,node_imbalance,node_logreg,node_nb,node_svm,node_rf,node_evaluation toneAmber
class node_data_train,node_data_test,node_docs,node_log,node_journey,node_requirements toneMint
```

## Components

### Notebook workflow

Interactive analysis and model selection live here.

- [Complete notebook](https://github.com/shawongazi7/spamshield-ai/blob/main/SpamShield_AI_Complete.ipynb): entry notebook · End-to-end exploratory version with model comparison, visualization, and interactive prediction. · `SpamShield_AI_Complete.ipynb`
- [Clean notebook](https://github.com/shawongazi7/spamshield-ai/blob/main/SpamShield_AI_Clean.ipynb): entry notebook · Streamlined implementation-oriented version of the spam classifier workflow. · `SpamShield_AI_Clean.ipynb`
- **Prediction UI**: notebook widget · ipywidgets-based interactive text entry and prediction inside the notebook.

### ML pipeline

Text processing, feature extraction, modeling, and evaluation.

- **Text cleaning**: preprocessing · Raw message normalization and preparation before vectorization.
- **TF-IDF vectors**: feature extraction · Unigram/bigram TF-IDF representation with feature cap for model input.
- **Imbalance handling**: sampling strategy · Class weighting and SMOTE experiments for skewed label distribution.
- **Logistic regression**: classifier · Preferred model due to best generalization on unseen data.
- **Naive Bayes**: classifier · Baseline probabilistic classifier compared on the same feature matrix.
- **Linear SVM**: classifier · Margin-based linear classifier included in the benchmark set.
- **Random forest**: classifier · Tree ensemble baseline used for comparison against linear models.
- **Evaluation loop**: metrics and validation · Training metrics, cross-validation, and external holdout evaluation.

### Support files

Data, documentation, and environment definitions.

- [Training data](https://github.com/shawongazi7/spamshield-ai/blob/main/spam.csv): labeled corpus · Primary spam/ham dataset used to train and compare classifiers. · `spam.csv`
- [External test set](https://github.com/shawongazi7/spamshield-ai/blob/main/spam_ham_test_dataset.csv): holdout corpus · Smaller out-of-domain dataset used to probe real-world generalization. · `spam_ham_test_dataset.csv`
- [Technical docs](https://github.com/shawongazi7/spamshield-ai/blob/main/TECHNICAL_DOCUMENTATION.md): documentation · Narrative and technical notes explaining design choices and project evolution. · `TECHNICAL_DOCUMENTATION.md`
- [Dev log](https://github.com/shawongazi7/spamshield-ai/blob/main/DEVELOPMENT_LOG.md): documentation · Iteration history and decisions during development. · `DEVELOPMENT_LOG.md`
- [Learning notes](https://github.com/shawongazi7/spamshield-ai/blob/main/LEARNING_JOURNEY.md): documentation · Contextual write-up of the learning and project process. · `LEARNING_JOURNEY.md`
- [Requirements](https://github.com/shawongazi7/spamshield-ai/blob/main/requirements.txt): environment spec · Pinned Python dependency set for reproducing the notebook workflow. · `requirements.txt`

## Connections

- Complete notebook → Text cleaning: runs · The complete notebook executes the preprocessing workflow.
- Clean notebook → Text cleaning: runs · The cleaned notebook applies the same text preparation stage.
- Training data → Text cleaning: feeds · Training emails are cleaned before feature extraction.
- External test set → Text cleaning: feeds · External evaluation emails are processed through the same text pipeline.
- Text cleaning → TF-IDF vectors: transforms to · Normalized text is converted into numeric TF-IDF features.
- TF-IDF vectors → Imbalance handling: optionally balances · Feature matrix is paired with balancing strategies for class skew.
- Imbalance handling → Logistic regression: trains · Balanced training is evaluated on logistic regression.
- Imbalance handling → Naive Bayes: trains · Balanced training is evaluated on naive Bayes.
- Imbalance handling → Linear SVM: trains · Balanced training is evaluated on linear SVM.
- Imbalance handling → Random forest: trains · Balanced training is evaluated on random forest.
- TF-IDF vectors → Logistic regression: features for · TF-IDF vectors are the shared input to the classifiers.
- TF-IDF vectors → Naive Bayes: features for · TF-IDF vectors are the shared input to the classifiers.
- TF-IDF vectors → Linear SVM: features for · TF-IDF vectors are the shared input to the classifiers.
- TF-IDF vectors → Random forest: features for · TF-IDF vectors are the shared input to the classifiers.
- Logistic regression → Evaluation loop: measured by · Logistic regression is assessed with internal and external validation.
- Naive Bayes → Evaluation loop: measured by · Naive Bayes is assessed with internal and external validation.
- Linear SVM → Evaluation loop: measured by · Linear SVM is assessed with internal and external validation.
- Random forest → Evaluation loop: measured by · Random forest is assessed with internal and external validation.
- Evaluation loop → Complete notebook: reported in · Results are visualized and compared in the full notebook.
- Evaluation loop → Clean notebook: reported in · Selected workflow is summarized in the cleaned notebook.
- Evaluation loop → Prediction UI: powers · The chosen model is exposed through interactive notebook prediction.
- Prediction UI → Complete notebook: embedded in · The UI lives inside the complete notebook environment.
- Technical docs → Complete notebook: explains · Documentation provides technical context for the main notebook.
- Dev log → Technical docs: supports · Development history supplements the technical documentation.
- Learning notes → Technical docs: supports · Learning narrative supplements the technical documentation.
- Requirements → Complete notebook: reproduces · Dependency pinning defines the notebook execution environment.