# artyomturkoff/imdb_sentiment_analysis architecture

> Architecture diagram of the GitHub repository artyomturkoff/imdb_sentiment_analysis, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/artyomturkoff/imdb_sentiment_analysis
- Repository: https://github.com/artyomturkoff/imdb_sentiment_analysis

## Overview

This repository is an IMDb binary sentiment-analysis project. Its main workflow downloads IMDb reviews, selects reproducible train/validation/test subsets, trains one of two classical ML pipelines, evaluates and saves the run, then uses a saved pipeline to predict a review. Separate documented workflows visualize and compare saved metrics and export them to CSV. The training and prediction wiring is supported by sampled source; visualization and comparison are documented capabilities, but their implementation was not sampled.

Main entry points: [`scripts/train_model.py`](https://github.com/artyomturkoff/imdb_sentiment_analysis/blob/main/scripts/train_model.py) (Train model), [`src/data_loader.py`](https://github.com/artyomturkoff/imdb_sentiment_analysis/blob/main/src/data_loader.py) (IMDb data loader), [`results/metrics`](https://github.com/artyomturkoff/imdb_sentiment_analysis/tree/main/results/metrics) (Saved metrics), [`scripts/visualise_model.py`](https://github.com/artyomturkoff/imdb_sentiment_analysis/blob/main/scripts/visualise_model.py) (Visualize run)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Data preparation"]
  node_subset_cmd["Generate subset<br/>[generate_subset.py]"]
  node_loader["IMDb data loader<br/>[data_loader.py]"]
  node_split_store[("Split indices")]
end

subgraph group_modeling["Model training"]
  node_train_cmd["Train model<br/>[train_model.py]"]
  node_preprocess["Text preprocessing<br/>[preprocess.py]"]
  node_features["Model pipelines<br/>[features.py]"]
  node_model_store[("Saved pipelines")]
end

subgraph group_results["Results analysis"]
  node_evaluation["Evaluation metrics<br/>[evaluate.py]"]
  node_metrics_store[("Saved metrics")]
  node_visualize["Visualize run<br/>[visualise_model.py]"]
  node_compare["Compare runs<br/>[compare_results.py]"]
  node_export["Export metrics CSV"]
  node_figures["Result figures"]
  node_csv["Metrics summary CSV"]
end

subgraph group_inference["Review prediction"]
  node_demo["Review prediction demo<br/>[predict_review.py]"]
  node_predict["Prediction service<br/>[predict.py]"]
end

node_user(("Project user"))
node_hf[("Hugging Face IMDb")]

node_user -->|"requests split"| node_subset_cmd
node_subset_cmd -->|"builds subset"| node_loader
node_loader -->|"downloads reviews"| node_hf
node_loader -->|"writes indices"| node_split_store
node_user -->|"starts training"| node_train_cmd
node_train_cmd -->|"loads subset"| node_loader
node_loader -->|"reads indices"| node_split_store
node_train_cmd -->|"creates pipeline"| node_features
node_features -->|"uses preprocessor"| node_preprocess
node_train_cmd -->|"scores model"| node_evaluation
node_train_cmd -->|"saves pipeline"| node_model_store
node_train_cmd -->|"saves metrics"| node_metrics_store
node_user -->|"provides review"| node_demo
node_demo -->|"requests prediction"| node_predict
node_predict -->|"loads pipeline"| node_model_store
node_user -->|"requests plots"| node_visualize
node_visualize -->|"reads run metrics"| node_metrics_store
node_visualize -->|"writes figures"| node_figures
node_user -->|"requests comparison"| node_compare
node_compare -->|"reads saved runs"| node_metrics_store
node_compare -->|"writes comparison plot"| node_figures
node_user -->|"requests export"| node_export
node_export -->|"reads saved metrics"| node_metrics_store
node_export -->|"writes CSV"| node_csv

click node_subset_cmd "https://github.com/artyomturkoff/imdb_sentiment_analysis/blob/main/scripts/generate_subset.py"
click node_loader "https://github.com/artyomturkoff/imdb_sentiment_analysis/blob/main/src/data_loader.py"
click node_train_cmd "https://github.com/artyomturkoff/imdb_sentiment_analysis/blob/main/scripts/train_model.py"
click node_preprocess "https://github.com/artyomturkoff/imdb_sentiment_analysis/blob/main/src/preprocess.py"
click node_features "https://github.com/artyomturkoff/imdb_sentiment_analysis/blob/main/src/features.py"
click node_evaluation "https://github.com/artyomturkoff/imdb_sentiment_analysis/blob/main/src/evaluate.py"
click node_model_store "https://github.com/artyomturkoff/imdb_sentiment_analysis/tree/main/models"
click node_metrics_store "https://github.com/artyomturkoff/imdb_sentiment_analysis/tree/main/results/metrics"
click node_visualize "https://github.com/artyomturkoff/imdb_sentiment_analysis/blob/main/scripts/visualise_model.py"
click node_compare "https://github.com/artyomturkoff/imdb_sentiment_analysis/blob/main/scripts/compare_results.py"
click node_export "https://github.com/artyomturkoff/imdb_sentiment_analysis/blob/main/scripts/export_results_csv.py"
click node_figures "https://github.com/artyomturkoff/imdb_sentiment_analysis/tree/main/results/figures"
click node_csv "https://github.com/artyomturkoff/imdb_sentiment_analysis/blob/main/results/metrics_summary.csv"
click node_demo "https://github.com/artyomturkoff/imdb_sentiment_analysis/blob/main/demo/predict_review.py"
click node_predict "https://github.com/artyomturkoff/imdb_sentiment_analysis/blob/main/src/predict.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_subset_cmd,node_loader,node_split_store,node_user toneBlue
class node_train_cmd,node_preprocess,node_features,node_model_store,node_hf toneAmber
class node_evaluation,node_metrics_store,node_visualize,node_compare,node_export,node_figures,node_csv toneMint
class node_demo,node_predict toneRose
```

## Components

### Data preparation

- [Generate subset](https://github.com/artyomturkoff/imdb_sentiment_analysis/blob/main/scripts/generate_subset.py): `scripts/generate_subset.py`
- [IMDb data loader](https://github.com/artyomturkoff/imdb_sentiment_analysis/blob/main/src/data_loader.py): `src/data_loader.py`
- **Split indices**

### Model training

- [Train model](https://github.com/artyomturkoff/imdb_sentiment_analysis/blob/main/scripts/train_model.py): `scripts/train_model.py`
- [Text preprocessing](https://github.com/artyomturkoff/imdb_sentiment_analysis/blob/main/src/preprocess.py): `src/preprocess.py`
- [Model pipelines](https://github.com/artyomturkoff/imdb_sentiment_analysis/blob/main/src/features.py): `src/features.py`
- [Saved pipelines](https://github.com/artyomturkoff/imdb_sentiment_analysis/tree/main/models): `models`

### Results analysis

- [Evaluation metrics](https://github.com/artyomturkoff/imdb_sentiment_analysis/blob/main/src/evaluate.py): `src/evaluate.py`
- [Saved metrics](https://github.com/artyomturkoff/imdb_sentiment_analysis/tree/main/results/metrics): `results/metrics`
- [Visualize run](https://github.com/artyomturkoff/imdb_sentiment_analysis/blob/main/scripts/visualise_model.py): `scripts/visualise_model.py`
- [Compare runs](https://github.com/artyomturkoff/imdb_sentiment_analysis/blob/main/scripts/compare_results.py): `scripts/compare_results.py`
- [Export metrics CSV](https://github.com/artyomturkoff/imdb_sentiment_analysis/blob/main/scripts/export_results_csv.py): `scripts/export_results_csv.py`
- [Result figures](https://github.com/artyomturkoff/imdb_sentiment_analysis/tree/main/results/figures): `results/figures`
- [Metrics summary CSV](https://github.com/artyomturkoff/imdb_sentiment_analysis/blob/main/results/metrics_summary.csv): `results/metrics_summary.csv`

### Review prediction

- [Review prediction demo](https://github.com/artyomturkoff/imdb_sentiment_analysis/blob/main/demo/predict_review.py): `demo/predict_review.py`
- [Prediction service](https://github.com/artyomturkoff/imdb_sentiment_analysis/blob/main/src/predict.py): `src/predict.py`

### Other components

- **Project user**
- **Hugging Face IMDb**

## Connections

- Project user → Generate subset: requests split
- Generate subset → IMDb data loader: builds subset
- IMDb data loader → Hugging Face IMDb: downloads reviews
- IMDb data loader → Split indices: writes indices
- Project user → Train model: starts training
- Train model → IMDb data loader: loads subset
- IMDb data loader → Split indices: reads indices
- Train model → Model pipelines: creates pipeline
- Model pipelines → Text preprocessing: uses preprocessor
- Train model → Evaluation metrics: scores model
- Train model → Saved pipelines: saves pipeline
- Train model → Saved metrics: saves metrics
- Project user → Review prediction demo: provides review
- Review prediction demo → Prediction service: requests prediction
- Prediction service → Saved pipelines: loads pipeline
- Project user → Visualize run: requests plots
- Visualize run → Saved metrics: reads run metrics
- Visualize run → Result figures: writes figures
- Project user → Compare runs: requests comparison
- Compare runs → Saved metrics: reads saved runs
- Compare runs → Result figures: writes comparison plot
- Project user → Export metrics CSV: requests export
- Export metrics CSV → Saved metrics: reads saved metrics
- Export metrics CSV → Metrics summary CSV: writes CSV