# lengochai97/thesis architecture

> Architecture diagram of the GitHub repository lengochai97/thesis, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-08-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/lengochai97/thesis
- Repository: https://github.com/lengochai97/thesis

## Overview

#### Purpose and Entry Points
This is a thesis-oriented machine-learning workflow organized entirely as Jupyter notebooks. Its apparent goal is to construct news/user interaction features, generate training samples and TFRecords, then run offline and online model stages. Primary entry points are the ordered notebooks under `notebooks/feature_construction/` and `notebooks/models/`.

#### 1. Data Cleaning and Profiling
The pipeline begins by cleaning source data and inspecting resulting distributions/statistics. These notebooks establish the usable dataset boundary before feature generation:
- `notebooks/feature_construction/01_Data_Clean.ipynb`
- `notebooks/feature_construction/02_Data_Clean_Statistics.ipynb`

#### 2. News Feature Construction
News/content-level attributes are derived first, likely producing reusable article features for later joins with user behavior and samples:
- `notebooks/feature_construction/11_News_Features.ipynb`
- `notebooks/feature_construction/12_News_Features_Results.ipynb`

The paired “Results” notebook pattern suggests each construction stage has a separate validation or inspection pass rather than an application runtime.

#### 3. User Feature Construction
User-level behavioral or profile features are built independently from news features:
- `notebooks/feature_construction/21_User_Features.ipynb`
- `notebooks/feature_construction/22_User_Features_Results.ipynb`

This is a key entity boundary: user-derived data can be joined with news-derived data only after each has been prepared.

#### 4. Sample Generation
Training/evaluation examples are assembled after core entity features exist:
- `notebooks/feature_construction/31_Samples.ipynb`
- `notebooks/feature_construction/32_Samples_Results.ipynb`

This stage likely defines the model-facing interaction records, labels, candidate examples, or dataset splits. It is the transition from feature engineering to supervised-learning inputs.

#### 5. Behavioral Aggregate Features
Two subsequent feature families enrich the sample data with historical behavior:
- News click/count features: `notebooks/feature_construction/41_News_Click_Counts.ipynb`, `notebooks/feature_construction/42_News_Click_Counts_Results.ipynb`
- User activity features: `notebooks/feature_construction/51_User_Activeness.ipynb`, `notebooks/feature_construction/52_User_Activeness_Results.ipynb`

These stages imply temporal or aggregate signals and should be treated carefully to prevent training-label leakage when constructing offline datasets.

#### 6. Feature Assembly and Tensor Serialization
All feature families are merged into the final model dataset, then converted to TensorFlow’s record format:
- `notebooks/feature_construction/61_All_Features.ipynb`
- `notebooks/feature_construction/62_All_Features_Results.ipynb`
- `notebooks/feature_construction/TFRecords.ipynb`

TFRecords are the principal interface between feature construction and model execution, indicating a TensorFlow-compatible training/input pipeline.

#### 7. Modeling: Offline and Online Parts
Model work is separated into offline and online notebook stages:
- `notebooks/models/Offline_Part.ipynb`
- `notebooks/models/Online_Part.ipynb`

The offline part likely trains, evaluates, or prepares model artifacts from TFRecords. The online part likely represents inference, recommendation/scoring, or deployment-oriented processing. No application source, service definitions, infrastructure manifests, or external-service integrations are present in the repository tree; execution depends on the notebook environment and whatever data/storage dependencies are configured within notebook cells.

Main entry points: [`notebooks/feature_construction/61_All_Features.ipynb`](https://github.com/lengochai97/thesis/blob/master/notebooks/feature_construction/61_All_Features.ipynb) (All features), [`notebooks/feature_construction/31_Samples.ipynb`](https://github.com/lengochai97/thesis/blob/master/notebooks/feature_construction/31_Samples.ipynb) (Sample generation), [`notebooks/feature_construction/01_Data_Clean.ipynb`](https://github.com/lengochai97/thesis/blob/master/notebooks/feature_construction/01_Data_Clean.ipynb) (Data cleaning), [`notebooks/feature_construction/11_News_Features.ipynb`](https://github.com/lengochai97/thesis/blob/master/notebooks/feature_construction/11_News_Features.ipynb) (News features)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_feature["Feature construction"]
  node_clean["Data cleaning<br/>notebook"]
  node_clean_stats["Cleaning statistics<br/>notebook"]
  node_news_features["News features<br/>notebook"]
  node_news_results["News feature results<br/>notebook"]
  node_user_features["User features<br/>notebook"]
  node_user_results["User feature results<br/>notebook"]
  node_samples["Sample generation<br/>notebook<br/>[31_Samples.ipynb]"]
  node_sample_results["Sample results<br/>notebook"]
  node_news_clicks["News click counts<br/>notebook"]
  node_user_activeness["User activeness<br/>notebook"]
  node_all_features["All features<br/>notebook"]
  node_feature_results["All-feature results<br/>notebook"]
  node_tfrecords["TFRecords<br/>serialization notebook<br/>[TFRecords.ipynb]"]
end

subgraph group_model["Modeling"]
  node_offline["Offline part<br/>model notebook<br/>[Offline_Part.ipynb]"]
  node_online["Online part<br/>model notebook<br/>[Online_Part.ipynb]"]
end

node_source_data[("Source data<br/>dataset")]

node_source_data -->|"raw input"| node_clean
node_clean -->|"cleaned data"| node_clean_stats
node_clean -->|"cleaned news"| node_news_features
node_clean -->|"cleaned users and interactions"| node_user_features
node_news_features -.->|"inspection"| node_news_results
node_user_features -.->|"inspection"| node_user_results
node_news_features -->|"article features"| node_samples
node_user_features -->|"user features"| node_samples
node_samples -.->|"inspection"| node_sample_results
node_samples -->|"interaction history"| node_news_clicks
node_samples -->|"interaction history"| node_user_activeness
node_samples -->|"base examples"| node_all_features
node_news_features -->|"content features"| node_all_features
node_user_features -->|"user features"| node_all_features
node_news_clicks -->|"engagement aggregates"| node_all_features
node_user_activeness -->|"activity aggregates"| node_all_features
node_all_features -.->|"inspection"| node_feature_results
node_all_features -->|"final dataset"| node_tfrecords
node_tfrecords -->|"training input"| node_offline
node_offline -->|"model artifacts"| node_online

click node_clean "https://github.com/lengochai97/thesis/blob/master/notebooks/feature_construction/01_Data_Clean.ipynb"
click node_clean_stats "https://github.com/lengochai97/thesis/blob/master/notebooks/feature_construction/02_Data_Clean_Statistics.ipynb"
click node_news_features "https://github.com/lengochai97/thesis/blob/master/notebooks/feature_construction/11_News_Features.ipynb"
click node_news_results "https://github.com/lengochai97/thesis/blob/master/notebooks/feature_construction/12_News_Features_Results.ipynb"
click node_user_features "https://github.com/lengochai97/thesis/blob/master/notebooks/feature_construction/21_User_Features.ipynb"
click node_user_results "https://github.com/lengochai97/thesis/blob/master/notebooks/feature_construction/22_User_Features_Results.ipynb"
click node_samples "https://github.com/lengochai97/thesis/blob/master/notebooks/feature_construction/31_Samples.ipynb"
click node_sample_results "https://github.com/lengochai97/thesis/blob/master/notebooks/feature_construction/32_Samples_Results.ipynb"
click node_news_clicks "https://github.com/lengochai97/thesis/blob/master/notebooks/feature_construction/41_News_Click_Counts.ipynb"
click node_user_activeness "https://github.com/lengochai97/thesis/blob/master/notebooks/feature_construction/51_User_Activeness.ipynb"
click node_all_features "https://github.com/lengochai97/thesis/blob/master/notebooks/feature_construction/61_All_Features.ipynb"
click node_feature_results "https://github.com/lengochai97/thesis/blob/master/notebooks/feature_construction/62_All_Features_Results.ipynb"
click node_tfrecords "https://github.com/lengochai97/thesis/blob/master/notebooks/feature_construction/TFRecords.ipynb"
click node_offline "https://github.com/lengochai97/thesis/blob/master/notebooks/models/Offline_Part.ipynb"
click node_online "https://github.com/lengochai97/thesis/blob/master/notebooks/models/Online_Part.ipynb"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_clean,node_clean_stats,node_news_features,node_news_results,node_user_features,node_user_results,node_samples,node_sample_results,node_news_clicks,node_user_activeness,node_all_features,node_feature_results,node_tfrecords toneBlue
class node_offline,node_online toneAmber
class node_source_data toneNeutral
```

## Components

### Feature construction

Ordered notebooks create model-ready interaction data.

- [Data cleaning](https://github.com/lengochai97/thesis/blob/master/notebooks/feature_construction/01_Data_Clean.ipynb): notebook · `notebooks/feature_construction/01_Data_Clean.ipynb`
- [Cleaning statistics](https://github.com/lengochai97/thesis/blob/master/notebooks/feature_construction/02_Data_Clean_Statistics.ipynb): notebook · Profiles the cleaned dataset. · `notebooks/feature_construction/02_Data_Clean_Statistics.ipynb`
- [News features](https://github.com/lengochai97/thesis/blob/master/notebooks/feature_construction/11_News_Features.ipynb): notebook · `notebooks/feature_construction/11_News_Features.ipynb`
- [News feature results](https://github.com/lengochai97/thesis/blob/master/notebooks/feature_construction/12_News_Features_Results.ipynb): notebook · Inspects content-level features. · `notebooks/feature_construction/12_News_Features_Results.ipynb`
- [User features](https://github.com/lengochai97/thesis/blob/master/notebooks/feature_construction/21_User_Features.ipynb): notebook · `notebooks/feature_construction/21_User_Features.ipynb`
- [User feature results](https://github.com/lengochai97/thesis/blob/master/notebooks/feature_construction/22_User_Features_Results.ipynb): notebook · Inspects user-level features. · `notebooks/feature_construction/22_User_Features_Results.ipynb`
- [Sample generation](https://github.com/lengochai97/thesis/blob/master/notebooks/feature_construction/31_Samples.ipynb): notebook · Builds labeled model-facing interaction examples. · `notebooks/feature_construction/31_Samples.ipynb`
- [Sample results](https://github.com/lengochai97/thesis/blob/master/notebooks/feature_construction/32_Samples_Results.ipynb): notebook · Inspects generated training and evaluation examples. · `notebooks/feature_construction/32_Samples_Results.ipynb`
- [News click counts](https://github.com/lengochai97/thesis/blob/master/notebooks/feature_construction/41_News_Click_Counts.ipynb): notebook · Adds historical news engagement aggregates. · `notebooks/feature_construction/41_News_Click_Counts.ipynb`
- [User activeness](https://github.com/lengochai97/thesis/blob/master/notebooks/feature_construction/51_User_Activeness.ipynb): notebook · Adds historical user activity aggregates. · `notebooks/feature_construction/51_User_Activeness.ipynb`
- [All features](https://github.com/lengochai97/thesis/blob/master/notebooks/feature_construction/61_All_Features.ipynb): notebook · Merges entity, sample, and behavioral feature families. · `notebooks/feature_construction/61_All_Features.ipynb`
- [All-feature results](https://github.com/lengochai97/thesis/blob/master/notebooks/feature_construction/62_All_Features_Results.ipynb): notebook · Validates the assembled model dataset. · `notebooks/feature_construction/62_All_Features_Results.ipynb`
- [TFRecords](https://github.com/lengochai97/thesis/blob/master/notebooks/feature_construction/TFRecords.ipynb): serialization notebook · Serializes the final dataset for TensorFlow input. · `notebooks/feature_construction/TFRecords.ipynb`

### Modeling

Offline and online notebook stages consume serialized data.

- [Offline part](https://github.com/lengochai97/thesis/blob/master/notebooks/models/Offline_Part.ipynb): model notebook · Trains, evaluates, or prepares model artifacts. · `notebooks/models/Offline_Part.ipynb`
- [Online part](https://github.com/lengochai97/thesis/blob/master/notebooks/models/Online_Part.ipynb): model notebook · Performs inference or recommendation-oriented processing. · `notebooks/models/Online_Part.ipynb`

### Other components

- **Source data**: dataset · Raw news and user-interaction data configured in notebook cells.

## Connections

- Source data → Data cleaning: raw input
- Data cleaning → Cleaning statistics: cleaned data
- Data cleaning → News features: cleaned news
- Data cleaning → User features: cleaned users and interactions
- News features → News feature results: inspection
- User features → User feature results: inspection
- News features → Sample generation: article features
- User features → Sample generation: user features
- Sample generation → Sample results: inspection
- Sample generation → News click counts: interaction history
- Sample generation → User activeness: interaction history
- Sample generation → All features: base examples
- News features → All features: content features
- User features → All features: user features
- News click counts → All features: engagement aggregates
- User activeness → All features: activity aggregates
- All features → All-feature results: inspection
- All features → TFRecords: final dataset
- TFRecords → Offline part: training input
- Offline part → Online part: model artifacts