lengochai97/thesis

Purpose and Entry Points

This is a thesis-oriented machine-learning workflow organized entirely as Jupyter notebooks. Its apparent goal is to construct news/user interaction features, generate training samples and TFRecords, then run offline and online model stages. Primary entry points are the ordered notebooks under notebooks/feature_construction/ and notebooks/models/.

1. Data Cleaning and Profiling

The pipeline begins by cleaning source data and inspecting resulting distributions/statistics. These notebooks establish the usable dataset boundary before feature generation:

• notebooks/feature_construction/01_Data_Clean.ipynb

• notebooks/feature_construction/02_Data_Clean_Statistics.ipynb

2. News Feature Construction

News/content-level attributes are derived first, likely producing reusable article features for later joins with user behavior and samples:

• notebooks/feature_construction/11_News_Features.ipynb

• notebooks/feature_construction/12_News_Features_Results.ipynb

The paired “Results” notebook pattern suggests each construction stage has a separate validation or inspection pass rather than an application runtime.

3. User Feature Construction

User-level behavioral or profile features are built independently from news features:

• notebooks/feature_construction/21_User_Features.ipynb

• notebooks/feature_construction/22_User_Features_Results.ipynb

This is a key entity boundary: user-derived data can be joined with news-derived data only after each has been prepared.

4. Sample Generation

Training/evaluation examples are assembled after core entity features exist:

• notebooks/feature_construction/31_Samples.ipynb

• notebooks/feature_construction/32_Samples_Results.ipynb

This stage likely defines the model-facing interaction records, labels, candidate examples, or dataset splits. It is the transition from feature engineering to supervised-learning inputs.

5. Behavioral Aggregate Features

Two subsequent feature families enrich the sample data with historical behavior:

• News click/count features: notebooks/feature_construction/41_News_Click_Counts.ipynb, notebooks/feature_construction/42_News_Click_Counts_Results.ipynb

• User activity features: notebooks/feature_construction/51_User_Activeness.ipynb, notebooks/feature_construction/52_User_Activeness_Results.ipynb

These stages imply temporal or aggregate signals and should be treated carefully to prevent training-label leakage when constructing offline datasets.

6. Feature Assembly and Tensor Serialization

All feature families are merged into the final model dataset, then converted to TensorFlow’s record format:

• notebooks/feature_construction/61_All_Features.ipynb

• notebooks/feature_construction/62_All_Features_Results.ipynb

• notebooks/feature_construction/TFRecords.ipynb

TFRecords are the principal interface between feature construction and model execution, indicating a TensorFlow-compatible training/input pipeline.

7. Modeling: Offline and Online Parts

Model work is separated into offline and online notebook stages:

• notebooks/models/Offline_Part.ipynb

• notebooks/models/Online_Part.ipynb

The offline part likely trains, evaluates, or prepares model artifacts from TFRecords. The online part likely represents inference, recommendation/scoring, or deployment-oriented processing. No application source, service definitions, infrastructure manifests, or external-service integrations are present in the repository tree; execution depends on the notebook environment and whatever data/storage dependencies are configured within notebook cells.