# otniel113/cf-analytics architecture

> Architecture diagram of the GitHub repository otniel113/cf-analytics, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/otniel113/cf-analytics
- Repository: https://github.com/otniel113/cf-analytics

## Overview

1. **Overall shape**
   - This repo is split into an **offline analytics/modeling area** and an **online dashboard app**.
   - The offline side lives in `notebooks/` and produces cleaned datasets, analysis tables, and trained ML artifacts.
   - The runtime app lives in `cf-analytics/` and serves a Comifuro analytics dashboard plus an LLM chatbot.

2. **Runtime entry point**
   - `cf-analytics/app.py` is the application entry point.
   - The app uses `templates/index.html` as the base page and `assets/css/style.css` for styling, so the UI is server-rendered and then assembled into feature sections from Python modules.

3. **Feature modularization**
   - The dashboard is decomposed into tab modules under `cf-analytics/tabs/`.
   - Top-level sections are:
     - `tab01_home.py` — landing / overview
     - `tab02_master_data.py` — searchable/filterable booth master table
     - `tab03_ed_analysis.py` — exploratory data analysis
     - `tab04_advanced_analysis.py` — ML-driven insights and recommendations
     - `tab05_trend_analysis.py` — comparison across Comifuro editions
   - This tab structure is the main presentation boundary in the app.

4. **Advanced analysis subsystem**
   - `tabs/subtabs/subtab04_recommendation.py` and `tabs/subtabs/subtab04_faktor.py` hold the advanced analytics views.
   - The detailed factor pages in `tabs/subtabs/faktor_detail/` break down what influences booth behavior, likely separating **rating** and **type/circle** effects.
   - This subsystem is powered by precomputed models and exported importance tables rather than live training.

5. **Trend / fandom analysis subsystem**
   - `subtab05_fandom.py` and `subtab05_returning_circle.py` support higher-level trend interpretation.
   - These pages rely on historical CF data to show returning circles and fandom grouping over time.
   - The repo’s README makes clear that fandom grouping is an explicit analytical layer, not just a label on raw data.

6. **Data artifacts used at runtime**
   - Runtime data is stored as pickle files in `cf-analytics/data/` and `cf-analytics/pipelines/`.
   - Key artifacts include:
     - `df_cf.pkl` — main booth dataset
     - `df_importance_rating.pkl`, `df_importance_circle_type.pkl` — feature importance / factor outputs
     - `df_returning.pkl` — returning booth analysis
     - `prediksi_rating_pipeline.pkl`, `prediksi_type_pipeline.pkl`, `recommender_pipeline.pkl` — inference pipelines
   - These artifacts are loaded by the dashboard to keep the app fast and avoid training at request time.

7. **Chatbot subsystem**
   - `cf-analytics/chatbot/kofu_chan.py` is a separate chatbot module for “Kofu-chan.”
   - It is presented in the README as an LLM chatbot for querying CF22 catalog details.
   - Architecturally, this is a distinct interaction path from the dashboard tabs: it answers questions over catalog data rather than rendering analytics views.

8. **Offline data engineering and model building**
   - `notebooks/` contains the data science pipeline that prepares the runtime artifacts.
   - Scripts such as `01_CF_EDA.py`, `02A_Recommendation_System.py`, `02B_*`, and `02C_*` indicate separate workstreams for EDA, recommendation, rating analysis, and type analysis.
   - `notebooks/results_export/` stores exported datasets and trained pipelines, which are then mirrored into the app’s `data/` and `pipelines/` folders.

9. **Historical and source data boundary**
   - The notebook area includes source catalog files like `cf21-catalog.json` and `cf22-catalog.json`, which appear to be the basis for historical and current booth analysis.
   - This indicates a clear separation between **raw/source catalog data** and **curated analytic datasets** used by the app.

10. **Deployment and packaging**
   - `cf-analytics/Dockerfile` suggests containerized deployment.
   - The README points to a hosted Hugging Face Space, so the app is intended to run as a packaged web service rather than a local notebook-only project.
   - `requirements.txt` defines the Python runtime dependencies for the dashboard and analytics code.

11. **Main data flow**
   - Source CF catalog data is gathered and explored in notebooks.
   - Notebooks transform it into analyzed tables and trained pipelines.
   - Those exported artifacts are copied into the app folder.
   - At runtime, `app.py` loads the artifacts and renders tabbed analytics pages plus the chatbot interface.

12. **Architectural takeaway**
   - This repo is best diagrammed as a **two-stage analytics system**: offline Python data science pipelines feeding a **runtime dashboard + chatbot**.
   - The key boundaries are:
     - source data vs curated analytics artifacts
     - offline training/analysis vs online inference/rendering
     - dashboard tabs vs chatbot interaction

Main entry points: [`notebooks/results_export`](https://github.com/otniel113/cf-analytics/tree/main/notebooks/results_export) (Exported results), [`cf-analytics/tabs/subtabs`](https://github.com/otniel113/cf-analytics/tree/main/cf-analytics/tabs/subtabs) (Advanced views), [`cf-analytics/app.py`](https://github.com/otniel113/cf-analytics/blob/main/cf-analytics/app.py) (App entry), [`cf-analytics/tabs`](https://github.com/otniel113/cf-analytics/tree/main/cf-analytics/tabs) (Tab modules)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_offline["Offline analytics"]
  node_src_catalogs["Catalog data<br/>source data<br/>[cf21-catalog.json]"]
  node_eda_notebook["EDA notebook<br/>analysis notebook<br/>[01_CF_EDA.py]"]
  node_reco_notebook["Rec notebook<br/>model notebook"]
  node_rating_notebook["Rating factors<br/>model notebook"]
  node_type_notebook["Type factors<br/>model notebook"]
end

subgraph group_runtime["Runtime app"]
  node_app_entry["App entry<br/>web app<br/>[app.py]"]
  node_template_shell["Page shell<br/>ui template<br/>[index.html]"]
  node_style_css["Stylesheet<br/>ui asset<br/>[style.css]"]
  node_tab_modules["Tab modules<br/>presentation modules"]
  node_advanced_subtabs["Advanced views<br/>analysis views"]
  node_trend_subtabs["Trend views<br/>analysis views"]
  node_faktor_detail["Factor detail<br/>detail views"]
  node_chatbot["Kofu-chan<br/>chatbot<br/>[kofu_chan.py]"]
end

subgraph group_artifacts["Runtime artifacts"]
  node_results_export[("Exported results<br/>artifact bundle")]
  node_df_cf[("Booth data<br/>dataset<br/>[df_cf.pkl]")]
  node_df_importance_rating[("Rating importance<br/>analytic table")]
  node_df_importance_circle_type[("Type importance<br/>analytic table")]
  node_df_returning[("Returning booths<br/>analytic table<br/>[df_returning.pkl]")]
  node_rating_pipeline["Rating pipeline<br/>inference pipeline"]
  node_type_pipeline["Type pipeline<br/>inference pipeline"]
  node_recommender_pipeline["Recommender<br/>inference pipeline"]
end

node_src_catalogs -->|"source input"| node_eda_notebook
node_eda_notebook -->|"curated output"| node_results_export
node_reco_notebook -->|"model output"| node_results_export
node_rating_notebook -->|"factor output"| node_results_export
node_type_notebook -->|"factor output"| node_results_export
node_results_export -->|"published artifact"| node_df_cf
node_results_export -->|"published artifact"| node_df_importance_rating
node_results_export -->|"published artifact"| node_df_importance_circle_type
node_results_export -->|"published artifact"| node_df_returning
node_results_export -->|"published artifact"| node_rating_pipeline
node_results_export -->|"published artifact"| node_type_pipeline
node_results_export -->|"published artifact"| node_recommender_pipeline
node_app_entry -->|"renders"| node_template_shell
node_app_entry -->|"styles"| node_style_css
node_app_entry -->|"loads tabs"| node_tab_modules
node_app_entry -->|"serves chat"| node_chatbot
node_tab_modules -->|"includes"| node_advanced_subtabs
node_tab_modules -->|"includes"| node_trend_subtabs
node_advanced_subtabs -->|"drills down"| node_faktor_detail
node_df_cf -->|"drives views"| node_tab_modules
node_df_importance_rating -->|"powers"| node_advanced_subtabs
node_df_importance_circle_type -->|"powers"| node_advanced_subtabs
node_df_returning -->|"powers"| node_trend_subtabs
node_rating_pipeline -->|"inference"| node_advanced_subtabs
node_type_pipeline -->|"inference"| node_advanced_subtabs
node_recommender_pipeline -->|"inference"| node_advanced_subtabs
node_df_cf -->|"catalog context"| node_chatbot

click node_src_catalogs "https://github.com/otniel113/cf-analytics/blob/main/notebooks/cf21-catalog.json"
click node_eda_notebook "https://github.com/otniel113/cf-analytics/blob/main/notebooks/01_CF_EDA.py"
click node_reco_notebook "https://github.com/otniel113/cf-analytics/blob/main/notebooks/02A_Recommendation_System.py"
click node_rating_notebook "https://github.com/otniel113/cf-analytics/blob/main/notebooks/02B_Faktor_Rating_Part_1.py"
click node_type_notebook "https://github.com/otniel113/cf-analytics/blob/main/notebooks/02C_Faktor_Tipe_Part_1.py"
click node_results_export "https://github.com/otniel113/cf-analytics/tree/main/notebooks/results_export"
click node_df_cf "https://github.com/otniel113/cf-analytics/blob/main/cf-analytics/data/df_cf.pkl"
click node_df_importance_rating "https://github.com/otniel113/cf-analytics/blob/main/cf-analytics/data/df_importance_rating.pkl"
click node_df_importance_circle_type "https://github.com/otniel113/cf-analytics/blob/main/cf-analytics/data/df_importance_circle_type.pkl"
click node_df_returning "https://github.com/otniel113/cf-analytics/blob/main/cf-analytics/data/df_returning.pkl"
click node_rating_pipeline "https://github.com/otniel113/cf-analytics/blob/main/cf-analytics/pipelines/prediksi_rating_pipeline.pkl"
click node_type_pipeline "https://github.com/otniel113/cf-analytics/blob/main/cf-analytics/pipelines/prediksi_type_pipeline.pkl"
click node_recommender_pipeline "https://github.com/otniel113/cf-analytics/blob/main/cf-analytics/pipelines/recommender_pipeline.pkl"
click node_app_entry "https://github.com/otniel113/cf-analytics/blob/main/cf-analytics/app.py"
click node_template_shell "https://github.com/otniel113/cf-analytics/blob/main/cf-analytics/templates/index.html"
click node_style_css "https://github.com/otniel113/cf-analytics/blob/main/cf-analytics/assets/css/style.css"
click node_tab_modules "https://github.com/otniel113/cf-analytics/tree/main/cf-analytics/tabs"
click node_advanced_subtabs "https://github.com/otniel113/cf-analytics/tree/main/cf-analytics/tabs/subtabs"
click node_trend_subtabs "https://github.com/otniel113/cf-analytics/tree/main/cf-analytics/tabs/subtabs"
click node_faktor_detail "https://github.com/otniel113/cf-analytics/tree/main/cf-analytics/tabs/subtabs/faktor_detail"
click node_chatbot "https://github.com/otniel113/cf-analytics/blob/main/cf-analytics/chatbot/kofu_chan.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_src_catalogs,node_eda_notebook,node_reco_notebook,node_rating_notebook,node_type_notebook toneBlue
class node_app_entry,node_template_shell,node_style_css,node_tab_modules,node_advanced_subtabs,node_trend_subtabs,node_faktor_detail,node_chatbot toneAmber
class node_results_export,node_df_cf,node_df_importance_rating,node_df_importance_circle_type,node_df_returning,node_rating_pipeline,node_type_pipeline,node_recommender_pipeline toneMint
```

## Components

### Offline analytics

Notebook-driven data preparation, exploration, and model building.

- [Catalog data](https://github.com/otniel113/cf-analytics/blob/main/notebooks/cf21-catalog.json): source data · Historical CF catalog JSON files used as analysis inputs. · `notebooks/cf21-catalog.json`
- [EDA notebook](https://github.com/otniel113/cf-analytics/blob/main/notebooks/01_CF_EDA.py): analysis notebook · Explores and cleans the core CF dataset. · `notebooks/01_CF_EDA.py`
- [Rec notebook](https://github.com/otniel113/cf-analytics/blob/main/notebooks/02A_Recommendation_System.py): model notebook · Builds recommendation logic and related exports. · `notebooks/02A_Recommendation_System.py`
- [Rating factors](https://github.com/otniel113/cf-analytics/blob/main/notebooks/02B_Faktor_Rating_Part_1.py): model notebook · Trains and analyzes rating-related factor outputs. · `notebooks/02B_Faktor_Rating_Part_1.py`
- [Type factors](https://github.com/otniel113/cf-analytics/blob/main/notebooks/02C_Faktor_Tipe_Part_1.py): model notebook · Trains and analyzes circle/type-related factor outputs. · `notebooks/02C_Faktor_Tipe_Part_1.py`

### Runtime app

Packaged dashboard and chatbot served from the cf-analytics app.

- [App entry](https://github.com/otniel113/cf-analytics/blob/main/cf-analytics/app.py): web app · Main runtime entry point that loads artifacts and assembles the interface. · `cf-analytics/app.py`
- [Page shell](https://github.com/otniel113/cf-analytics/blob/main/cf-analytics/templates/index.html): ui template · Base HTML template for the server-rendered dashboard. · `cf-analytics/templates/index.html`
- [Stylesheet](https://github.com/otniel113/cf-analytics/blob/main/cf-analytics/assets/css/style.css): ui asset · Global visual styling for the dashboard. · `cf-analytics/assets/css/style.css`
- [Tab modules](https://github.com/otniel113/cf-analytics/tree/main/cf-analytics/tabs): presentation modules · Top-level dashboard sections loaded by the app. · `cf-analytics/tabs`
- [Advanced views](https://github.com/otniel113/cf-analytics/tree/main/cf-analytics/tabs/subtabs): analysis views · Recommendation and factor breakdown views under advanced analysis. · `cf-analytics/tabs/subtabs`
- [Trend views](https://github.com/otniel113/cf-analytics/tree/main/cf-analytics/tabs/subtabs): analysis views · Fandom and returning-circle views for edition comparison. · `cf-analytics/tabs/subtabs`
- [Factor detail](https://github.com/otniel113/cf-analytics/tree/main/cf-analytics/tabs/subtabs/faktor_detail): detail views · Detailed rating and type factor explanation pages. · `cf-analytics/tabs/subtabs/faktor_detail`
- [Kofu-chan](https://github.com/otniel113/cf-analytics/blob/main/cf-analytics/chatbot/kofu_chan.py): chatbot · LLM chatbot for querying CF22 catalog details. · `cf-analytics/chatbot/kofu_chan.py`

### Runtime artifacts

Curated datasets and trained pipelines loaded by the app.

- [Exported results](https://github.com/otniel113/cf-analytics/tree/main/notebooks/results_export): artifact bundle · Notebook-exported tables and trained pipelines for runtime use. · `notebooks/results_export`
- [Booth data](https://github.com/otniel113/cf-analytics/blob/main/cf-analytics/data/df_cf.pkl): dataset · Main cleaned booth dataset used by the dashboard. · `cf-analytics/data/df_cf.pkl`
- [Rating importance](https://github.com/otniel113/cf-analytics/blob/main/cf-analytics/data/df_importance_rating.pkl): analytic table · Feature importance table for rating analysis. · `cf-analytics/data/df_importance_rating.pkl`
- [Type importance](https://github.com/otniel113/cf-analytics/blob/main/cf-analytics/data/df_importance_circle_type.pkl): analytic table · Feature importance table for circle/type analysis. · `cf-analytics/data/df_importance_circle_type.pkl`
- [Returning booths](https://github.com/otniel113/cf-analytics/blob/main/cf-analytics/data/df_returning.pkl): analytic table · Historical returning-circle analysis table. · `cf-analytics/data/df_returning.pkl`
- [Rating pipeline](https://github.com/otniel113/cf-analytics/blob/main/cf-analytics/pipelines/prediksi_rating_pipeline.pkl): inference pipeline · Precomputed pipeline for rating prediction and analysis. · `cf-analytics/pipelines/prediksi_rating_pipeline.pkl`
- [Type pipeline](https://github.com/otniel113/cf-analytics/blob/main/cf-analytics/pipelines/prediksi_type_pipeline.pkl): inference pipeline · Precomputed pipeline for type prediction and analysis. · `cf-analytics/pipelines/prediksi_type_pipeline.pkl`
- [Recommender](https://github.com/otniel113/cf-analytics/blob/main/cf-analytics/pipelines/recommender_pipeline.pkl): inference pipeline · Precomputed recommendation pipeline used in advanced analysis. · `cf-analytics/pipelines/recommender_pipeline.pkl`

## Connections

- Catalog data → EDA notebook: source input · Catalog data feeds exploration and cleaning.
- EDA notebook → Exported results: curated output · EDA produces cleaned data and analysis-ready exports.
- Rec notebook → Exported results: model output · Recommendation work exports trained pipelines and tables.
- Rating factors → Exported results: factor output · Rating analysis exports importance tables and models.
- Type factors → Exported results: factor output · Type analysis exports importance tables and models.
- Exported results → Booth data: published artifact · Main cleaned booth dataset is mirrored into the app runtime.
- Exported results → Rating importance: published artifact · Rating importance table is mirrored into the app runtime.
- Exported results → Type importance: published artifact · Type importance table is mirrored into the app runtime.
- Exported results → Returning booths: published artifact · Returning analysis table is mirrored into the app runtime.
- Exported results → Rating pipeline: published artifact · Rating prediction pipeline is deployed for runtime inference.
- Exported results → Type pipeline: published artifact · Type prediction pipeline is deployed for runtime inference.
- Exported results → Recommender: published artifact · Recommendation pipeline is deployed for runtime inference.
- App entry → Page shell: renders · The app uses the base HTML template for the dashboard shell.
- App entry → Stylesheet: styles · The app applies the shared stylesheet.
- App entry → Tab modules: loads tabs · The entry point assembles the tabbed dashboard sections.
- App entry → Kofu-chan: serves chat · The app exposes the chatbot interaction path.
- Tab modules → Advanced views: includes · Advanced analysis tab expands into recommendation and factor subtabs.
- Tab modules → Trend views: includes · Trend analysis tab expands into fandom and returning subtabs.
- Advanced views → Factor detail: drills down · Factor pages break advanced analysis into detailed explanations.
- Booth data → Tab modules: drives views · Core dataset powers master data, EDA, and overview tabs.
- Rating importance → Advanced views: powers · Rating importance tables feed advanced analysis views.
- Type importance → Advanced views: powers · Type importance tables feed advanced analysis views.
- Returning booths → Trend views: powers · Returning analysis data feeds trend views.
- Rating pipeline → Advanced views: inference · Rating pipeline supports recommendation and factor insights.
- Type pipeline → Advanced views: inference · Type pipeline supports recommendation and factor insights.
- Recommender → Advanced views: inference · Recommender pipeline powers suggestion views.
- Booth data → Kofu-chan: catalog context · The chatbot queries booth/catalog details from the curated dataset.