How otniel113/cf-analytics works
1. Overall shape
- This repo is split into an offline analytics/modeling area and an online dashboard app.
- The offline side lives in notebooks/ and produces cleaned datasets, analysis tables, and trained ML artifacts.
- The runtime app lives in cf-analytics/ and serves a Comifuro analytics dashboard plus an LLM chatbot.
2. Runtime entry point
- cf-analytics/app.py is the application entry point.
- The app uses templates/index.html as the base page and assets/css/style.css for styling, so the UI is server-rendered and then assembled into feature sections from Python modules.
3. Feature modularization
- The dashboard is decomposed into tab modules under cf-analytics/tabs/.
- Top-level sections are:
- tab01_home.py — landing / overview
- tab02_master_data.py — searchable/filterable booth master table
- tab03_ed_analysis.py — exploratory data analysis
- tab04_advanced_analysis.py — ML-driven insights and recommendations
- tab05_trend_analysis.py — comparison across Comifuro editions
- This tab structure is the main presentation boundary in the app.
4. Advanced analysis subsystem
- tabs/subtabs/subtab04_recommendation.py and tabs/subtabs/subtab04_faktor.py hold the advanced analytics views.
- The detailed factor pages in tabs/subtabs/faktor_detail/ break down what influences booth behavior, likely separating rating and type/circle effects.
- This subsystem is powered by precomputed models and exported importance tables rather than live training.
5. Trend / fandom analysis subsystem
- subtab05_fandom.py and subtab05_returning_circle.py support higher-level trend interpretation.
- These pages rely on historical CF data to show returning circles and fandom grouping over time.
- The repo’s README makes clear that fandom grouping is an explicit analytical layer, not just a label on raw data.
6. Data artifacts used at runtime
- Runtime data is stored as pickle files in cf-analytics/data/ and cf-analytics/pipelines/.
- Key artifacts include:
- df_cf.pkl — main booth dataset
- df_importance_rating.pkl, df_importance_circle_type.pkl — feature importance / factor outputs
- df_returning.pkl — returning booth analysis
- prediksi_rating_pipeline.pkl, prediksi_type_pipeline.pkl, recommender_pipeline.pkl — inference pipelines
- These artifacts are loaded by the dashboard to keep the app fast and avoid training at request time.
7. Chatbot subsystem
- cf-analytics/chatbot/kofu_chan.py is a separate chatbot module for “Kofu-chan.”
- It is presented in the README as an LLM chatbot for querying CF22 catalog details.
- Architecturally, this is a distinct interaction path from the dashboard tabs: it answers questions over catalog data rather than rendering analytics views.
8. Offline data engineering and model building
- notebooks/ contains the data science pipeline that prepares the runtime artifacts.
- Scripts such as 01_CF_EDA.py, 02A_Recommendation_System.py, 02B_*, and 02C_* indicate separate workstreams for EDA, recommendation, rating analysis, and type analysis.
- notebooks/results_export/ stores exported datasets and trained pipelines, which are then mirrored into the app’s data/ and pipelines/ folders.
9. Historical and source data boundary
- The notebook area includes source catalog files like cf21-catalog.json and cf22-catalog.json, which appear to be the basis for historical and current booth analysis.
- This indicates a clear separation between raw/source catalog data and curated analytic datasets used by the app.
10. Deployment and packaging
- cf-analytics/Dockerfile suggests containerized deployment.
- The README points to a hosted Hugging Face Space, so the app is intended to run as a packaged web service rather than a local notebook-only project.
- requirements.txt defines the Python runtime dependencies for the dashboard and analytics code.
11. Main data flow
- Source CF catalog data is gathered and explored in notebooks.
- Notebooks transform it into analyzed tables and trained pipelines.
- Those exported artifacts are copied into the app folder.
- At runtime, app.py loads the artifacts and renders tabbed analytics pages plus the chatbot interface.
12. Architectural takeaway
- This repo is best diagrammed as a two-stage analytics system: offline Python data science pipelines feeding a runtime dashboard + chatbot.
- The key boundaries are:
- source data vs curated analytics artifacts
- offline training/analysis vs online inference/rendering
- dashboard tabs vs chatbot interaction
Main entry points: results_export Exported results · subtabs Advanced views · app.py App entry · tabs Tab modules
Components 21
Offline analytics
Notebook-driven data preparation, exploration, and model building.
- Catalog data · Historical CF catalog JSON files used as analysis inputs.
notebooks/cf21-catalog.json - EDA notebook · Explores and cleans the core CF dataset.
notebooks/01_CF_EDA.py - Rec notebook · Builds recommendation logic and related exports.
notebooks/02A_Recommendation_System.py - Rating factors · Trains and analyzes rating-related factor outputs.
notebooks/02B_Faktor_Rating_Part_1.py - Type factors · Trains and analyzes circle/type-related factor outputs.
notebooks/02C_Faktor_Tipe_Part_1.py
Runtime app
Packaged dashboard and chatbot served from the cf-analytics app.
- App entry · Main runtime entry point that loads artifacts and assembles the interface.
cf-analytics/app.py - Page shell · Base HTML template for the server-rendered dashboard.
cf-analytics/templates/index.html - Stylesheet · Global visual styling for the dashboard.
cf-analytics/assets/css/style.css - Tab modules · Top-level dashboard sections loaded by the app.
cf-analytics/tabs - Advanced views · Recommendation and factor breakdown views under advanced analysis.
cf-analytics/tabs/subtabs - Trend views · Fandom and returning-circle views for edition comparison.
cf-analytics/tabs/subtabs - Factor detail · Detailed rating and type factor explanation pages.
cf-analytics/tabs/subtabs/faktor_detail - Kofu-chan · LLM chatbot for querying CF22 catalog details.
cf-analytics/chatbot/kofu_chan.py
Runtime artifacts
Curated datasets and trained pipelines loaded by the app.
- Exported results · Notebook-exported tables and trained pipelines for runtime use.
notebooks/results_export - Booth data · Main cleaned booth dataset used by the dashboard.
cf-analytics/data/df_cf.pkl - Rating importance · Feature importance table for rating analysis.
cf-analytics/data/df_importance_rating.pkl - Type importance · Feature importance table for circle/type analysis.
cf-analytics/data/df_importance_circle_type.pkl - Returning booths · Historical returning-circle analysis table.
cf-analytics/data/df_returning.pkl - Rating pipeline · Precomputed pipeline for rating prediction and analysis.
cf-analytics/pipelines/prediksi_rating_pipeline.pkl - Type pipeline · Precomputed pipeline for type prediction and analysis.
cf-analytics/pipelines/prediksi_type_pipeline.pkl - Recommender · Precomputed recommendation pipeline used in advanced analysis.
cf-analytics/pipelines/recommender_pipeline.pkl
Connections 27
- Catalog data → EDA notebook: source input · Catalog data feeds exploration and cleaning.
- EDA notebook → Exported results: curated output · EDA produces cleaned data and analysis-ready exports.
- Rec notebook → Exported results: model output · Recommendation work exports trained pipelines and tables.
- Rating factors → Exported results: factor output · Rating analysis exports importance tables and models.
- Type factors → Exported results: factor output · Type analysis exports importance tables and models.
- Exported results → Booth data: published artifact · Main cleaned booth dataset is mirrored into the app runtime.
- Exported results → Rating importance: published artifact · Rating importance table is mirrored into the app runtime.
- Exported results → Type importance: published artifact · Type importance table is mirrored into the app runtime.
- Exported results → Returning booths: published artifact · Returning analysis table is mirrored into the app runtime.
- Exported results → Rating pipeline: published artifact · Rating prediction pipeline is deployed for runtime inference.
- Exported results → Type pipeline: published artifact · Type prediction pipeline is deployed for runtime inference.
- Exported results → Recommender: published artifact · Recommendation pipeline is deployed for runtime inference.
- App entry → Page shell: renders · The app uses the base HTML template for the dashboard shell.
- App entry → Stylesheet: styles · The app applies the shared stylesheet.
- App entry → Tab modules: loads tabs · The entry point assembles the tabbed dashboard sections.
- App entry → Kofu-chan: serves chat · The app exposes the chatbot interaction path.
- Tab modules → Advanced views: includes · Advanced analysis tab expands into recommendation and factor subtabs.
- Tab modules → Trend views: includes · Trend analysis tab expands into fandom and returning subtabs.
- Advanced views → Factor detail: drills down · Factor pages break advanced analysis into detailed explanations.
- Booth data → Tab modules: drives views · Core dataset powers master data, EDA, and overview tabs.
- Rating importance → Advanced views: powers · Rating importance tables feed advanced analysis views.
- Type importance → Advanced views: powers · Type importance tables feed advanced analysis views.
- Returning booths → Trend views: powers · Returning analysis data feeds trend views.
- Rating pipeline → Advanced views: inference · Rating pipeline supports recommendation and factor insights.
- Type pipeline → Advanced views: inference · Type pipeline supports recommendation and factor insights.
- Recommender → Advanced views: inference · Recommender pipeline powers suggestion views.
- Booth data → Kofu-chan: catalog context · The chatbot queries booth/catalog details from the curated dataset.