How navikt/innholdsoversikt works
Key Components to Include:
Main System Components
• Kubernetes Cronjob (Primary Orchestrator)
• Python Application Container
• External Services:
• Enonic XP CMS (Data Source)
• BigQuery (Data Storage)
• Google Cloud Storage (GCS)
• Metabase Dashboard (Data Visualization)
Data Flow Architecture
a. Cronjob Trigger → Python Application
b. Python Application → Enonic XP Data Query API
c. Data Processing Pipeline
d. Processed Data → BigQuery Storage
e. BigQuery → Metabase Dashboard
Component Details
Python Application (Core):
• enonic_data_api.py (Data Fetching)
• parse_json.py (Data Processing)
• bygg_df.py (DataFrame Construction)
• bq_tabell_jobber.py (BigQuery Operations)
• gcs_api.py (Cloud Storage Operations)
Infrastructure Layer
• NAIS Platform (NAV's Kubernetes Platform)
• Secrets Management
• Logging System (Kibana)
Labels
• Component names
• Key technologies
• Data flow directions
• Cronjob schedule indication
Security Layer
• Secret management integration
• Secure connections
Monitoring
• Kibana logging flow
• Metabase dashboard connection
Development vs Production
• Adding a small subset showing local development setup
• Environment-specific configurations
Level of Detail:
• Data processing pipeline
• Infrastructure setup
• Local development environment
Security and monitoring aspects
This system design should provide a clear understanding of how the application collects, processes, and visualizes content statistics from nav.no while maintaining appropriate security and monitoring capabilities.