# navikt/innholdsoversikt architecture

> Architecture diagram of the GitHub repository navikt/innholdsoversikt, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2024-12-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/navikt/innholdsoversikt
- Repository: https://github.com/navikt/innholdsoversikt

## Overview

Key Components to Include:

##### Main System Components
- Kubernetes Cronjob (Primary Orchestrator)
- Python Application Container
- External Services:
- Enonic XP CMS (Data Source)
- BigQuery (Data Storage)
- Google Cloud Storage (GCS)
- Metabase Dashboard (Data Visualization)

##### Data Flow Architecture
a. Cronjob Trigger → Python Application
b. Python Application → Enonic XP Data Query API
c. Data Processing Pipeline
d. Processed Data → BigQuery Storage
e. BigQuery → Metabase Dashboard

##### Component Details

Python Application (Core):
- enonic_data_api.py (Data Fetching)
- parse_json.py (Data Processing)
- bygg_df.py (DataFrame Construction)
- bq_tabell_jobber.py (BigQuery Operations)
- gcs_api.py (Cloud Storage Operations)

##### Infrastructure Layer
- NAIS Platform (NAV's Kubernetes Platform)
- Secrets Management
- Logging System (Kibana)

##### Labels
- Component names
- Key technologies
- Data flow directions
- Cronjob schedule indication

##### Security Layer
- Secret management integration
- Secure connections

##### Monitoring
- Kibana logging flow
- Metabase dashboard connection

##### Development vs Production
- Adding a small subset showing local development setup
- Environment-specific configurations

Level of Detail:
- Data processing pipeline
- Infrastructure setup
- Local development environment

##### Security and monitoring aspects

This system design should provide a clear understanding of how the application collects, processes, and visualizes content statistics from nav.no while maintaining appropriate security and monitoring capabilities.

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph Kubernetes["Kubernetes/NAIS Platform"]
        CRON[("Kubernetes CronJob")]:::k8s
        POD[("Python Application Pod")]:::k8s
    end

    subgraph PythonApp["Python Application Container"]
        MAIN["Main Application"]:::core
        ENONIC_API["Enonic Data API"]:::module
        PARSE["JSON Parser"]:::module
        DF["DataFrame Builder"]:::module
        BQ_OPS["BigQuery Operations"]:::module
        GCS_OPS["GCS Operations"]:::module
        URL_MGR["URL Manager"]:::module
        SCHEMA["Data Schema"]:::module
    end

    subgraph External["External Services"]
        ENONIC["Enonic XP CMS"]:::external
        GCS["Google Cloud Storage"]:::storage
        BQ[(BigQuery)]:::storage
        META["Metabase Dashboard"]:::viz
    end

    subgraph Security["Security Layer"]
        SECRETS["Secrets Management"]:::security
    end

    subgraph Monitoring["Monitoring"]
        KIBANA["Kibana Logs"]:::monitoring
    end

    %% Connections
    CRON --> POD
    POD --> MAIN
    MAIN --> ENONIC_API
    MAIN --> URL_MGR
    ENONIC_API --> PARSE
    PARSE --> DF
    DF --> BQ_OPS
    DF --> GCS_OPS
    BQ_OPS --> BQ
    GCS_OPS --> GCS
    SCHEMA --> BQ_OPS
    
    ENONIC_API --> ENONIC
    BQ --> META
    
    POD -.-> KIBANA
    POD -.-> SECRETS

    %% Click Events
    click POD "https://github.com/navikt/innholdsoversikt/tree/main/Dockerfile"
    click MAIN "https://github.com/navikt/innholdsoversikt/blob/main/src/innholdsoversikt/main.py"
    click ENONIC_API "https://github.com/navikt/innholdsoversikt/blob/main/src/innholdsoversikt/enonic_data_api.py"
    click PARSE "https://github.com/navikt/innholdsoversikt/blob/main/src/innholdsoversikt/parse_json.py"
    click DF "https://github.com/navikt/innholdsoversikt/blob/main/src/innholdsoversikt/bygg_df.py"
    click BQ_OPS "https://github.com/navikt/innholdsoversikt/blob/main/src/innholdsoversikt/bq_tabell_jobber.py"
    click GCS_OPS "https://github.com/navikt/innholdsoversikt/blob/main/src/innholdsoversikt/gcs_api.py"
    click URL_MGR "https://github.com/navikt/innholdsoversikt/blob/main/src/innholdsoversikt/get_url.py"
    click SCHEMA "https://github.com/navikt/innholdsoversikt/blob/main/src/innholdsoversikt/schema_tabell.json"
    click Kubernetes "https://github.com/navikt/innholdsoversikt/blob/main/nais.yaml"

    %% Styles
    classDef k8s fill:#326CE5,stroke:#fff,stroke-width:2px,color:#fff
    classDef core fill:#1F70C1,stroke:#fff,stroke-width:2px,color:#fff
    classDef module fill:#3498DB,stroke:#fff,stroke-width:1px,color:#fff
    classDef external fill:#27AE60,stroke:#fff,stroke-width:2px,color:#fff
    classDef storage fill:#F1C40F,stroke:#fff,stroke-width:2px,color:#000
    classDef security fill:#E74C3C,stroke:#fff,stroke-width:2px,color:#fff
    classDef monitoring fill:#8E44AD,stroke:#fff,stroke-width:2px,color:#fff
    classDef viz fill:#2ECC71,stroke:#fff,stroke-width:2px,color:#fff

    %% Legend
    subgraph Legend
        L1[Kubernetes Components]:::k8s
        L2[Core Application]:::core
        L3[Application Modules]:::module
        L4[External Services]:::external
        L5[Data Storage]:::storage
        L6[Security]:::security
        L7[Monitoring]:::monitoring
    end
```
