# cavart/revenue-decay-analyser architecture

> Architecture diagram of the GitHub repository cavart/revenue-decay-analyser, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-02-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/cavart/revenue-decay-analyser
- Repository: https://github.com/cavart/revenue-decay-analyser

## Overview

##### Project type + purpose (what you’re diagramming)
This repository is a **Python data/analytics pipeline tool** (not a long-running service). Its purpose is to help SEO teams detect **page-level organic “revenue/click decay”** by:
- Extracting monthly data from **GA4** and **GSC**
- Building a **historical panel** from stored monthly chunks
- Applying **de-seasonalization** (by `page_type × month_of_year`)
- Computing **robust trend slopes** (Theil–Sen with fallback)
- Producing **decay risk scores**
- Exporting **charts + PPTX “attention pack”**

The “system” is therefore best represented as a **batch pipeline** with multiple steps, inputs/outputs, and external dependencies (Google APIs + local storage + exported artifacts).

##### File tree cues → architectural decomposition
Key structural signals:

- `scripts/`
Thin CLI entrypoints for operators (“run X”). These are your **execution triggers**.

- `src/pipelines/`
The actual pipeline implementations; each file corresponds to one stage:
- `ga4_organic_seo_extract.py`
- `gsc_organic_page_extract.py`
- `revenue_decay_chunked_pull.py`
- `revenue_decay_model_input.py`
- `revenue_decay_scoring.py`
- `revenue_decay_chart_pack.py`
- `revenue_decay_monthly_run.py` (the orchestrator)

- `src/common/config.py`

- `.env.example` + README mention of `secrets/`
Indicates environment-driven config and credential files. Even if `secrets/` isn’t in the repo tree, it’s an **important runtime dependency**.

- `docs/*.md`
Operational manuals. Use them to confirm intermediate artifacts and the operator workflow; they often imply storage formats/locations and run sequences.

##### What the README implies about runtime flow
The README’s “One Command (Monthly)” establishes a canonical orchestrated run:

`scripts/run_revenue_decay_monthly_run.py --history-start-date ...`

##### Scoring stage reads Model Input → outputs “Decay Risk Scores”
10. **Chart Pack stage reads Scores (+ possibly Model Input/Historical Panel) → outputs Charts + PPTX**

##### Architectural patterns/design principles visible here

- **Pipeline / staged batch processing** (each stage is its own module)
- **Orchestration via a single “monthly run”** entrypoint
- **Idempotent-ish incremental data approach** (“stored monthly chunks” suggests append/refresh strategy)
- **Separation of concerns**:
- extraction (API I/O)
- data assembly (panel build)
- transformation (de-seasonalize)
- analytics (robust slope)
- presentation (charts/PPTX)
- **Configuration boundary** (environment + secrets decoupled from code)

##### Relevant technologies/frameworks/libraries
- **Python** runtime
- **Google Analytics 4 (GA4) API**
- **Google Search Console (GSC) API**
- Statistical method: **Theil–Sen slope estimator** (+ fallback)
- Outputs: **charts** and **PPTX** generation (label generically if library not known)

##### Practical “don’t overthink it” approach: separate into many components
To maximize detail without getting lost:

## Diagram (Mermaid)

```mermaid
flowchart TD

%% =========================
%% Execution / Control Layer
%% =========================
subgraph "Execution Triggers (Batch CLI)"
direction TB
OP["Analyst / Scheduler\n(manual run, cron, GitHub Actions, etc.)"]:::external

subgraph "scripts/ (thin CLI entrypoints)"
direction TB
CLI_MONTHLY["run_revenue_decay_monthly_run.py\n(one-command monthly run)"]:::cli
CLI_GA4["run_ga4_organic_seo_extract.py\n(GA4 extraction)"]:::cli
CLI_GSC["run_gsc_organic_page_extract.py\n(GSC extraction)"]:::cli
CLI_CHUNK["run_revenue_decay_chunked_pull.py\n(rebuild historical panel)"]:::cli
CLI_MODEL["run_revenue_decay_model_input.py\n(feature engineering)"]:::cli
CLI_SCORE["run_revenue_decay_scoring.py\n(scoring)"]:::cli
CLI_PACK["run_revenue_decay_chart_pack.py\n(charts + PPTX)"]:::cli
end
end

OP -->|"triggers"| CLI_MONTHLY
OP -->|"optional_manual_run"| CLI_GA4
OP -->|"optional_manual_run"| CLI_GSC
OP -->|"optional_manual_run"| CLI_CHUNK
OP -->|"optional_manual_run"| CLI_MODEL
OP -->|"optional_manual_run"| CLI_SCORE
OP -->|"optional_manual_run"| CLI_PACK

%% =========================
%% Orchestration
%% =========================
subgraph "Orchestration"
direction TB
ORCH["revenue_decay_monthly_run.py\n(orchestrates stage sequence)"]:::orchestrator
end

CLI_MONTHLY -.->|"calls"| ORCH

%% =========================
%% Pipeline Stages
%% =========================
subgraph "Pipeline Stages (src/pipelines/)"
direction TB

GA4_STAGE["ga4_organic_seo_extract.py\n(GA4 organic page metrics extract)"]:::stage
GSC_STAGE["gsc_organic_page_extract.py\n(GSC organic page metrics extract)"]:::stage
CHUNK_STAGE["revenue_decay_chunked_pull.py\n(read stored monthly chunks → rebuild panel)"]:::stage
MODEL_STAGE["revenue_decay_model_input.py\n(de-seasonalize by page_type×month_of_year)"]:::stage
SCORE_STAGE["revenue_decay_scoring.py\n(Theil–Sen slope + fallback → risk score)"]:::stage
PACK_STAGE["revenue_decay_chart_pack.py\n(charts + PPTX attention pack)"]:::stage

end

ORCH -.->|"step_1"| GA4_STAGE
ORCH -.->|"step_2"| GSC_STAGE
ORCH -.->|"step_3"| CHUNK_STAGE
ORCH -.->|"step_4"| MODEL_STAGE
ORCH -.->|"step_5"| SCORE_STAGE
ORCH -.->|"step_6"| PACK_STAGE

%% Also allow direct CLI execution of individual stages
CLI_GA4 -.->|"calls"| GA4_STAGE
CLI_GSC -.->|"calls"| GSC_STAGE
CLI_CHUNK -.->|"calls"| CHUNK_STAGE
CLI_MODEL -.->|"calls"| MODEL_STAGE
CLI_SCORE -.->|"calls"| SCORE_STAGE
CLI_PACK -.->|"calls"| PACK_STAGE

%% =========================
%% External Systems
%% =========================
subgraph "External Google Systems"
direction TB
GA4_API(["Google Analytics 4 API"]):::external
GSC_API(["Google Search Console API"]):::external
end

GA4_STAGE -->|"requests_metrics"| GA4_API
GSC_STAGE -->|"requests_metrics"| GSC_API

%% =========================
%% Local Storage / Artifacts
%% =========================
subgraph "Local Environment (Artifacts + Persistent Storage)"
direction TB

CHUNK_STORE[("Monthly Chunks Store\n(local files: CSV/Parquet/etc.)")]:::datastore

A_GA4["Monthly GA4 Chunk\n(page-level organic metrics)"]:::artifact
A_GSC["Monthly GSC Chunk\n(page-level organic metrics)"]:::artifact

A_PANEL["Historical Panel Dataset\n(reconstructed from chunks)"]:::artifact
A_MODEL["Model Input Dataset\n(de-seasonalized features)"]:::artifact
A_SCORES["Decay Risk Scores Dataset\n(page-level risk scoring)"]:::artifact

OUT_CHARTS["Charts Output Directory"]:::output
OUT_PPTX["PPTX Attention Pack"]:::output
end

GA4_STAGE -->|"writes"| A_GA4
GSC_STAGE -->|"writes"| A_GSC
A_GA4 -->|"persist"| CHUNK_STORE
A_GSC -->|"persist"| CHUNK_STORE

CHUNK_STORE -->|"reads_monthly_chunks"| CHUNK_STAGE
CHUNK_STAGE -->|"outputs"| A_PANEL
MODEL_STAGE -->|"reads"| A_PANEL
MODEL_STAGE -->|"outputs"| A_MODEL
SCORE_STAGE -->|"reads"| A_MODEL
SCORE_STAGE -->|"outputs"| A_SCORES

PACK_STAGE -->|"reads"| A_SCORES
PACK_STAGE -->|"optional_reads"| A_MODEL
PACK_STAGE -->|"optional_reads"| A_PANEL
PACK_STAGE -->|"outputs"| OUT_CHARTS
PACK_STAGE -->|"outputs"| OUT_PPTX

%% =========================
%% Config + Secrets (cross-cutting)
%% =========================
subgraph "Config + Secrets Boundary"
direction TB
ENV[".env\n(runtime env vars; template in .env.example)"]:::config
CONFIG["src/common/config.py\n(config loader: paths, params, credentials wiring)"]:::config
SECRETS["secrets/\n(Google credentials JSON files)"]:::secret
REQ["requirements.decay_analyser.txt\n(Python dependencies)"]:::meta
end

ENV -->|"loaded_by"| CONFIG
SECRETS -->|"used_by"| CONFIG

CONFIG -.->|"config"| ORCH
CONFIG -.->|"config"| GA4_STAGE
CONFIG -.->|"config"| GSC_STAGE
CONFIG -.->|"config"| CHUNK_STAGE
CONFIG -.->|"config"| MODEL_STAGE
CONFIG -.->|"config"| SCORE_STAGE
CONFIG -.->|"config"| PACK_STAGE

REQ -.->|"installs_libs_for"| GA4_STAGE
REQ -.->|"installs_libs_for"| GSC_STAGE
REQ -.->|"installs_libs_for"| PACK_STAGE
REQ -.->|"installs_libs_for"| SCORE_STAGE

%% =========================
%% Documentation / Runbooks
%% =========================
subgraph "Operator Documentation (docs/)"
direction TB
DOCS_DIR["docs/"]:::docs
DOC_MONTHLY["REVENUE_DECAY_MONTHLY_RUN_USAGE.md"]:::docs
DOC_CHUNK["REVENUE_DECAY_CHUNKED_PULL_USAGE.md"]:::docs
DOC_MODEL["REVENUE_DECAY_MODEL_INPUT_USAGE.md"]:::docs
DOC_SCORE["REVENUE_DECAY_SCORING_USAGE.md"]:::docs
DOC_PACK["REVENUE_DECAY_CHART_PACK_USAGE.md"]:::docs
DOC_MANUAL["REVENUE_DECAY_ANALYSER_MANUAL.md"]:::docs
end

OP -.->|"follows"| DOC_MONTHLY
OP -.->|"follows"| DOC_CHUNK
OP -.->|"follows"| DOC_MODEL
OP -.->|"follows"| DOC_SCORE
OP -.->|"follows"| DOC_PACK
OP -.->|"reads"| DOC_MANUAL

DOCS_DIR --> DOC_MONTHLY
DOCS_DIR --> DOC_CHUNK
DOCS_DIR --> DOC_MODEL
DOCS_DIR --> DOC_SCORE
DOCS_DIR --> DOC_PACK
DOCS_DIR --> DOC_MANUAL

%% =========================
%% Click Events (component_mapping)
%% =========================
click CLI_MONTHLY "https://github.com/cavart/revenue-decay-analyser/blob/main/scripts/run_revenue_decay_monthly_run.py"
click CLI_GA4 "https://github.com/cavart/revenue-decay-analyser/blob/main/scripts/run_ga4_organic_seo_extract.py"
click CLI_GSC "https://github.com/cavart/revenue-decay-analyser/blob/main/scripts/run_gsc_organic_page_extract.py"
click CLI_CHUNK "https://github.com/cavart/revenue-decay-analyser/blob/main/scripts/run_revenue_decay_chunked_pull.py"
click CLI_MODEL "https://github.com/cavart/revenue-decay-analyser/blob/main/scripts/run_revenue_decay_model_input.py"
click CLI_SCORE "https://github.com/cavart/revenue-decay-analyser/blob/main/scripts/run_revenue_decay_scoring.py"
click CLI_PACK "https://github.com/cavart/revenue-decay-analyser/blob/main/scripts/run_revenue_decay_chart_pack.py"
click ORCH "https://github.com/cavart/revenue-decay-analyser/blob/main/src/pipelines/revenue_decay_monthly_run.py"
click GA4_STAGE "https://github.com/cavart/revenue-decay-analyser/blob/main/src/pipelines/ga4_organic_seo_extract.py"
click GSC_STAGE "https://github.com/cavart/revenue-decay-analyser/blob/main/src/pipelines/gsc_organic_page_extract.py"
click CHUNK_STAGE "https://github.com/cavart/revenue-decay-analyser/blob/main/src/pipelines/revenue_decay_chunked_pull.py"
click MODEL_STAGE "https://github.com/cavart/revenue-decay-analyser/blob/main/src/pipelines/revenue_decay_model_input.py"
click SCORE_STAGE "https://github.com/cavart/revenue-decay-analyser/blob/main/src/pipelines/revenue_decay_scoring.py"
click PACK_STAGE "https://github.com/cavart/revenue-decay-analyser/blob/main/src/pipelines/revenue_decay_chart_pack.py"
click CONFIG "https://github.com/cavart/revenue-decay-analyser/blob/main/src/common/config.py"
click ENV "https://github.com/cavart/revenue-decay-analyser/blob/main/.env.example"
click DOCS_DIR "https://github.com/cavart/revenue-decay-analyser/tree/main/docs/"
click DOC_MONTHLY "https://github.com/cavart/revenue-decay-analyser/blob/main/docs/REVENUE_DECAY_MONTHLY_RUN_USAGE.md"
click DOC_CHUNK "https://github.com/cavart/revenue-decay-analyser/blob/main/docs/REVENUE_DECAY_CHUNKED_PULL_USAGE.md"
click DOC_MODEL "https://github.com/cavart/revenue-decay-analyser/blob/main/docs/REVENUE_DECAY_MODEL_INPUT_USAGE.md"
click DOC_SCORE "https://github.com/cavart/revenue-decay-analyser/blob/main/docs/REVENUE_DECAY_SCORING_USAGE.md"
click DOC_PACK "https://github.com/cavart/revenue-decay-analyser/blob/main/docs/REVENUE_DECAY_CHART_PACK_USAGE.md"
click DOC_MANUAL "https://github.com/cavart/revenue-decay-analyser/blob/main/docs/REVENUE_DECAY_ANALYSER_MANUAL.md"
click REQ "https://github.com/cavart/revenue-decay-analyser/blob/main/requirements.decay_analyser.txt"
click CLI_MONTHLY "https://github.com/cavart/revenue-decay-analyser/tree/main/scripts/"
click CLI_GA4 "https://github.com/cavart/revenue-decay-analyser/tree/main/scripts/"
click CLI_GSC "https://github.com/cavart/revenue-decay-analyser/tree/main/scripts/"
click CLI_CHUNK "https://github.com/cavart/revenue-decay-analyser/tree/main/scripts/"
click CLI_MODEL "https://github.com/cavart/revenue-decay-analyser/tree/main/scripts/"
click CLI_SCORE "https://github.com/cavart/revenue-decay-analyser/tree/main/scripts/"
click CLI_PACK "https://github.com/cavart/revenue-decay-analyser/tree/main/scripts/"

%% =========================
%% Styles
%% =========================
classDef external fill:#E3F2FD,stroke:#1565C0,stroke-width:1.5px,color:#0D47A1
classDef cli fill:#FFF3E0,stroke:#EF6C00,stroke-width:1.5px,color:#4E342E
classDef orchestrator fill:#E8F5E9,stroke:#2E7D32,stroke-width:2px,color:#1B5E20
classDef stage fill:#E8F5E9,stroke:#2E7D32,stroke-width:1.5px,color:#1B5E20
classDef datastore fill:#F3E5F5,stroke:#6A1B9A,stroke-width:2px,color:#4A148C
classDef artifact fill:#FCE4EC,stroke:#AD1457,stroke-width:1.5px,color:#880E4F
classDef output fill:#E0F7FA,stroke:#00838F,stroke-width:2px,color:#006064
classDef config fill:#FFFDE7,stroke:#F9A825,stroke-width:1.5px,color:#6D4C41
classDef secret fill:#FFEBEE,stroke:#C62828,stroke-width:2px,color:#B71C1C
classDef docs fill:#ECEFF1,stroke:#455A64,stroke-width:1.5px,color:#263238
classDef meta fill:#F1F8E9,stroke:#558B2F,stroke-width:1.5px,color:#33691E
```
