# jupyter/security architecture

> Architecture diagram of the GitHub repository jupyter/security, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jupyter/security
- Repository: https://github.com/jupyter/security

## Overview

##### Project type and purpose
- This is not a classic web app or microservice suite but a “governance & documentation” repository for the Jupyter Security Subproject.
- It centralizes:
- Documentation (rst/md) on security topics (docs/)
- Meeting archives (meetings/)
- Lightweight Python automation tools (tools/) for gathering security metrics and reports
- A JSON state file (last_user_activity.json) tracking activity

##### File structure
- Top-level:
- docs/ – reStructuredText guides & FAQ
- meetings/ – chronological meeting notes in Markdown
- tools/ – Python scripts to interact with external services
- last_user_activity.json – state file updated by tools
- Standard repo metadata (.gitignore, LICENSE, README.md)
- There is no separate “frontend/backend” code; the “build” is a docs site, and the “runtime” is a set of Python scripts run on demand or via CI.

##### Extract insights from README
- The README underscores this as an entirely public security knowledge base and points at private‐disclosure channels.
- No mention of a web server or database—implying a static site (Sphinx) plus scripts calling external APIs (GitHub, 1Password).

##### Main components and how to represent them
a. Repository (central hub)
b. Docs build pipeline
– Source: docs/*.rst + meetings/*.md + JSON reports
– Tool: Sphinx (or equivalent) → static HTML site → hosted on jupyter.org
c. Automation tools (tools/*.py)
– Scripts:
- all_repos.py
- last_user_activity.py → updates last_user_activity.json
- private_sec_report.py / private-sec-reporting.py → aggregate & format security data
- tide.py → integrates with GitHub org
– Triggered via CLI or CI (e.g., GitHub Actions)
d. Data store
– last_user_activity.json (local file in repo)
e. External services
– GitHub API (for repo stats, issues, PRs)
– 1Password (private credential store)
– Jupyter Software Steering Council and broader org/webhooks
f. Deployment
– CI/CD pipeline that runs tools, commits updates, builds docs, and pushes the site

##### Relationships and data flows
- Tools update last_user_activity.json → commit back to Repository
- Docs Build Pipeline pulls in: docs/, meetings/, last_user_activity.json → generates HTML → deployed to Hosting
- CI/CD orchestrates Tools and Docs Build, merging artifacts back into the repo

##### Architectural patterns & design principles
- ETL-style data pipeline for security metrics
- Static-site generation for docs (Sphinx)
- Separation of concerns:
- Content (RST/MD)
- Automation (Python scripts)
- Deployment (CI/CD)
- Minimal state (single JSON file) to simplify versioning

##### Final advice
- Model the data flow: scripts fetch, transform, persist, then docs consume.
- Where human input happens (authoring meetings/docs) versus automated steps.
- Keep it modular and extensible—future scripts or docs should slot into your pipeline cleanly.
- The goal is a single‐page overview that any new contributor or stakeholder can read in 60 seconds and understand how content, code, and external systems tie together.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% External Services
    github["GitHub API"]:::external
    onepass["1Password"]:::external
    jssc["Jupyter SSC"]:::external

    %% CI/CD Orchestration
    subgraph "CI/CD Pipeline" 
        style CI_CD stroke-dasharray: 5 5
        CI_CD["CI/CD (GitHub Actions)"]:::ci
    end

    %% Repository Structure
    subgraph "Repository (Central Hub)"
        direction TB
        docs["docs/"]:::code
        meetings["meetings/"]:::code
        tools_dir["tools/"]:::code
        json["last_user_activity.json"]:::data
        readme["README.md"]:::code
        ignore1["private_sec_ignore.txt"]:::code
        ignore2["psc_ignore.txt"]:::code
    end

    %% Automation Tools
    subgraph "tools/"
        direction TB
        allrepos["all_repos.py"]:::code
        lastupdate["last_user_activity.py"]:::code
        privsec_report["private-sec-reporting.py"]:::code
        privsec_alt["private_sec_report.py"]:::code
        tide["tide.py"]:::code
    end

    %% Docs Build and Hosting
    docsbuild["Docs Build (Sphinx)"]:::code
    hosting["Hosting (jupyter.org)"]:::external

    %% Flows: CI/CD triggers tools and docs build
    CI_CD -->|runs| allrepos
    CI_CD -->|runs| lastupdate
    CI_CD -->|runs| privsec_report
    CI_CD -->|runs| privsec_alt
    CI_CD -->|runs| tide
    CI_CD -->|triggers| docsbuild

    %% External Services to Tools
    github --> allrepos
    github --> tide
    onepass --> privsec_report
    onepass --> privsec_alt
    jssc --> tide

    %% Tools output to Data Store
    allrepos --> json
    lastupdate --> json
    privsec_report --> json
    privsec_alt --> json
    tide --> json

    %% Data Store commits back to Repository
    json -->|commit updates| docsbuild
    json -->|persist| docsbuild

    %% Docs Build pipeline sources
    docs --> docsbuild
    meetings --> docsbuild
    json --> docsbuild

    %% Docs Build to Hosting
    docsbuild -->|deploy| hosting

    %% Click Events
    click docs "https://github.com/jupyter/security/tree/main/docs/"
    click meetings "https://github.com/jupyter/security/tree/main/meetings/"
    click tools_dir "https://github.com/jupyter/security/tree/main/tools/"
    click allrepos "https://github.com/jupyter/security/blob/main/tools/all_repos.py"
    click lastupdate "https://github.com/jupyter/security/blob/main/tools/last_user_activity.py"
    click privsec_report "https://github.com/jupyter/security/blob/main/tools/private-sec-reporting.py"
    click privsec_alt "https://github.com/jupyter/security/blob/main/tools/private_sec_report.py"
    click tide "https://github.com/jupyter/security/blob/main/tools/tide.py"
    click json "https://github.com/jupyter/security/blob/main/last_user_activity.json"
    click readme "https://github.com/jupyter/security/blob/main/README.md"
    click ignore1 "https://github.com/jupyter/security/blob/main/private_sec_ignore.txt"
    click ignore2 "https://github.com/jupyter/security/blob/main/psc_ignore.txt"

    %% Styles
    classDef code fill:#E0F7FA,stroke:#006064,color:#006064;
    classDef data fill:#E8F5E9,stroke:#2E7D32,color:#2E7D32;
    classDef external fill:#FFF3E0,stroke:#EF6C00,color:#EF6C00;
    classDef ci fill:none,stroke:#9E9E9E,color:#616161,stroke-dasharray: 5 5;
```
