How mikemichel/gitlab-pipeline-cleanup-tool works
Project Type and Purpose
• This is an open-source command-line tool (Python script) whose sole responsibility is to connect to a GitLab instance, enumerate pipelines, delete those older than a given threshold, and preserve a configured number of recent successful runs. It is not a multi-tier web application but a single-entrypoint CLI utility.
File Structure Analysis
• Top level:
• README.md – describes behavior, parameters, logging, examples.
• delete_old_pipelines.py – the entire implementation lives here.
• Runtime artifacts:
• A logs/ directory (created at runtime) holds timestamped log files.
• External/implicit elements:
• GitLab REST API (external service)
• Local filesystem (for logs)
• Python runtime (3.11+) and the requests library
Key Components
a. CLI Interface
• Entry point: parameter parsing (argparse)
• Validates required flags (--gitlab-host, --pat, --older-than-days, etc.)
b. Core Orchestrator
• Orchestrates the sequence:
1. Projects (optionally limited by --max-projects or --project-id)
2. For each project:
• Fetch pipelines via GitLab API
• Filter by age and status
• Pipelines to delete vs. keep
• Issue delete calls
• Handles rate limiting, archived-project unarchiving/re-archiving
c. GitLab API Client
• Wraps HTTP calls (using requests) to:
• GET /projects
• GET /projects/:id/pipelines
• DELETE /projects/:id/pipelines/:pipeline_id
• Handles authentication (Personal Access Token or Job Token)
d. Logging Subsystem
• Console logger (stdout)
• File logger (writes pipeline_cleanup_YYYYMMDD_HHMMSS.log into logs/)
• Records every major event: API calls, errors, rate-limit waits, summary counts
e. External Services & Data Stores
• GitLab instance (external REST API)
• Local filesystem (logs directory)
Relationships and Interactions
• User invokes delete_old_pipelines.py → CLI Interface
• CLI Interface → Orchestrator (passes parsed config)
• Orchestrator ↔ GitLab API Client → GitLab REST API
• Orchestrator → Logging Subsystem → Console & File system
• Orchestrator iterates over projects/pipelines, issues delete operations via API Client
• Final summary logged and exit
Architectural Patterns and Design Principles
• Layered approach (even in a single script):
• Presentation/CLI layer
• Application/orchestration layer
• Integration layer (API client)
• Infrastructure layer (logging, file system)
• Single Responsibility: each logical block (arg parsing, API calls, cleanup logic, logging) can be separated into functions or modules if the tool grows.
• Idempotence & Dry-run support: design cleanup logic so that a “dry run” flag only simulates API calls without state change.
Technologies and Frameworks
• Python 3.11+
• requests library for HTTP interactions
• argparse (or click) for parameter parsing
• Standard logging module for console/file output
• Docker (for development environment)
Essential Elements to Capture
• Entrypoint and configuration parsing
• Core loop: list projects → list pipelines → filter → delete
• Rate limit & archive handling as sub-steps of the Orchestrator
• Logging to both console and file
• External dependency: GitLab REST API