How greogory/hibp-checker works
Project Type and Purpose
• This is an open-source security automation tool (“HIBP Comprehensive Checker”) written in Python and Bash, with an optional Flask-based web dashboard.
• Purpose: Automate Have I Been Pwned (HIBP) breach and credential-stuffing checks for one or many emails and passwords, produce reports (text/JSON/CSV), and provide a real-time dashboard. It supports native Linux scheduling (systemd/cron) and cross-platform Docker deployment.
File Structure Analysis
• Root scripts & binaries
• hibp_comprehensive_checker.py – core Python engine that calls HIBP APIs
• hibp_workflow.sh – Bash CLI wrapper for setup, check, schedule, email management
• quick_start.sh, multi_email_setup.sh – helper scripts for initial configuration
• Configuration & orchestration
• hibp_config.conf.example – sample config used by both Python and Bash scripts
• docker-compose.yml + docker-compose.scheduled.yml, Dockerfile – container builds and multi-service composition
• Web Dashboard (dashboard/)
• app.py – Flask application entry point
• templates/index.html – UI definition
• start-dashboard.* scripts & systemd/hibp-dashboard.service – service definitions for Linux
• Scheduling & automation
• systemd/ – timers and service units for Linux native scheduling
• scripts/ – cron/systemd setup helpers
• CI/CD & GitHub
• .github/workflows/ – auto-approve, auto-merge, security checks, Docker publish pipelines
• Documentation & guides
• README.md, DOCKER.md, WINDOWS_INSTALL.md, SCHEDULING.md, etc.
Key Components
a. Core Engine – Python script (hibp_comprehensive_checker.py) that:
• Reads config (env vars or hibp_config.conf)
• Calls external HIBP APIs (breaches, stealer logs, pastes, pwned-passwords)
• Generates reports to local filesystem
b. CLI Workflow – Bash wrapper (hibp_workflow.sh)
• Provides commands: setup, check, schedule, email management
• Orchestrates engine execution and triggers follow-up actions
c. Scheduler
• Linux: systemd services (.timer + .service) and cron (via scripts/setup-cron.sh)
• Docker: docker-compose.scheduled.yml runs scheduled container jobs
d. Web Dashboard
• Flask app (dashboard/app.py) serving real-time stats, logs, and report browser
• Reads report files and logs from shared volume or host paths
e. External Services
• Have I Been Pwned API v3 endpoints
f. Deployment Environments
• Native Linux (systemd, cron)
• Docker (multi-container compose)
• Windows/macOS via Docker
• WSL2 on Windows
Component Relationships and Data Flow
1. Scheduler → CLI Workflow: systemd/cron/docker-scheduler triggers hibp_workflow.sh schedule/check.
2. CLI Workflow → Core Engine: wrapper invokes Python script with parameters.
3. Core Engine ↔ HIBP API: REST calls to breach, paste, stealer log, and pwned-passwords endpoints.
4. Core Engine → Filesystem: writes scan results (reports) and logs.
5. Web Dashboard → Filesystem: reads results and logs; serves HTML UI to user’s browser.
6. User Browser ↔ Web Dashboard: HTTP interaction over port 5000 or container mapping.
7. CI/CD Pipelines (GitHub Actions) build Docker image and publish; can also trigger security checks via CLI.
Architectural Patterns and Principles
• Separation of Concerns:
• Core logic (Python) separated from orchestration (Bash) and presentation (Flask)
• Modularity & Composability:
• Independent scripts for setup, check, scheduling, dashboard
• Docker Compose for cross-platform deployment
• Infrastructure as Code:
• systemd units and cron scripts in version control
• GitHub Actions workflows for CI/CD
• 3-Tier-ish Structure:
• Presentation (Flask UI)
• Application Logic (Python/Bash)
• Data (reports/log files + external API)
Detail Level
• Conditional paths (e.g., Docker compose scheduled vs. manual cron).
• Shared volumes or host-container mounts for report/log access.