How parklez/price-monitor-alerter works
Project Type and Purpose
• Project Type: It is a monitoring and notification application.
• Purpose: The system periodically (hourly) scrapes product prices from several e-commerce websites and logs any changes into a CSV file (prices.csv). It also triggers alerts via a Telegram bot when a price discount is detected.
• Environment: It’s automatically executed via a GitHub Actions workflow, indicating an automated scheduled job rather than a continuously running service.
File Structure
• Top-Level Directories/Files:
• .github/workflows: Contains the cronjob.yml file used to schedule the hourly run.
• monitor: This core module holds the main program (main.py) and the scraper utility (scraper.py). The “sites” subdirectory includes individual scraper implementations for different websites (angeloni, dell, goimports, lg, pontofrio).
• monitor/utils: Contains helper modules like safe_writer.py (for handling CSV writes safely) and telegram_notifier.py (for sending alerts).
• docs: Provides the web page and supporting resources (JavaScript and CSS) for historical data analysis.
• tools: Includes auxiliary scripts for tasks (e.g., downloading pages or reading Telegram messages), which could be used for testing or troubleshooting.
• prices.csv: The data store for the logged price changes.
• Configuration Files: .env-example, .gitignore, requirements.txt, and other metadata files suggest clear separation of configuration, code, and documentation.
README for Key Insights
• The README highlights critical functions:
• Data Logging: Any price changes are recorded in the CSV file, and GitHub Actions commits these updates.
• Telegram Integration: A notification is sent via a Telegram bot to alert users if a discount is detected.
• Supported Websites: The README explicitly lists the websites for which scraping is implemented, matching the modules in monitor/sites.
• Deployment Setup: Environment variables (like PRODUCTS and TELEGRAM_API_TOKEN) are crucial and are set in the GitHub Actions secrets.
a. Main Components:
• GitHub Actions Scheduler:
• Represents the automation layer that triggers the system on an hourly basis.
• Monitor Engine:
• A “Main Scraper” component (main.py) that orchestrates the overall process.
• Site Scrapers:
• CSV Logger:
• A component labeled “CSV Logger (prices.csv)” that receives data from the monitor engine to record historical price changes.
• Telegram Notifier:
• A module/component for “Telegram Alerts” (using telegram_notifier.py) that is triggered when discounts are detected.
• Utility Tools:
• An optional section representing the “Tools” directory for supporting scripts and testing utilities.
• Historical Data Page:
• A separate component labeled “Docs/Webpage” that represents the visualization of historical data.
b. Relationships and Interactions:
c. Architectural Patterns & Design Principles:
• Modular Design: Each site’s scraper is implemented as its own module (under monitor/sites), making it easy to add or modify scraping rules.
• Separation of Concerns: The system cleanly separates concerns (data scraping, logging, notifying, and visualization).
• Automation & Scheduling: The GitHub Actions workflow is used to periodically trigger the process.
• Extensibility: With separate utilities and tool modules, new features (additional site support, alternative notifications) can be integrated with minimal disruption.
d. Relevant Technologies & Frameworks:
• Python: The programming language used for the monitor’s core logic.
• GitHub Actions: For CI/CD and scheduling.
• Telegram API: For sending bot messages (using the token provided in TELEGRAM_API_TOKEN).
• CSV Files: For simple data storage and logging.
• Web technologies (HTML, JavaScript, CSS): For displaying historical data analysis.