How eilonh/s3crets_scanner works
Main Components
• CLI/Entry Point: The main.py file acts as the main entry point where command-line arguments (like AWS profile, scanner role, scan method, and scan window) are parsed.
• AWS Interaction Module: The aws.py file likely encapsulates interactions with AWS S3 (e.g., listing buckets, retrieving bucket metadata, downloading files). This is a central component that interfaces with AWS.
• CSV Accounts Handler: Under the csv directory, the accounts.csv file provides a list of AWS accounts. This module/component may be used to iterate over different AWS accounts.
• File Scanning and Rules: Although there isn’t a dedicated module solely for scanning, the scanning process leverages truffleHog3 via command execution from within the tool. Additionally, the rules.yml file represents configurable scanning rules or criteria for secret detection.
• Logging and Configuration: The logger_config.py configures the logging mechanism (with logs written to logger.log, as mentioned in the README). This is critical for tracking operations.
• Utility Functions & Scripts: The utils.py and scripts.py files likely contain helper functions and additional automation tasks that support the scanning process (e.g., file cleanup, scanning orchestration).
Relationships and Interactions
• The CLI (main.py) parses user parameters and initiates the scan process.
• main.py calls functions in aws.py to authenticate with AWS, list public buckets, and then iterate over the files.
• For each identified file, the tool downloads them locally and then triggers the scanning process using truffleHog3 (external dependency). Once scanning is complete, files are deleted.
• Processing of scanning rules as defined in the rules.yml file is integrated, possibly through utility functions (utils.py).
• If the CSV based account selection is used, main.py or an associated module reads from csv/accounts.csv to iterate over provided AWS accounts.
• All significant actions are logged using settings from logger_config.py.
Architectural Patterns & Design Principles
• Modularity: Separate modules for AWS interactions, logging, file handling, and utility functions help in maintaining separation of concerns.
• External Dependencies: The design leverages truffleHog3 for secret scanning and uses configuration files (CSV and rules.yml) for dynamic behavior.
• Simple execute-and-cleanup workflow: Designed to be run as a scheduled task or on-demand, processing happens in a serial workflow: list -> download -> scan -> delete.
Relevant Technologies and Frameworks
• Python (3.6+)
• truffleHog3 as the scanning tool
• AWS S3 for storage operations
• CSV for account data configuration
• YAML configuration for rules (rules.yml)
• Standard logging library (configured in logger_config.py)