How cloud-discovery/prophet works
PROJECT TYPE & PURPOSE
• Prophet is a CLI-based tool that automates network host scanning, detailed resource information collection (for Linux, Windows, and VMware environments), and report generation.
• It is intended for cloud migration and disaster recovery research, validating that source hosts are ready for migration by comparing technical metrics.
ANALYSIS OF THE FILE STRUCTURE
• The top-level “prophet” directory contains the core modules:
– “cmd”: The command-line interface (CLI) entry point that orchestrates the tool’s workflow.
– “scanner”: Implements network scanning (using nmap and related functionality) to detect live hosts.
– “collector”: Contains submodules for host-specific data collection. There are separate subdirectories for Linux, Windows, and VMware collectors.
– “parser”: Normalizes and parses the collected host information for further analysis.
– “report”: Generates the final analysis report and supports packaging of the scanned/collected data.
– “cloud”: Handles integrations with cloud platforms (e.g., OpenStack) and provides HTTP request/error handling modules.
– “ansible_api”: Supports gathering details from Linux hosts using Ansible.
• Other directories like “docs,” “examples,” “tools,” and “etc” provide supplementary material, sample configuration files, deployment scripts (e.g., Dockerfile), and configuration settings.
READ ME INSIGHTS
• The README explains three main functionalities:
a. Scanning: Using “prophet-cli scan” with network input to produce a CSV of discovered hosts.
b. Collection: Using “prophet-cli collect” to connect to each host (via SSH, WMI, or VMware APIs) and retrieve detailed resource data.
c. Reporting: Using “prophet-cli report” to analyze gathered information and output a migration or disaster recovery readiness report.
• The project uses external tools and APIs (nmap, VMware API, Ansible for Linux, and WMI for Windows) to meet its functionality.
a. Main Components to Represent:
– User/Client Interface: The CLI (prophet/cmd/cli.py) that receives commands with subcommands (scan, collect, report).
– Input Data Sources:
• CSV files (scan_results, host collection inputs) and configuration files (e.g., pip.conf, repo configs).
– Scanning Module: The “prophet/scanner” module responsible for executing nmap scans and generating host discovery results.
– Collection Module: The “prophet/collector” module that handles detailed data collection. Split into separate submodules for Linux (using Ansible), Windows (via WMI), and VMware (via VMware API).
– Parsing Module: The “prophet/parser” module that processes and normalizes host data from different environments.
– Cloud Module: Connects to cloud APIs/externals if cloud resources need to be included (openstack.py etc.).
– Reporting Module: The “prophet/report” module for analyzing the collected data and generating detailed output reports, including final packaged files (.zip).
– Utility & Exception Handling: Modules like “prophet/utils.py” and “prophet/exceptions.py” which provide shared functionality/error management.
– Deployment Artifacts: Dockerfile and GitHub workflows that support containerization and CI/CD pipelines.
b. Relationships and Data Flow:
– The CLI component (front-end) dispatches commands to the scanning, collection, and reporting modules.
– In scanning, the input IP range is processed by the scanner which then outputs a CSV file.
– This CSV acts as the input for the collection module where, based on host OS type, the appropriate collector (Linux, Windows, or VMware) is invoked.
– Raw collected data is then passed to the parser to normalize the information before it is used by the report module to generate a final analysis output.
– Integration points with external systems (nmap, VMware APIs, Ansible, Windows WMI) and potential cloud resource modules are clearly marked.
– Additionally, logging (via prophet.log) and error/exception handling across modules should be visually indicated.
c. Architectural Patterns & Design Principles:
– Modular design: Each function (scanning, collection, parsing, reporting) is contained in its own module with specific responsibilities.
– Extensibility: Separate directories for each host type (Linux, Windows, VMware) indicate that additional collectors or parsers can be added with minimal changes to core flow.
– Reusability: Common utilities and configurations are centralized.
d. Relevant Technologies/Frameworks:
– Python is the primary language, with tools like Ansible (for Linux), nmap (for network scanning), and WMI (for Windows).
– VMware API use in the collector modules.
– Docker for containerized deployment as explained in the README and Dockerfile.
FINAL REMINDERS
• Do not shy away from detailing the interactions between all the key components. The goal is to capture even subtle dependencies (such as how collector modules depend on configuration provided via sample CSV files from the examples directory).
• Modular design: each module (scanning, collecting, parsing, reporting) should be represented as distinct components interacting in a well-defined sequence.