How nopals-sub/dir-scan works
Main Components
• CLI/Input Handler: The entry point where the user provides the domain URL and optional flags (e.g., resume scan).
• URL/Protocol Validator: A component that confirms the URL format and adds "http://" if needed.
• Recursive Scanner Engine: The core component that implements the recursive directory and file scanning.
• Duplicate Filtering Module: A layer/functionality that ensures duplicates are filtered out before being output.
• Output Handler: The mechanism to display or save the results (including support for stopping/resuming the scan).
Out the Relationships and Interactions
• The user invokes the tool from the command line providing a domain URL.
• The URL/Protocol Validator processes the input, ensuring a valid HTTP/HTTPS URL.
• Next, the Recursive Scanner Engine uses the validated URL to start scanning directories and files recursively.
• Within this scanning process, the Duplicate Filtering module is used to check and eliminate duplicates.
• Finally, the Output Handler manages how the results are displayed to or stored for the user.
Architectural Patterns and Principles
• Modularity & Separation of Concerns: Even though implemented in a single file, conceptually separate the program into distinct responsibilities (input, validation, scanning, filtering, and output).
• Recursive Processing: Emphasize the recursive nature of the directory scan and the handling of multiple protocols.
• Error Handling & Interruption Management: The design should allow for resume or stop functionality if the scan is interrupted.
Specify the Technologies and Frameworks
• Go (Golang) is the core language, so note that the architecture leverages Go's concurrency and even possibly its context cancellation features for interruptible scanning.
• While no external libraries are specified, consider the built-in Go libraries (like net/http, os, etc.) which support these functionalities.