# gigachad80/grep-backurls architecture

> Architecture diagram of the GitHub repository gigachad80/grep-backurls, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/gigachad80/grep-backurls
- Repository: https://github.com/gigachad80/grep-backurls

## Overview

- Automatically discovering subdomains using an external tool (subfinder)
- Fetching historical URLs from those subdomains via another tool (waybackurls)
- Using a grep-like keyword filtering (driven by keywords defined in grep_keywords.txt) to extract potentially sensitive credentials

##### Project Type and Purpose
- This is a command line security automation tool written in Go.
- Its purpose is to automate the process of finding potential vulnerabilities or sensitive information by orchestrating multiple external utilities.

##### File Structure
- Top-level files: LICENSE, README.md, and a keywords file (grep_keywords.txt) exist.
- Main working logic is contained in the main.go file inside the "grep-backURLs" directory.
- The project makes use of a keywords file (present both in the root and within the directory) which suggests configuration or redundancy for ease of access.
- There are no multiple services or layers (i.e., no separate frontend/backend directories) – the tool acts as a single orchestrating binary.

##### Components
a. External Tools/Services:
- Subfinder: Depicted as an external service or process responsible for subdomain discovery.
- WaybackURLs: Another external dependency that provides historic URL data.
- Grep (or internal filtering logic invoking similar functionality): Used to scan URL outputs using keywords.

b. Internal Application Components:
- Main Orchestrator (main.go): This is the central component that calls and connects all subprocesses.
- Keyword Configuration: Represent the "grep_keywords.txt" file as a configuration/input source for filtering keywords.

c. Data Flow (Process Flow):
- Final output is the extracted “juicy” credentials or important links.

##### Architectural Patterns & Design Principles
- The design follows a simple pipeline architecture: Input → Processing Stages (subfinder, waybackurls, filter) → Output.
- Modular interactions with external command-line tools, ensuring each tool’s responsibility is clearly separated.
- Emphasis is on automation where one process’s output feeds directly as input into the next.

## Diagram (Mermaid)

```mermaid
graph TD
    USER["User Input"]:::input
    MAIN["Main Orchestrator"]:::internal
    SUBFINDER["Subfinder (external)"]:::external
    WAYBACK["WaybackURLs (external)"]:::external
    GREP["Grep Filter"]:::internal
    OUTPUT["Output (Sensitive Data)"]:::output

    KEY1["Keyword Config"]:::config
    KEY2["Keyword Config (Alternate)"]:::config

    USER -->|"start"| MAIN
    MAIN -->|"call_subfinder-get_subdomains"| SUBFINDER
    SUBFINDER -->|"return_subdomains"| MAIN
    MAIN -->|"pass_subdomains"| WAYBACK
    WAYBACK -->|"return_URLs"| MAIN
    MAIN -->|"perform_grep_filtering"| GREP
    KEY1 ---|"keywords"| GREP
    KEY2 ---|"keywords"| GREP
    GREP -->|"extract_sensitive_data"| OUTPUT

    click MAIN "https://github.com/gigachad80/grep-backurls/blob/main/grep-backURLs/main.go"
    click KEY1 "https://github.com/gigachad80/grep-backurls/blob/main/grep-backURLs/grep_keywords.txt"
    click KEY2 "https://github.com/gigachad80/grep-backurls/blob/main/grep_keywords.txt"

    classDef input fill:#D5F5E3,stroke:#27AE60,stroke-width:2px;
    classDef internal fill:#D1E8FF,stroke:#1F618D,stroke-width:2px;
    classDef external fill:#FADBD8,stroke:#922B21,stroke-width:2px;
    classDef config fill:#FCF3CF,stroke:#B7950B,stroke-width:2px;
    classDef output fill:#F9E79F,stroke:#B7950B,stroke-width:2px;
```
