# edoardottt/cariddi architecture

> Architecture diagram of the GitHub repository edoardottt/cariddi, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/edoardottt/cariddi
- Repository: https://github.com/edoardottt/cariddi

## Overview

Cariddi is a command-line web crawler and security scanner. Users pipe targets into the CLI, which reads scan options and starts a configured crawler; crawled responses and URLs are analyzed for secrets, endpoints, extensions, errors, and useful information. Results are deduplicated and emitted to console or files, with optional JSON output and HTTP-response storage. The README documents these capabilities; some detector and output wiring is therefore shown at the capability level where source excerpts are incomplete.

Main entry points: [`cmd/cariddi/main.go`](https://github.com/edoardottt/cariddi/blob/main/cmd/cariddi/main.go) (Scan coordinator), [`pkg/crawler/colly.go`](https://github.com/edoardottt/cariddi/blob/main/pkg/crawler/colly.go) (Colly crawler), [`pkg/crawler/scan.go`](https://github.com/edoardottt/cariddi/blob/main/pkg/crawler/scan.go) (Response analysis), [`pkg/output/output.go`](https://github.com/edoardottt/cariddi/blob/main/pkg/output/output.go) (Output formatting)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_cli["CLI and input"]
  node_stdin["Target input<br/>[input.go]"]
  node_flags["Options and validation<br/>[flags.go]"]
  node_main["Scan coordinator<br/>[main.go]"]
end

subgraph group_crawl["Crawling"]
  node_crawler["Colly crawler<br/>[colly.go]"]
  node_scan["Response analysis<br/>[scan.go]"]
  node_urlutil["URL utilities<br/>[url.go]"]
end

subgraph group_detect["Detection"]
  node_secrets["Secret detection<br/>[secrets.go]"]
  node_endpoints["Endpoint detection<br/>[endpoints.go]"]
  node_extensions["Extension detection<br/>[extensions.go]"]
  node_errors["Error detection<br/>[error.go]"]
  node_info["Info detection<br/>[info.go]"]
end

subgraph group_results["Results"]
  node_results["Scan results<br/>[options.go]"]
  node_output["Output formatting<br/>[output.go]"]
  node_json["JSON output<br/>[jsonl.go]"]
  node_html["HTML report<br/>[html.go]"]
  node_response_store[("HTTP response archive<br/>[responses.go]")]
end

node_user(("CLI user"))
node_web(("Target websites"))
node_files(("Output files"))
node_terminal(("Terminal"))

node_user -->|"provides targets"| node_stdin
node_user -->|"sets options"| node_flags
node_main -->|"reads options"| node_flags
node_main -->|"reads targets"| node_stdin
node_main -->|"starts scans"| node_crawler
node_crawler -->|"requests pages"| node_web
node_crawler -->|"normalizes targets"| node_urlutil
node_crawler -->|"analyzes responses"| node_scan
node_scan -->|"uses rules"| node_secrets
node_scan -->|"matches parameters"| node_endpoints
node_scan -->|"matches extensions"| node_extensions
node_scan -->|"matches errors"| node_errors
node_scan -->|"matches information"| node_info
node_crawler -->|"collects findings"| node_results
node_main -->|"deduplicates findings"| node_results
node_main -->|"formats results"| node_output
node_main -.->|"emits JSON"| node_json
node_main -->|"creates report"| node_html
node_crawler -->|"stores responses"| node_response_store
node_output -->|"writes reports"| node_files
node_html -->|"writes HTML"| node_files
node_response_store -->|"archives responses"| node_files
node_output -.->|"prints findings"| node_terminal

click node_stdin "https://github.com/edoardottt/cariddi/blob/main/pkg/input/input.go"
click node_flags "https://github.com/edoardottt/cariddi/blob/main/pkg/input/flags.go"
click node_main "https://github.com/edoardottt/cariddi/blob/main/cmd/cariddi/main.go"
click node_crawler "https://github.com/edoardottt/cariddi/blob/main/pkg/crawler/colly.go"
click node_scan "https://github.com/edoardottt/cariddi/blob/main/pkg/crawler/scan.go"
click node_urlutil "https://github.com/edoardottt/cariddi/blob/main/internal/url/url.go"
click node_secrets "https://github.com/edoardottt/cariddi/blob/main/pkg/scanner/secrets.go"
click node_endpoints "https://github.com/edoardottt/cariddi/blob/main/pkg/scanner/endpoints.go"
click node_extensions "https://github.com/edoardottt/cariddi/blob/main/pkg/scanner/extensions.go"
click node_errors "https://github.com/edoardottt/cariddi/blob/main/pkg/scanner/error.go"
click node_info "https://github.com/edoardottt/cariddi/blob/main/pkg/scanner/info.go"
click node_results "https://github.com/edoardottt/cariddi/blob/main/pkg/crawler/options.go"
click node_output "https://github.com/edoardottt/cariddi/blob/main/pkg/output/output.go"
click node_json "https://github.com/edoardottt/cariddi/blob/main/pkg/output/jsonl.go"
click node_html "https://github.com/edoardottt/cariddi/blob/main/pkg/output/html.go"
click node_response_store "https://github.com/edoardottt/cariddi/blob/main/pkg/output/responses.go"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_stdin,node_flags,node_main,node_user toneBlue
class node_crawler,node_scan,node_urlutil toneAmber
class node_secrets,node_endpoints,node_extensions,node_errors,node_info toneMint
class node_results,node_output,node_json,node_html,node_response_store toneRose
class node_web,node_files,node_terminal toneIndigo
```

## Components

### CLI and input

- [Target input](https://github.com/edoardottt/cariddi/blob/main/pkg/input/input.go): `pkg/input/input.go`
- [Options and validation](https://github.com/edoardottt/cariddi/blob/main/pkg/input/flags.go): `pkg/input/flags.go`
- [Scan coordinator](https://github.com/edoardottt/cariddi/blob/main/cmd/cariddi/main.go): `cmd/cariddi/main.go`

### Crawling

- [Colly crawler](https://github.com/edoardottt/cariddi/blob/main/pkg/crawler/colly.go): `pkg/crawler/colly.go`
- [Response analysis](https://github.com/edoardottt/cariddi/blob/main/pkg/crawler/scan.go): `pkg/crawler/scan.go`
- [URL utilities](https://github.com/edoardottt/cariddi/blob/main/internal/url/url.go): `internal/url/url.go`

### Detection

- [Secret detection](https://github.com/edoardottt/cariddi/blob/main/pkg/scanner/secrets.go): `pkg/scanner/secrets.go`
- [Endpoint detection](https://github.com/edoardottt/cariddi/blob/main/pkg/scanner/endpoints.go): `pkg/scanner/endpoints.go`
- [Extension detection](https://github.com/edoardottt/cariddi/blob/main/pkg/scanner/extensions.go): `pkg/scanner/extensions.go`
- [Error detection](https://github.com/edoardottt/cariddi/blob/main/pkg/scanner/error.go): `pkg/scanner/error.go`
- [Info detection](https://github.com/edoardottt/cariddi/blob/main/pkg/scanner/info.go): `pkg/scanner/info.go`

### Results

- [Scan results](https://github.com/edoardottt/cariddi/blob/main/pkg/crawler/options.go): `pkg/crawler/options.go`
- [Output formatting](https://github.com/edoardottt/cariddi/blob/main/pkg/output/output.go): `pkg/output/output.go`
- [JSON output](https://github.com/edoardottt/cariddi/blob/main/pkg/output/jsonl.go): `pkg/output/jsonl.go`
- [HTML report](https://github.com/edoardottt/cariddi/blob/main/pkg/output/html.go): `pkg/output/html.go`
- [HTTP response archive](https://github.com/edoardottt/cariddi/blob/main/pkg/output/responses.go): `pkg/output/responses.go`

### Other components

- **CLI user**
- **Target websites**
- **Output files**
- **Terminal**

## Connections

- CLI user → Target input: provides targets (evidence: `README.md`)
- CLI user → Options and validation: sets options (evidence: `README.md`)
- Scan coordinator → Options and validation: reads options (evidence: `cmd/cariddi/main.go`)
- Scan coordinator → Target input: reads targets (evidence: `cmd/cariddi/main.go`)
- Scan coordinator → Colly crawler: starts scans (evidence: `cmd/cariddi/main.go`)
- Colly crawler → Target websites: requests pages (evidence: `pkg/crawler/colly.go`)
- Colly crawler → URL utilities: normalizes targets (evidence: `pkg/crawler/colly.go`)
- Colly crawler → Response analysis: analyzes responses (evidence: `pkg/crawler/colly.go`)
- Response analysis → Secret detection: uses rules (evidence: `pkg/crawler/scan.go`)
- Response analysis → Endpoint detection: matches parameters (evidence: `pkg/crawler/scan.go`)
- Response analysis → Extension detection: matches extensions (evidence: `pkg/crawler/scan.go`)
- Response analysis → Error detection: matches errors (evidence: `pkg/crawler/scan.go`)
- Response analysis → Info detection: matches information (evidence: `pkg/crawler/scan.go`)
- Colly crawler → Scan results: collects findings (evidence: `pkg/crawler/colly.go`)
- Scan coordinator → Scan results: deduplicates findings (evidence: `cmd/cariddi/main.go`)
- Scan coordinator → Output formatting: formats results (evidence: `cmd/cariddi/main.go`)
- Scan coordinator → JSON output: emits JSON (evidence: `README.md`)
- Scan coordinator → HTML report: creates report (evidence: `cmd/cariddi/main.go`)
- Colly crawler → HTTP response archive: stores responses (evidence: `pkg/crawler/colly.go`)
- Output formatting → Output files: writes reports (evidence: `pkg/output/output.go`)
- HTML report → Output files: writes HTML (evidence: `pkg/output/html.go`)
- HTTP response archive → Output files: archives responses (evidence: `pkg/output/responses.go`)
- Output formatting → Terminal: prints findings (evidence: `README.md`)