# thoughtworks/talisman architecture

> Architecture diagram of the GitHub repository thoughtworks/talisman, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/thoughtworks/talisman
- Repository: https://github.com/thoughtworks/talisman

## Overview

##### Project type and purpose
- This is a Go-based CLI tool (Talisman) that integrates as a Git pre-commit/pre-push hook or standalone scanner to detect and prevent secrets (keys, credentials, passwords) from being committed.
- The README and file structure show it’s a modular command-line utility with multiple sub-commands (scan, checksum, interactive hooks, HTML reporting) and a suite of “detectors” that inspect file names, contents, sizes, entropy, patterns, etc.

##### File structure
- **cmd/**: entry points for each CLI command (`talisman`, `checksum`, `scanner`, hook installers).
- **checksumcalculator/**, **scanner/**, **report/**, **prompt/**, **utility/**: core functional packages called by the commands.
- **detector/**: the heart of the detection engine, organized into sub-packages:
- chain.go → orchestrates multiple detectors
- filecontent (base64, hex, entropy, credit-card, word checks)
- filename, filesize, pattern, severity
- **gitrepo/**: abstractions to read Git diffs or history.
- **talismanrc/**: loading and parsing of the `.talismanrc` configuration (ignore rules, thresholds, custom patterns).
- **global_install_scripts/** & **install.sh**: shell scripts to install/uninstall Git hooks.
- Standard Go config files (go.mod, pre-commit hooks, CI workflow definitions).

##### Key insights from README
- CLI supports multiple modes: `--scan`, `--checksum`, `--githook`, `--interactive`, `--scanWithHtml`.
- Detects secrets via pluggable “detectors” and allows `.talismanrc` overrides.
- Produces console reports and optional HTML reports (via separate `talisman-html-report` plugin).
- Installs as Git hooks by symlinking an executable into `.git/hooks`.

##### Main components
a. **CLI Layer**
- Commands: `talisman` (with flags), `checksum`, `scanner`, `pre_commit_hook`, `pre_push_hook`, etc.
b. **Orchestration/Core**
- **Runner** (cmd/runner.go): dispatches commands to subsystems.
- **Config Manager** (talismanrc package): loads ignore rules, thresholds, custom patterns.
c. **Git Interface**
- **GitReader** (gitrepo): fetches staged diffs or history.
d. **Detection Engine**
- **Detector Chain**: executes detectors in sequence.
- **Detectors**: filecontent, encoded values, entropy, pattern, filename, filesize, credit cards.
- **Severity Config**: assigns severity and filters based on threshold.
e. **Reporting**
- **Console Reporter**: tabular error report on stdout.
- **HTML Reporter**: optional, writes HTML to `talisman_html_report`.
f. **Utilities and Helpers**
- **Checksum Calculator**: computes file checksums for `.talismanrc`.
- **Prompt/Interactive**: handles interactive ignore-rule generation (`survey` wrapper).
- **Utility**: progress bar, SHA-256 hasher, file-system helpers.
g. **Installation Scripts**
- Shell scripts under **global_install_scripts/** and `install.sh` for global or repo-specific hook setup.

##### Relationships and interactions
1. **User invokes CLI** (shell or hook).
2. **Runner** parses flags and loads **Config**.
3. For scanning:
- **GitReader** supplies changed files or history.
- **DetectorChain** iterates through configured **Detectors**, passing each file or diff.
- Each detector returns **DetectionResult** (pass/fail, severity, message).
- **Chain** aggregates results, applies ignore rules from **Config**.
4. **Reporter** formats and outputs failures to console or writes HTML.
5. For checksum mode:
- **ChecksumCalculator** finds files matching patterns, computes checksums, formats YAML snippet.
6. For interactive mode:
- **Prompt** module asks user which files/detectors to ignore, then updates **.talismanrc**.
7. **Installation Scripts** link the `talisman` executable into Git hook folders.

##### Architectural patterns and design principles
- **Modular layering**: clear separation between CLI, core orchestration, detection logic, configuration, and presentation.
- **Pluggable detectors**: new detectors can be added under `detector/`.
- **Chain of Responsibility**: DetectorChain invokes each detector in turn.
- **Dependency injection** via interfaces (`gitrepo`, `prompt`, `checksumcalculator`) to facilitate testing (mocks in `internal/mock`).
- **Configuration‐driven behavior**: thresholds, ignore rules, custom patterns all read from YAML.

##### Final reminders
- **essential elements**: invocation paths, configuration overlay, detectors, reporting outputs.
- Do not over-complicate: lean on the modular boundaries already present in the codebase.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% CLI Layer
    subgraph "CLI Layer"
        direction TB
        Talisman["Talisman CLI"]:::cli
        ChecksumCmd["Checksum Sub-command"]:::cli
        ScannerCmd["Scanner Sub-command"]:::cli
        PatternCmd["Pattern Sub-command"]:::cli
        PreCommitHook["Pre-commit Hook Installer"]:::cli
        PrePushHook["Pre-push Hook Installer"]:::cli
    end

    %% Orchestration/Core
    subgraph "Orchestration / Core"
        direction TB
        Runner["Runner (flag parsing & dispatch)"]:::orchestration
        ConfigManager["Config Manager (.talismanrc loader)"]:::orchestration
    end

    %% Git Interface
    subgraph "Git Interface"    
        GitReader["GitReader"]:::io
    end

    %% Detection Engine
    subgraph "Detection Engine"
        direction TB
        DetectorChain["DetectorChain"]:::detection
        subgraph "Detectors"
            direction TB
            FileContent["File-content Detectors"]:::detection
            FilenameDet["Filename Detector"]:::detection
            FilesizeDet["Filesize Detector"]:::detection
            PatternDet["Pattern Detector"]:::detection
            SeverityCfg["Severity Config"]:::detection
            Helpers["Helper Utilities"]:::detection
        end
    end

    %% Reporting
    subgraph "Reporting"
        ConsoleReporter["Console Reporter"]:::io
    end

    %% Utilities
    subgraph "Utilities"
        direction TB
        ChecksumCalc["Checksum Calculator"]:::util
        Prompt["Interactive Prompt"]:::util
        GeneralUtil["General Helpers"]:::util
    end

    %% Installation Scripts
    subgraph "Installation Scripts"
        GlobalScripts["Global & Repo Hook Scripts"]:::install
    end

    %% Connections
    Talisman -->|flags| Runner
    ChecksumCmd -->|flags| Runner
    ScannerCmd -->|flags| Runner
    PatternCmd -->|flags| Runner
    PreCommitHook -->|install| Runner
    PrePushHook -->|install| Runner

    Runner -->|load config| ConfigManager
    Runner -->|invoke git| GitReader
    Runner -->|checksum mode| ChecksumCalc
    Runner -->|interactive mode| Prompt

    GitReader -->|diffs| DetectorChain
    DetectorChain -->|runs| FileContent
    DetectorChain -->|runs| FilenameDet
    DetectorChain -->|runs| FilesizeDet
    DetectorChain -->|runs| PatternDet
    DetectorChain -->|uses| SeverityCfg
    DetectorChain -->|uses| Helpers
    DetectorChain -->|results| ConsoleReporter

    Runner -->|report html| ConsoleReporter

    Runner -->|install hooks| GlobalScripts

    %% Click Events
    click Talisman "https://github.com/thoughtworks/talisman/blob/main/cmd/talisman.go"
    click ChecksumCmd "https://github.com/thoughtworks/talisman/blob/main/cmd/checksum_cmd.go"
    click ScannerCmd "https://github.com/thoughtworks/talisman/blob/main/cmd/scanner_cmd.go"
    click PatternCmd "https://github.com/thoughtworks/talisman/blob/main/cmd/pattern_cmd.go"
    click PreCommitHook "https://github.com/thoughtworks/talisman/blob/main/cmd/pre_commit_hook.go"
    click PrePushHook "https://github.com/thoughtworks/talisman/blob/main/cmd/pre_push_hook.go"
    click Runner "https://github.com/thoughtworks/talisman/blob/main/cmd/runner.go"
    click ConfigManager "https://github.com/thoughtworks/talisman/blob/main/talismanrc/rc_file.go"
    click GitReader "https://github.com/thoughtworks/talisman/blob/main/gitrepo/gitrepo.go"
    click DetectorChain "https://github.com/thoughtworks/talisman/blob/main/detector/chain.go"
    click FileContent "https://github.com/thoughtworks/talisman/tree/main/detector/filecontent/"
    click FilenameDet "https://github.com/thoughtworks/talisman/blob/main/detector/filename/filename_detector.go"
    click FilesizeDet "https://github.com/thoughtworks/talisman/blob/main/detector/filesize/filesize_detector.go"
    click PatternDet "https://github.com/thoughtworks/talisman/blob/main/detector/pattern/pattern_detector.go"
    click SeverityCfg "https://github.com/thoughtworks/talisman/blob/main/detector/severity/severity_config.go"
    click Helpers "https://github.com/thoughtworks/talisman/blob/main/detector/helpers/ignore_evaluator.go"
    click ConsoleReporter "https://github.com/thoughtworks/talisman/blob/main/report/report.go"
    click ChecksumCalc "https://github.com/thoughtworks/talisman/blob/main/checksumcalculator/checksumcalculator.go"
    click Prompt "https://github.com/thoughtworks/talisman/blob/main/prompt/survey.go"
    click GeneralUtil "https://github.com/thoughtworks/talisman/blob/main/utility/utility.go"
    click GlobalScripts "https://github.com/thoughtworks/talisman/tree/main/global_install_scripts/"

    %% Styles
    classDef cli fill:#e3f2fd,stroke:#1e88e5,color:#0d47a1
    classDef orchestration fill:#fff3e0,stroke:#ffb300,color:#bf360c
    classDef io fill:#e8f5e9,stroke:#43a047,color:#1b5e20
    classDef detection fill:#f3e5f5,stroke:#8e24aa,color:#4a148c
    classDef util fill:#ffe0b2,stroke:#fb8c00,color:#e65100
    classDef install fill:#ffebee,stroke:#e53935,color:#b71c1c
```
