How thoughtworks/talisman works
Project type and purpose
• This is a Go-based CLI tool (Talisman) that integrates as a Git pre-commit/pre-push hook or standalone scanner to detect and prevent secrets (keys, credentials, passwords) from being committed.
• The README and file structure show it’s a modular command-line utility with multiple sub-commands (scan, checksum, interactive hooks, HTML reporting) and a suite of “detectors” that inspect file names, contents, sizes, entropy, patterns, etc.
File structure
• cmd/: entry points for each CLI command (talisman, checksum, scanner, hook installers).
• checksumcalculator/, scanner/, report/, prompt/, utility/: core functional packages called by the commands.
• detector/: the heart of the detection engine, organized into sub-packages:
• chain.go → orchestrates multiple detectors
• filecontent (base64, hex, entropy, credit-card, word checks)
• filename, filesize, pattern, severity
• gitrepo/: abstractions to read Git diffs or history.
• talismanrc/: loading and parsing of the .talismanrc configuration (ignore rules, thresholds, custom patterns).
• global_install_scripts/ & install.sh: shell scripts to install/uninstall Git hooks.
• Standard Go config files (go.mod, pre-commit hooks, CI workflow definitions).
Key insights from README
• CLI supports multiple modes: --scan, --checksum, --githook, --interactive, --scanWithHtml.
• Detects secrets via pluggable “detectors” and allows .talismanrc overrides.
• Produces console reports and optional HTML reports (via separate talisman-html-report plugin).
• Installs as Git hooks by symlinking an executable into .git/hooks.
Main components
a. CLI Layer
• Commands: talisman (with flags), checksum, scanner, pre_commit_hook, pre_push_hook, etc.
b. Orchestration/Core
• Runner (cmd/runner.go): dispatches commands to subsystems.
• Config Manager (talismanrc package): loads ignore rules, thresholds, custom patterns.
c. Git Interface
• GitReader (gitrepo): fetches staged diffs or history.
d. Detection Engine
• Detector Chain: executes detectors in sequence.
• Detectors: filecontent, encoded values, entropy, pattern, filename, filesize, credit cards.
• Severity Config: assigns severity and filters based on threshold.
e. Reporting
• Console Reporter: tabular error report on stdout.
• HTML Reporter: optional, writes HTML to talisman_html_report.
f. Utilities and Helpers
• Checksum Calculator: computes file checksums for .talismanrc.
• Prompt/Interactive: handles interactive ignore-rule generation (survey wrapper).
• Utility: progress bar, SHA-256 hasher, file-system helpers.
g. Installation Scripts
• Shell scripts under global_install_scripts/ and install.sh for global or repo-specific hook setup.
Relationships and interactions
1. User invokes CLI (shell or hook).
2. Runner parses flags and loads Config.
3. For scanning:
• GitReader supplies changed files or history.
• DetectorChain iterates through configured Detectors, passing each file or diff.
• Each detector returns DetectionResult (pass/fail, severity, message).
• Chain aggregates results, applies ignore rules from Config.
4. Reporter formats and outputs failures to console or writes HTML.
5. For checksum mode:
• ChecksumCalculator finds files matching patterns, computes checksums, formats YAML snippet.
6. For interactive mode:
• Prompt module asks user which files/detectors to ignore, then updates .talismanrc.
7. Installation Scripts link the talisman executable into Git hook folders.
Architectural patterns and design principles
• Modular layering: clear separation between CLI, core orchestration, detection logic, configuration, and presentation.
• Pluggable detectors: new detectors can be added under detector/.
• Chain of Responsibility: DetectorChain invokes each detector in turn.
• Dependency injection via interfaces (gitrepo, prompt, checksumcalculator) to facilitate testing (mocks in internal/mock).
• Configuration‐driven behavior: thresholds, ignore rules, custom patterns all read from YAML.
Final reminders
• essential elements: invocation paths, configuration overlay, detectors, reporting outputs.
• Do not over-complicate: lean on the modular boundaries already present in the codebase.