# ramkumareinstein/noseyparker architecture

> Architecture diagram of the GitHub repository ramkumareinstein/noseyparker, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ramkumareinstein/noseyparker
- Repository: https://github.com/ramkumareinstein/noseyparker

## Overview

##### Project type and purpose
- Nosey Parker is a Rust-based, command-line application (CLI) designed to find secrets in textual data.
- It is organized as a multi-crate Cargo workspace that builds several libraries and a top-level CLI binary (`noseyparker-cli`).
- Key use cases: scanning file systems and Git history, querying GitHub repos, applying regex-based rules, persisting results in a local datastore, and rendering reports.

##### File structure
- Top level:
- Cargo.toml (workspace)
- Dockerfiles, CI workflows, build scripts under `.github` and `scripts/`.
- `crates/` directory containing:
– `noseyparker-cli`: CLI entry point and subcommands.
– `noseyparker`: core library implementing blob representation, matching engine, datastore API, Git/GitHub clients.
– `noseyparker-rules`: rule loader and ruleset definitions.
– `noseyparker-digest`: SHA-1 digest functions for blob IDs.
– `input-enumerator`: modules to enumerate inputs (filesystem, Git, Git history, GitHub API).
– `content-guesser`: modules to guess file content types.
– `bstring-serde`: serialization support.
– `progress`: progress-bar library.
- `crates/noseyparker/data`: embedded YAML rule definitions and default rulesets.
- `docs/`, `scripts/`, and CI configurations.

##### README for additional insights
- Three-phase workflow: Scan → Report → Summarize.
- Data model: blobs, provenance, matches, findings, rules, rulesets, datastore.
- Supported inputs: filesystem, Git repos (local and via URL), GitHub user/org via API.
- The datastore is a SQLite schema under a directory named by the user.

##### Final recommendations
- Bidirectional relationship: reporting commands read from the same datastore that scanning commands write to.
- That the rules directory under `crates/noseyparker/data` is an external resource loaded at runtime.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% User and CLI
    User["User"]:::external
    CLI["CLI Frontend\\nnoseyparker-cli"]:::cli
    RuleLoader["Rule Loader Utility"]:::cli

    User -->|"invokes"| CLI
    CLI -->|"loads rules"| RuleLoader

    %% Input Layer
    subgraph "Input Sources"
        direction TB
        FS["Local Filesystem"]:::external
        GitBinary["Git CLI Wrapper"]:::external
        GitHub["GitHub REST API"]:::external
    end

    CLI -->|"scan command"| FS
    CLI -->|"scan command"| GitBinary
    CLI -->|"scan command"| GitHub

    %% Enumeration
    InputEnum["Input Enumeration Layer"]:::core
    FS -->|"enumerate blobs"| InputEnum
    GitBinary -->|"enumerate blobs"| InputEnum
    GitHub -->|"enumerate blobs"| InputEnum

    %% Processing Pipeline
    Digest["Digest Module\\n(SHA-1)"]:::core
    ContentGuess["Content Detection"]:::core
    RulesEngine["Rules Engine"]:::core
    DefaultRules["YAML Rules\\n(default)"]:::core
    DefaultRulesets["YAML Rulesets\\n(default)"]:::core
    ScanningCore["Scanning Core"]:::core
    Persistence["Persistence Layer\\n(SQLite)"]:::datastore
    Report["Reporting & Formatting"]:::cli
    Progress["Progress Display"]:::core

    InputEnum -->|"blobs stream"| Digest
    Digest -->|"hashes & provenance"| ContentGuess
    ContentGuess -->|"annotated blobs"| ScanningCore
    RulesEngine -->|"loads definitions"| ScanningCore
    DefaultRules -->|"external rules"| RulesEngine
    DefaultRulesets -->|"external rulesets"| RulesEngine
    ScanningCore -->|"findings"| Persistence
    Persistence -->|"read data"| Report
    Report -->|"output"| User
    ScanningCore -->|"progress events"| Progress
    InputEnum -->|"progress events"| Progress
    CLI -->|"report/summarize"| Persistence
    CLI -->|"report/summarize"| Report

    %% Click Events
    click CLI "https://github.com/ramkumareinstein/noseyparker/tree/main/crates/noseyparker-cli"
    click RuleLoader "https://github.com/ramkumareinstein/noseyparker/blob/main/crates/noseyparker-cli/src/rule_loader.rs"
    click InputEnum "https://github.com/ramkumareinstein/noseyparker/tree/main/crates/input-enumerator"
    click ContentGuess "https://github.com/ramkumareinstein/noseyparker/tree/main/crates/content-guesser"
    click RulesEngine "https://github.com/ramkumareinstein/noseyparker/tree/main/crates/noseyparker-rules"
    click DefaultRules "https://github.com/ramkumareinstein/noseyparker/tree/main/crates/noseyparker/data/default/builtin/rules"
    click DefaultRulesets "https://github.com/ramkumareinstein/noseyparker/tree/main/crates/noseyparker/data/default/builtin/rulesets"
    click ScanningCore "https://github.com/ramkumareinstein/noseyparker/tree/main/crates/noseyparker/src"
    click Digest "https://github.com/ramkumareinstein/noseyparker/tree/main/crates/noseyparker-digest"
    click Persistence "https://github.com/ramkumareinstein/noseyparker/tree/main/crates/noseyparker/src/datastore"
    click Report "https://github.com/ramkumareinstein/noseyparker/tree/main/crates/noseyparker-cli/src/cmd_report"
    click Progress "https://github.com/ramkumareinstein/noseyparker/tree/main/crates/progress"
    click GitHub "https://github.com/ramkumareinstein/noseyparker/tree/main/crates/noseyparker/src/github"
    click GitBinary "https://github.com/ramkumareinstein/noseyparker/blob/main/crates/noseyparker/src/git_binary.rs"

    %% Styles
    classDef cli fill:#b3e6b3,stroke:#333,stroke-width:1px
    classDef core fill:#cce0ff,stroke:#333,stroke-width:1px
    classDef datastore fill:#fff2b3,stroke:#333,stroke-width:1px
    classDef external fill:#ffdab3,stroke:#333,stroke-width:1px
```
