# bitfield/script architecture

> Architecture diagram of the GitHub repository bitfield/script, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bitfield/script
- Repository: https://github.com/bitfield/script

## Overview

##### Main Components
- Library Core – At the heart is the API exposure (the “script” package) which includes the three kinds of operations:
- Sources: Functions that inject initial data into a pipeline (e.g. Args, Echo, File, StdIn, Exec, FindFiles, etc.).
- Modifiers/Filters: A large set of chainable methods that operate on data, transforming it line by line or in bulk (for example, Column, Match, Replace, Tee, Filter, JQ, etc.). They are implemented as concurrent functions akin to middleware.
- Sinks: The endpoints of a pipeline that output the resulting data (e.g. Stdout, String, WriteFile, CountLines).
- Extended Functionality – Support for HTTP operations, error handling, environment configuration, and integration with external commands.
- Testing and Examples – With test files in testdata, unit tests (script_test.go family) and examples shown in the README that illustrate real-world use cases (e.g., piping file data through commands).
- Auxiliary Assets – Documentation files, CI/CD workflows (GitHub Actions), and examples that assist with integration and developer usage.

##### Relationships and Interactions
- The data flow begins at “Sources” (for instance, File, Echo, or Stdin) and then is passed through a series of “Filters/Modifiers” (which are chained operations altering or augmenting the data stream), ultimately ending in “Sinks” where the final output is collected or used.
- How modifiers are implemented concurrently (using Go’s concurrency features) so that each pipe stage processes data as it becomes available, similar to Unix pipes.
- Interactions between the library’s API and external services (like HTTP calls, external process execution with Exec and ExecForEach).
- Error-handling mechanism where each pipe stage “remembers” errors that propagate downstream.

##### Architectural Patterns and Design Principles
- Separation of Concerns: Clearly divide the components into Sources, Filters (or Modifiers), and Sinks.
- Concurrency: Note that filters run concurrently, so include a note on how “data flows” asynchronously.
- Functional API design: The library’s composable functions and methods resemble a domain-specific language (DSL) for system scripting.

##### Relevant Technologies and Framework/Libraries
- The project is written in Go, so mention modules like go.mod and the standard library (io, bufio, http, etc.).
- CI/CD is set up with GitHub Actions (audit.yml, ci.yml), and testing is integrated across different environments (script_unix_test.go, script_windows_test.go).
- Documentation references like pkg.go.dev links, showcasing integration with the broader Go ecosystem.

##### Capturing Essential Architectural Elements
- Avoid overcomplicating; focus on conveying the chaining nature, the three-layer architecture of Sources → Filters → Sinks, and the supporting concerns (concurrency, error handling, integration with OS processes and HTTP).
- Every component seen in the README and file tree is included in some way, ensuring that future maintainers and contributors understand the design decisions and flow.

## Diagram (Mermaid)

```mermaid
graph TD
    A["Script Library"]:::scriptCore

    subgraph "Pipeline Core"
        subgraph "Sources"
            S1["Args"]:::source
            S2["Echo"]:::source
            S3["File"]:::source
            S4["StdIn"]:::source
            S5["Exec"]:::source
            S6["FindFiles"]:::source
        end
        subgraph "Filters/Modifiers"
            M1["Column"]:::modifier
            M2["Match"]:::modifier
            M3["Replace"]:::modifier
            M4["Tee"]:::modifier
            M5["Filter"]:::modifier
            M6["JQ"]:::modifier
            M7["ExecForEach"]:::modifier
        end
        subgraph "Sinks"
            K1["Stdout"]:::sink
            K2["String"]:::sink
            K3["WriteFile"]:::sink
            K4["CountLines"]:::sink
        end
    end

    %% Pipeline flow (representative chain)
    A -->|"start"| S3
    S3 -->|"process"| M3
    M3 -->|"output"| K3

    %% Extended and ancillary components
    A --> EF["Extended Functionality"]:::extra
    A --> TE["Testing & Examples"]:::extra
    A --> CD["CI/CD"]:::extra
    A --> DOC["Documentation"]:::extra

    %% Testing & Examples subgraph
    subgraph "Testing & Examples"
        T1["script_test.go"]:::test
        T2["script_unix_test.go"]:::test
        T3["script_windows_test.go"]:::test
        T4["testdata"]:::test
    end
    TE --> T1
    TE --> T2
    TE --> T3
    TE --> T4

    %% CI/CD subgraph
    subgraph "CI/CD"
        C1[".github/workflows"]:::ci
    end
    CD --> C1

    %% Documentation subgraph
    subgraph "Documentation"
        D1["README.md"]:::doc
        D2["doc.go"]:::doc
    end
    DOC --> D1
    DOC --> D2

    %% Click Events for Component Mapping
    click A "https://github.com/bitfield/script/blob/master/script.go"
    click EF "https://github.com/bitfield/script/blob/master/script.go"
    click T1 "https://github.com/bitfield/script/blob/master/script_test.go"
    click T2 "https://github.com/bitfield/script/blob/master/script_unix_test.go"
    click T3 "https://github.com/bitfield/script/blob/master/script_windows_test.go"
    click T4 "https://github.com/bitfield/script/tree/master/testdata"
    click C1 "https://github.com/bitfield/script/tree/master/.github/workflows"
    click D1 "https://github.com/bitfield/script/blob/master/README.md"
    click D2 "https://github.com/bitfield/script/blob/master/doc.go"

    %% Styles
    classDef scriptCore fill:#DDEEFF,stroke:#003366,stroke-width:2px;
    classDef source fill:#FFCC99,stroke:#CC6600,stroke-width:2px;
    classDef modifier fill:#CCFFCC,stroke:#339933,stroke-width:2px;
    classDef sink fill:#FFCCCC,stroke:#CC3333,stroke-width:2px;
    classDef extra fill:#E6E6FA,stroke:#6666FF,stroke-width:2px;
    classDef test fill:#FFFACD,stroke:#CCCC00,stroke-width:2px;
    classDef ci fill:#CCE5FF,stroke:#004080,stroke-width:2px;
    classDef doc fill:#CCE5FF,stroke:#004080,stroke-width:2px;
```
