# phiresky/ripgrep-all architecture

> Architecture diagram of the GitHub repository phiresky/ripgrep-all, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2024-12-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/phiresky/ripgrep-all
- Repository: https://github.com/phiresky/ripgrep-all

## Overview

##### Main Components

a. Core Components:
- Command Line Interface (rga binary)
- Adapter System
- Cache System
- File Processing Pipeline
- Integration with ripgrep

b. Adapter Components:
- Built-in Adapters (pandoc, poppler, ffmpeg, etc.)
- Custom Adapter Interface
- File Type Detection System

**Top Level (User Interface):**

- CLI Interface    |
- (rga, rga-fzf)  |

**Core Processing Layer:**

- File Detection   | --> | Adapter Selection| --> | Content          |
- (MIME/Extension) |     | & Management     |     | Extraction       |

- v

- Cache System     | <-- | Text Processing  | <-- | ripgrep          |
- (ZSTD compress)  |     | & Transformation |     | Integration      |

**Adapter Layer:**

- Pandoc |  |Poppler |  |FFmpeg  |  |SQLite  |  |Archive |  |Custom  |
- Adapter |  |Adapter |  |Adapter |  |Adapter |  |Adapters|  |Adapters|

##### Important Details to Highlight

a. File Processing Flow:
- How files are initially detected and routed to appropriate adapters
- Caching mechanism's position in the pipeline
- Demonstrate how processed content flows to ripgrep

b. Adapter System:
- Plugin-like nature of the adapter system
- How different adapters handle specific file types
- Archive recursion capability (up to configured depth)

c. Cache System:
- Cache database location and structure
- Compression and size limits
- Cache bypass paths

##### Integration Points

- How rga integrates with external tools (fzf, ripgrep)
- Where custom adapters can be plugged in
- Configuration points for adapters and cache

##### The flexibility of the file type detection system

This design should reflect the Rust-based architecture visible in the src directory structure, particularly the adapters module organization and the separation of concerns between preprocessing, caching, and matching components.

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph CLI["Command Line Interface"]
        direction TB
        RGA["rga"]
        RGAFZF["rga-fzf"]
        RGAFZFOPEN["rga-fzf-open"]
        RGAPREPROC["rga-preproc"]
    end

    subgraph Core["Core Processing Layer"]
        direction TB
        FD["File Detection"]:::core
        AS["Adapter Selection"]:::core
        CE["Content Extraction"]:::core
        TP["Text Processing"]:::core
        RG["ripgrep Integration"]:::core
    end

    subgraph Cache["Cache System"]
        direction TB
        CS["Cache Storage"]:::cache
        CW["Cache Writer"]:::cache
    end

    subgraph Adapters["Adapter Layer"]
        direction TB
        FFM["FFmpeg Adapter"]:::adapter
        SQL["SQLite Adapter"]:::adapter
        ZIP["ZIP Adapter"]:::adapter
        TAR["TAR Adapter"]:::adapter
        DEC["Decompress Adapter"]:::adapter
        MBX["Mbox Adapter"]:::adapter
        CST["Custom Adapters"]:::adapter
    end

    subgraph Config["Configuration"]
        direction TB
        CF["Config System"]:::config
    end

    subgraph Core_Lib["Core Library"]
        direction TB
        LIB["Library Interface"]:::lib
        REC["Recursion Handler"]:::lib
        EXP["File Expansion"]:::lib
    end

    %% Relationships
    RGA --> FD
    RGAFZF --> FD
    RGAFZFOPEN --> FD
    RGAPREPROC --> FD

    FD --> AS
    AS --> CE
    CE --> TP
    TP --> RG
    
    CE --> CW
    CW --> CS
    CS --> TP

    AS --> FFM
    AS --> SQL
    AS --> ZIP
    AS --> TAR
    AS --> DEC
    AS --> MBX
    AS --> CST

    CF --> AS
    CF --> CS

    LIB --> FD
    REC --> CE
    EXP --> FD

    %% Styles
    classDef core fill:#3498db,stroke:#2980b9,color:white
    classDef adapter fill:#2ecc71,stroke:#27ae60,color:white
    classDef cache fill:#e67e22,stroke:#d35400,color:white
    classDef config fill:#9b59b6,stroke:#8e44ad,color:white
    classDef lib fill:#34495e,stroke:#2c3e50,color:white

    %% Click Events
    click RGA "https://github.com/phiresky/ripgrep-all/blob/master/src/bin/rga.rs"
    click RGAFZF "https://github.com/phiresky/ripgrep-all/blob/master/src/bin/rga-fzf.rs"
    click RGAFZFOPEN "https://github.com/phiresky/ripgrep-all/blob/master/src/bin/rga-fzf-open.rs"
    click RGAPREPROC "https://github.com/phiresky/ripgrep-all/blob/master/src/bin/rga-preproc.rs"
    click AS "https://github.com/phiresky/ripgrep-all/blob/master/src/adapters.rs"
    click FFM "https://github.com/phiresky/ripgrep-all/blob/master/src/adapters/ffmpeg.rs"
    click SQL "https://github.com/phiresky/ripgrep-all/blob/master/src/adapters/sqlite.rs"
    click ZIP "https://github.com/phiresky/ripgrep-all/blob/master/src/adapters/zip.rs"
    click TAR "https://github.com/phiresky/ripgrep-all/blob/master/src/adapters/tar.rs"
    click DEC "https://github.com/phiresky/ripgrep-all/blob/master/src/adapters/decompress.rs"
    click MBX "https://github.com/phiresky/ripgrep-all/blob/master/src/adapters/mbox.rs"
    click CST "https://github.com/phiresky/ripgrep-all/blob/master/src/adapters/custom.rs"
    click CW "https://github.com/phiresky/ripgrep-all/blob/master/src/caching_writer.rs"
    click CS "https://github.com/phiresky/ripgrep-all/blob/master/src/preproc_cache.rs"
    click TP "https://github.com/phiresky/ripgrep-all/blob/master/src/preproc.rs"
    click CF "https://github.com/phiresky/ripgrep-all/blob/master/src/config.rs"
    click EXP "https://github.com/phiresky/ripgrep-all/blob/master/src/expand.rs"
    click REC "https://github.com/phiresky/ripgrep-all/blob/master/src/recurse.rs"
    click LIB "https://github.com/phiresky/ripgrep-all/blob/master/src/lib.rs"
```
