# tnypxl/rollup architecture

> Architecture diagram of the GitHub repository tnypxl/rollup, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tnypxl/rollup
- Repository: https://github.com/tnypxl/rollup

## Overview

##### Main Components

a) CLI Interface Layer:
- Root Command Handler (cmd/root.go)
- Sub-command Handlers:
- Files Command (cmd/files.go)
- Web Command (cmd/web.go)
- Generate Command (cmd/generate.go)

b) Core Services:
- Configuration Service (internal/config)
- Web Scraping Service (internal/scraper)

c) External Interactions:
- Local File System
- Web Resources (URLs)
- Output Files (Markdown)

**User Input → CLI Layer → Core Services → Output**

##### Important Elements to Highlight

- Local files (Green)
- Web URLs (Blue)
- Configuration files (Yellow)

b) Processing Pipeline:
- File filtering and aggregation
- Web content extraction
- Markdown conversion

c) Configuration Management:
- YAML file parsing
- Default config generation
- Override handling

##### Specific Implementation Details

a) For the Files Command:
- File type filtering
- Ignore pattern handling
- Code-generated file detection

b) For the Web Command:
- Depth traversal logic
- CSS selector processing
- Rate limiting mechanism
- Output format handling

##### Additional Notes

- Version information and important dependencies
- Configuration inheritance and override patterns

Remember to maintain a balance between detail and clarity, focusing on the architectural decisions that make Rollup unique, such as its flexible configuration system and content aggregation capabilities.

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph CLI
        CLI_ENTRY[Main Entry Point]
        ROOT[Root Command Handler]
        FILES[Files Command]
        WEB[Web Command]
        GEN[Generate Command]
        
        CLI_ENTRY --> ROOT
        ROOT --> FILES
        ROOT --> WEB
        ROOT --> GEN
    end

    subgraph Core_Services
        CONFIG[Configuration Service]
        SCRAPER[Web Scraping Service]
        
        CONFIG --> SCRAPER
    end

    subgraph Input_Sources
        FS[("Local File System")]:::filesystem
        WEB_RES[("Web Resources")]:::web
        YAML[("YAML Config")]:::config
    end

    subgraph Processing
        FILTER[File Filtering]:::process
        EXTRACT[Content Extraction]:::process
        CONVERT[Markdown Conversion]:::process
        
        FILTER --> EXTRACT
        EXTRACT --> CONVERT
    end

    subgraph Output
        MD[("Markdown Files")]:::output
    end

    %% Connections
    FILES --> FS
    WEB --> WEB_RES
    CONFIG --> YAML
    SCRAPER --> WEB_RES
    Processing --> MD

    %% Click Events
    click ROOT "https://github.com/tnypxl/rollup/blob/main/cmd/root.go"
    click FILES "https://github.com/tnypxl/rollup/blob/main/cmd/files.go"
    click WEB "https://github.com/tnypxl/rollup/blob/main/cmd/web.go"
    click GEN "https://github.com/tnypxl/rollup/blob/main/cmd/generate.go"
    click CONFIG "https://github.com/tnypxl/rollup/blob/main/internal/config/config.go"
    click SCRAPER "https://github.com/tnypxl/rollup/blob/main/internal/scraper/scraper.go"
    click CLI_ENTRY "https://github.com/tnypxl/rollup/blob/main/main.go"

    %% Styles
    classDef filesystem fill:#90EE90
    classDef web fill:#87CEEB
    classDef config fill:#FFD700
    classDef process fill:#F0F8FF
    classDef output fill:#98FB98

    %% Legend
    subgraph Legend
        L1[Command]:::command
        L2[("Storage")]:::storage
        L3[Service]:::service
        L4[Process]:::process
    end

    classDef command fill:#f9f,stroke:#333
    classDef storage fill:#fcf,stroke:#333
    classDef service fill:#ccf,stroke:#333
    classDef process fill:#F0F8FF,stroke:#333
```
