# shinrenpan/webparser architecture

> Architecture diagram of the GitHub repository shinrenpan/webparser, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/shinrenpan/webparser
- Repository: https://github.com/shinrenpan/webparser

## Overview

##### Project Type and Purpose
- This is a Swift Package (“WebParser”) that provides a WKWebView-based web crawler/parser for iOS (13+), plus a Demo iOS app that exercises the library.
- The library’s goal is to:
- Load a URL in an off-screen WKWebView
- Execute custom JavaScript
- And decode the result into a Swift `Decodable` model
- Handle retry logic and cookie management

##### File Structure Analysis
- Top‐level:
- Package.swift – defines the Swift Package (WebParser)
- Sources/… – the core library modules:
– CookiePool.swift
– Parser.swift
– ParserConfiguration.swift
– ParserError.swift
- Demo/… – an Xcode iOS app project demonstrating use of the library
- Demo app follows an MVVM-style pattern (Main folder: Model, ViewModel, ViewController, Router, ViewOutlet).
- WebParser.xcodeproj – header for potential Obj-C bridging (WebParser.h).

##### README Insights
- Public API revolves around `ParserConfiguration` and `Parser`:
1. Create `ParserConfiguration` (URL request, windowSize, user agent, retry settings, JavaScript)
2. Init `Parser(parserConfiguration:)`
3. Call `try await parser.decodeResult(SomeData.self)` or `anyResult()`
- No external services beyond iOS frameworks (WKWebKit, Foundation).

##### Main Components & Interactions
a. Components
- Demo App
– UI Layer: ViewController, ViewModel, Router
– Model Layer: Decodable models for parsed data
- WebParser Library
– Parser API (public interface)
– Parser Core (Parser.swift)
– ParserConfiguration (holds settings)
– CookiePool (manages cookies across loads)
– ParserError (error handling)
– Hidden WKWebView instance
– JavaScript execution engine
- External Dependencies
– WKWebKit (WKWebView)
– Foundation (URLSession under the hood, JSONDecoder)
b. Interactions / Data Flow
1. Demo ViewModel → builds ParserConfiguration
2. ViewModel calls `Parser(parserConfiguration:)` → public Parser API
3. Parser
– configures an off-screen WKWebView (windowSize, userAgent)
– loads the URL request
– injects & runs the provided JavaScript
– collects JS result (e.g., JSON or HTML string)
– uses `JSONDecoder` or fallback to `Any`
– manages retryCount/retryDuration on failures
– uses CookiePool to preserve session cookies across loads
4. Parsed data → returned asynchronously to ViewModel → UI
c. Architectural Patterns
- Demo: MVVM (ViewController ↔ ViewModel ↔ Model)
- Library: Modular separation of concerns (configuration, execution, error handling, cookie management)

##### Essential Elements
- Clear component names
- Annotations for retry logic and cookie management loops
- Error-handling paths back to the caller (ParserError)

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph "Demo App"
        AppEntry["AppDelegate / Main"]:::ui
        ViewController["ViewController"]:::ui
        ViewModel["ViewModel"]:::ui
        Router["Router"]:::ui
        ViewOutlet["ViewOutlet"]:::ui
        Model["Model"]:::data
    end

    subgraph "WebParser Library"
        ParserAPI["Parser"]:::api
        ParserConfig["ParserConfiguration"]:::data
        CookiePool["CookiePool"]:::data
        ParserError["ParserError"]:::data
        WKWebViewHidden["Hidden WKWebView"]:::execution
        JSEngine["JavaScript Engine"]:::execution
        JSONDec["JSONDecoder"]:::data
    end

    subgraph "iOS Frameworks"
        WKWebKit["WKWebKit"]:::external
        Foundation["Foundation"]:::external
    end

    %% Demo App Flow
    AppEntry -->|launches| ViewController
    ViewController -->|binds to| ViewModel
    ViewController -->|outlets| ViewOutlet
    ViewModel -->|calls| ParserAPI
    ViewModel -->|uses| Router
    ViewModel -->|receives| Model

    %% Library Flow
    ParserAPI -->|uses config| ParserConfig
    ParserAPI -->|manages| CookiePool
    ParserAPI -->|handles errors| ParserError
    ParserAPI -->|creates| WKWebViewHidden
    WKWebViewHidden -->|injects| JSEngine
    JSEngine -->|returns result| ParserAPI
    ParserAPI -->|decodes via| JSONDec
    JSONDec -->|produces| Model
    ParserAPI -->|retry loop| CookiePool
    ParserError -->|propagates to| ViewModel

    %% Framework Integrations
    WKWebViewHidden -->|framework| WKWebKit
    JSONDec -->|framework| Foundation
    CookiePool -->|uses| Foundation

    %% Click Events
    click ViewController "https://github.com/shinrenpan/webparser/blob/main/Demo/Sources/Main/ViewController.swift"
    click ViewOutlet "https://github.com/shinrenpan/webparser/blob/main/Demo/Sources/Main/ViewOutlet.swift"
    click ViewModel "https://github.com/shinrenpan/webparser/blob/main/Demo/Sources/Main/ViewModel.swift"
    click Router "https://github.com/shinrenpan/webparser/blob/main/Demo/Sources/Main/Router.swift"
    click Model "https://github.com/shinrenpan/webparser/blob/main/Demo/Sources/Main/Model.swift"
    click AppEntry "https://github.com/shinrenpan/webparser/blob/main/Demo/Sources/AppDelegate.swift"
    click ParserAPI "https://github.com/shinrenpan/webparser/blob/main/Sources/Parser.swift"
    click ParserConfig "https://github.com/shinrenpan/webparser/blob/main/Sources/ParserConfiguration.swift"
    click CookiePool "https://github.com/shinrenpan/webparser/blob/main/Sources/CookiePool.swift"
    click ParserError "https://github.com/shinrenpan/webparser/blob/main/Sources/ParserError.swift"

    %% Styles
    classDef ui fill:#ADD8E6,stroke:#000,stroke-width:1px
    classDef api fill:#98FB98,stroke:#000,stroke-width:1px
    classDef execution fill:#FFA500,stroke:#000,stroke-width:1px
    classDef data fill:#FFFF99,stroke:#000,stroke-width:1px
    classDef external fill:#D3D3D3,stroke:#000,stroke-width:1px
```
