# winstonfb/wa-historical-business-license-patterns architecture

> Architecture diagram of the GitHub repository winstonfb/wa-historical-business-license-patterns, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2024-12-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/winstonfb/wa-historical-business-license-patterns
- Repository: https://github.com/winstonfb/wa-historical-business-license-patterns

## Overview

Key Components to Include in the System Design:

##### Data Collection Layer
- Python scraper (wa_blscrape_by_ubi.py) that collects business license images from WA Department of Revenue
- External data sources:
- WA Secretary of State corporation data
- Dow Jones Industrial Average data
- Census Bureau population data

##### Data Processing Layer
- Image Processing Pipeline:
- license_image_process.sh (ImageMagick for image processing)
- Tesseract OCR component
- Data Processing Scripts:
- process_license_data.py for license text processing
- WA_business_licenses.R for data combination and analysis

##### Data Storage Layer
- Raw Data Storage:
- Batch results files (batch_n_results.txt)
- External data files (dow_jones_data.csv, wa_combined_intercensal_pop_data.txt)
- Processed Data Storage:
- Shiny app data files (wa_df.rds, wa_license_activity.rds, wa_sector_data.rds)

##### Visualization Layer (Shiny Application)
- Server component (server.R)
- UI component (ui.R)
- Helper functions (WA_BLS_helper.R)

Design Guidelines:

- Processing steps in the middle
- End with the Shiny visualization app at the bottom

##### Technology Labels
- Python
- R/Shiny
- ImageMagick
- Tesseract OCR
- RDS storage

##### Process Steps
- Web scraping
- Image processing
- OCR conversion
- Data merging
- Statistical analysis
- Interactive visualization

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Define styles
    classDef external fill:#b3d9ff,stroke:#333
    classDef processing fill:#90EE90,stroke:#333
    classDef storage fill:#FFE5B4,stroke:#333
    classDef visualization fill:#DDA0DD,stroke:#333
    
    %% External Data Sources
    subgraph DataSources
        WA[WA Secretary of State Data]:::external
        DJ["Dow Jones Data"]:::external
        CB[Census Bureau Data]:::external
        click DJ "https://github.com/winstonfb/WA-historical-business-license-patterns/blob/master/data/dow_jones_data.csv"
        click CB "https://github.com/winstonfb/WA-historical-business-license-patterns/blob/master/data/wa_combined_intercensal_pop_data.txt"
    end

    %% Data Collection
    WebScraper["Python Web Scraper"]:::processing
    click WebScraper "https://github.com/winstonfb/WA-historical-business-license-patterns/blob/master/wa_blscrape_by_ubi.py"

    %% Image Processing Pipeline
    subgraph ImageProcessing
        ImgProc["ImageMagick Processing"]:::processing
        OCR["Tesseract OCR"]:::processing
        click ImgProc "https://github.com/winstonfb/WA-historical-business-license-patterns/blob/master/license_image_process.sh"
    end

    %% Data Processing
    subgraph DataProcessing
        LicenseProc["License Data Processing"]:::processing
        BusinessAnalysis["Business License Analysis"]:::processing
        click LicenseProc "https://github.com/winstonfb/WA-historical-business-license-patterns/blob/master/process_license_data.py"
        click BusinessAnalysis "https://github.com/winstonfb/WA-historical-business-license-patterns/blob/master/WA_business_licenses.R"
    end

    %% Data Storage
    subgraph RawStorage
        BatchFiles["Batch Results Files"]:::storage
        ExtData["External Data Files"]:::storage
        click BatchFiles "https://github.com/winstonfb/WA-historical-business-license-patterns/tree/master/data/"
        click ExtData "https://github.com/winstonfb/WA-historical-business-license-patterns/tree/master/data/"
    end

    subgraph ProcessedStorage
        RDSFiles["RDS Data Files"]:::storage
        click RDSFiles "https://github.com/winstonfb/WA-historical-business-license-patterns/tree/master/shiny-app/data/"
    end

    %% Visualization Layer
    subgraph ShinyApp
        Server["Server Component"]:::visualization
        UI["UI Component"]:::visualization
        Helper["Helper Functions"]:::visualization
        click Server "https://github.com/winstonfb/WA-historical-business-license-patterns/blob/master/shiny-app/server.R"
        click UI "https://github.com/winstonfb/WA-historical-business-license-patterns/blob/master/shiny-app/ui.R"
        click Helper "https://github.com/winstonfb/WA-historical-business-license-patterns/blob/master/shiny-app/WA_BLS_helper.R"
    end

    %% Data Flow
    WA --> BusinessAnalysis
    DJ --> BusinessAnalysis
    CB --> BusinessAnalysis
    WebScraper --> ImgProc
    ImgProc --> OCR
    OCR --> LicenseProc
    LicenseProc --> BatchFiles
    BatchFiles --> BusinessAnalysis
    ExtData --> BusinessAnalysis
    BusinessAnalysis --> RDSFiles
    RDSFiles --> Server
    Server --> UI
    Helper --> Server

    %% Legend
    subgraph Legend
        E["External Sources"]:::external
        P["Processing"]:::processing
        S["Storage"]:::storage
        V["Visualization"]:::visualization
    end
```
