# aidanallchin/cse6242-patentproject architecture

> Architecture diagram of the GitHub repository aidanallchin/cse6242-patentproject, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/aidanallchin/cse6242-patentproject
- Repository: https://github.com/aidanallchin/cse6242-patentproject

## Overview

##### Project type and purpose
- This is a data‐processing and analysis pipeline (not a web app). It ingests patent and economic data, cleans/enriches it (including a large LLM for geocoding), generates county‐level features, trains a predictive model (LightGBM), and outputs visual overlays for Tableau.

##### File structure
- Top‐level script:
- main.py and __init__.py orchestrate setup, directory creation, downloads, and menu‐driven pipeline steps.
- src/data_cleaning: all modules that ingest raw USPTO/BEA/Census files, clean patents, assign lat/long, map to FIPS.
- src/modeling_overlays: generate_predictors.py and model.py for feature assembly and training the Innovation Hub Predictor Model.
- src/other: helpers.py and logging.py for shared utilities.
- Config/dependencies: requirements.txt; PatentPulse_Stats.twb is the Tableau workbook; images/model_flow_diagram.png is a reference flowchart.

##### Key insights from README
- Sequential ETL → cleaning → feature generation → modeling → visualization.
- External data sources: USPTO S3 endpoints, BEA ZIPs, Census GeoJSON.
- Optional manual downloads to save time (three intermediate files).
- Uses a 32B‐parameter LLM for geolocation corrections.
- LightGBM with DART boosting for prediction, output visualized on a map in Tableau.

##### Final recommendations
- Be exhaustive: represent every top‐level module/directory as a component.
- Auxiliary utilities (helpers, logging) as side‐components.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Sources Layer
    subgraph "Data Sources" 
        direction TB
        USPTO["USPTO S3"]:::external
        BEAZIP["BEA ZIP Files"]:::external
        CENSUS["Census GeoJSON"]:::external
        OPTFILES["Optional Manual Files"]:::external
    end

    %% Ingestion & Storage Layer
    subgraph "Ingestion & Storage"
        direction TB
        Orchestration["main.py"]:::ingestion
        Init["__init__.py"]:::ingestion
    end

    %% Data Cleaning Layer
    subgraph "Data Cleaning"
        direction TB
        DCdir["src/data_cleaning/"]:::cleaning
        ImportData["import_data.py"]:::cleaning
        BEAMod["bea_data.py"]:::cleaning
        CensusMod["census_data.py"]:::cleaning
        FREDMod["fred_data.py"]:::cleaning
        PatentClean["patent_cleanup.py"]:::cleaning
        LLMLoc["llm_location_cleaner.py"]:::cleaning
        FIPSMod["patent_fips.py"]:::cleaning
    end

    %% Feature Engineering & Modeling Layer
    subgraph "Feature Engineering & Modeling"
        direction TB
        ModelingDir["src/modeling_overlays/"]:::modeling
        GenPred["generate_predictors.py"]:::modeling
        ModelMod["model.py"]:::modeling
    end

    %% Output & Visualization Layer
    subgraph "Output & Visualization"
        direction TB
        Outputs["Predicted Outputs (TSV/CSV)"]:::output
        Tableau["PatentPulse_Stats.twb"]:::output
        RefImage["images/model_flow_diagram.png"]:::output
    end

    %% Utilities
    subgraph "Utilities"
        direction TB
        UtilDir["src/other/"]:::util
        Helpers["helpers.py"]:::util
        Logging["logging.py"]:::util
    end

    %% Data Flow Connections
    USPTO --> Orchestration
    BEAZIP --> Orchestration
    CENSUS --> Orchestration
    OPTFILES --> Orchestration

    Orchestration --> ImportData
    Orchestration --> BEAMod
    Orchestration --> CensusMod
    Orchestration --> FREDMod
    Orchestration --> PatentClean

    ImportData --> DCdir
    BEAMod --> DCdir
    CensusMod --> DCdir
    FREDMod --> DCdir
    PatentClean --> DCdir
    DCdir --> LLMLoc
    LLMLoc --> FIPSMod

    FIPSMod --> GenPred
    GenPred --> ModelMod
    ModelMod --> Outputs
    Outputs --> Tableau
    Outputs --> RefImage

    %% Utilities connections
    UtilDir --> Helpers
    UtilDir --> Logging
    Orchestration --> Helpers
    ImportData --> Logging
    ModelMod --> Logging

    %% Click Events
    click Orchestration "https://github.com/aidanallchin/cse6242-patentproject/blob/main/main.py"
    click Init "https://github.com/aidanallchin/cse6242-patentproject/blob/main/__init__.py"
    click DCdir "https://github.com/aidanallchin/cse6242-patentproject/tree/main/src/data_cleaning/"
    click ImportData "https://github.com/aidanallchin/cse6242-patentproject/blob/main/src/data_cleaning/import_data.py"
    click BEAMod "https://github.com/aidanallchin/cse6242-patentproject/blob/main/src/data_cleaning/bea_data.py"
    click CensusMod "https://github.com/aidanallchin/cse6242-patentproject/blob/main/src/data_cleaning/census_data.py"
    click FREDMod "https://github.com/aidanallchin/cse6242-patentproject/blob/main/src/data_cleaning/fred_data.py"
    click PatentClean "https://github.com/aidanallchin/cse6242-patentproject/blob/main/src/data_cleaning/patent_cleanup.py"
    click LLMLoc "https://github.com/aidanallchin/cse6242-patentproject/blob/main/src/data_cleaning/llm_location_cleaner.py"
    click FIPSMod "https://github.com/aidanallchin/cse6242-patentproject/blob/main/src/data_cleaning/patent_fips.py"
    click ModelingDir "https://github.com/aidanallchin/cse6242-patentproject/tree/main/src/modeling_overlays/"
    click GenPred "https://github.com/aidanallchin/cse6242-patentproject/blob/main/src/modeling_overlays/generate_predictors.py"
    click ModelMod "https://github.com/aidanallchin/cse6242-patentproject/blob/main/src/modeling_overlays/model.py"
    click UtilDir "https://github.com/aidanallchin/cse6242-patentproject/tree/main/src/other/"
    click Helpers "https://github.com/aidanallchin/cse6242-patentproject/blob/main/src/other/helpers.py"
    click Logging "https://github.com/aidanallchin/cse6242-patentproject/blob/main/src/other/logging.py"
    click Tableau "https://github.com/aidanallchin/cse6242-patentproject/blob/main/PatentPulse_Stats.twb"
    click RefImage "https://github.com/aidanallchin/cse6242-patentproject/blob/main/images/model_flow_diagram.png"

    %% Styles
    classDef external fill:#ADD8E6,stroke:#000
    classDef ingestion fill:#FFD580,stroke:#000
    classDef cleaning fill:#98FB98,stroke:#000
    classDef modeling fill:#FFA500,stroke:#000
    classDef output fill:#DDA0DD,stroke:#000
    classDef util fill:#D3D3D3,stroke:#000
```
