# kylebutts/generalized-imputation-estimator architecture

> Architecture diagram of the GitHub repository kylebutts/generalized-imputation-estimator, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kylebutts/generalized-imputation-estimator
- Repository: https://github.com/kylebutts/generalized-imputation-estimator

## Overview

##### Project Type and Purpose
- This project is a research replication and extension study focusing on dynamic treatment effect estimation. It uses simulations and real-world data from Walmart openings along with extensive estimators (TWFE, generalized imputation, QLD, etc.) to analyze dynamic effects.
- The README and file tree indicate two main applications: simulation experiments (to produce tables and figures for the paper) and application analyses (replicating the Walmart study).

##### File Structure
- Code is organized into directories by task and methodology:
- The "code/Simulation" folder contains scripts in R and Julia for running simulation experiments (data-generating processes, various estimators, and summarization routines).
- The "code/Walmart" folder includes scripts for data preprocessing, running the analysis on Walmart data, and generating figures.
- The "code/qld" folder appears dedicated to specific estimation routines (likely related to QLD estimators).
- Other top-level directories:
- "data" holds processed simulation and application datasets.
- "estimates" stores the output of various estimation procedures.
- "out" contains generated outputs such as figures, tables, PDFs (manuscript outputs/slides).
- "raw-data" is used for storing the original datasets (1980 Census, County Business Patterns, etc.).
- Configuration files like ".JuliaFormatter.toml" and ".gitignore" indicate development best practices.

##### README for Additional Insights
- The README outlines both the simulation experiments (e.g., Table 1, Figure 1) and the application analysis (e.g., Walmart openings effect on local economies) as key components.
- It also specifies the order of operations (which scripts run first) and data dependencies (for instance, external downloads into "raw-data").

a. Main Components:
- Data Management:
- Raw-data acquisition and storage (raw-data folder).
- Processed data handling (data folder, Walmart-data.R in “code/Walmart”).
- Simulation Module:
- Simulation code in code/Simulation (several scripts for generating simulation experiments).
- Helper functions for estimator implementations.
- Application Analysis Module:
- Walmart analysis scripts in code/Walmart (scripting of data cleaning, factor selection, estimation, figure generation).
- QLD module in code/qld handling specialized estimation routines.
- Output & Reporting:
- Estimates results (estimates folder).
- Visualizations, figures, tables, papers (out folder with subfolders for figures, tables, and slides).
- External Dependencies:
- Data sources noted in the raw-data folder (Census and business patterns data) and external downloads required (e.g., efsy_panel_naics.csv).

b. Relationships and Interactions:
- Raw-data feeds into the Data Management component which processes and cleans data for both simulation and application analyses.
- The Simulation Module uses helper functions and estimator scripts to generate experimental outputs that are then stored as estimates and figures.
- The Application Analysis Module (including Walmart and QLD components) depends on the cleaned, processed data and uses various scripts to estimate treatment effects.
- All modules output results (tables, figures, reports) into the "out" and "estimates" directories.
- There is a flow from data ingestion → simulation/analysis → output generation.

c. Architectural Patterns and Design Principles:
- A modular separation of concerns is evident: simulation routines, application analysis, and specialized estimators are kept in separate folders.
- The project follows a pipeline workflow: preprocessing raw data, running estimators, and finally generating output artifacts.
- Both R and Julia are used, which indicates a polyglot coding environment with each language being used for components where they excel.

d. Relevant Technologies, Frameworks, and Libraries:
- Julia and R are the primary programming languages.
- The use of scripts to produce LaTeX and PDF outputs (evident from .tex files and .pdf outputs) suggests integration with document generation tools.
- Data manipulation and statistical estimation libraries (likely within R and Julia) are central to the estimation process.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Storage
    A["Raw Data Storage"]:::dataStore

    subgraph "Data Management"
        PD["Processed Data Directory"]:::dataStore
        WD["Walmart Data Preprocessing (R)"]:::dataStore
    end

    %% Data ingestion arrows
    A -->|"ingests"| PD
    A -->|"ingests"| WD

    %% Simulation Module
    subgraph "Simulation Module"
        SM["Simulation Experiments & Estimators"]:::simulation
    end

    %% Application Analysis Module
    subgraph "Application Analysis Module"
        WA["Walmart Analysis"]:::analysis
        QA["QLD Estimators"]:::analysis
    end

    %% Output & Reporting
    subgraph "Output & Reporting"
        OR["Estimation Results"]:::output
        OF["Visualizations & Documents"]:::output
    end

    %% Data flow linking Data Management to Processing Modules
    PD -->|"feeds"| SM
    WD -->|"feeds"| WA
    WD -->|"feeds"| QA

    %% Modules producing outputs
    SM -->|"produces"| OR
    SM -->|"produces"| OF
    WA -->|"produces"| OR
    WA -->|"produces"| OF
    QA -->|"produces"| OR
    QA -->|"produces"| OF

    %% Click Events
    click A "https://github.com/kylebutts/generalized-imputation-estimator/tree/main/raw-data/"
    click PD "https://github.com/kylebutts/generalized-imputation-estimator/tree/main/data/"
    click WD "https://github.com/kylebutts/generalized-imputation-estimator/blob/main/code/Walmart/data.R"
    click SM "https://github.com/kylebutts/generalized-imputation-estimator/tree/main/code/Simulation/"
    click WA "https://github.com/kylebutts/generalized-imputation-estimator/tree/main/code/Walmart/"
    click QA "https://github.com/kylebutts/generalized-imputation-estimator/tree/main/code/qld/"
    click OR "https://github.com/kylebutts/generalized-imputation-estimator/tree/main/estimates/"
    click OF "https://github.com/kylebutts/generalized-imputation-estimator/tree/main/out/"

    %% Styles
    classDef dataStore fill:#f9f,stroke:#333,stroke-width:2px;
    classDef simulation fill:#bbf,stroke:#333,stroke-width:2px;
    classDef analysis fill:#bfb,stroke:#333,stroke-width:2px;
    classDef output fill:#ff9,stroke:#333,stroke-width:2px;
```
