How kylebutts/generalized-imputation-estimator works
Project Type and Purpose
• This project is a research replication and extension study focusing on dynamic treatment effect estimation. It uses simulations and real-world data from Walmart openings along with extensive estimators (TWFE, generalized imputation, QLD, etc.) to analyze dynamic effects.
• The README and file tree indicate two main applications: simulation experiments (to produce tables and figures for the paper) and application analyses (replicating the Walmart study).
File Structure
• Code is organized into directories by task and methodology:
• The "code/Simulation" folder contains scripts in R and Julia for running simulation experiments (data-generating processes, various estimators, and summarization routines).
• The "code/Walmart" folder includes scripts for data preprocessing, running the analysis on Walmart data, and generating figures.
• The "code/qld" folder appears dedicated to specific estimation routines (likely related to QLD estimators).
• Other top-level directories:
• "data" holds processed simulation and application datasets.
• "estimates" stores the output of various estimation procedures.
• "out" contains generated outputs such as figures, tables, PDFs (manuscript outputs/slides).
• "raw-data" is used for storing the original datasets (1980 Census, County Business Patterns, etc.).
• Configuration files like ".JuliaFormatter.toml" and ".gitignore" indicate development best practices.
README for Additional Insights
• The README outlines both the simulation experiments (e.g., Table 1, Figure 1) and the application analysis (e.g., Walmart openings effect on local economies) as key components.
• It also specifies the order of operations (which scripts run first) and data dependencies (for instance, external downloads into "raw-data").
a. Main Components:
• Data Management:
• Raw-data acquisition and storage (raw-data folder).
• Processed data handling (data folder, Walmart-data.R in “code/Walmart”).
• Simulation Module:
• Simulation code in code/Simulation (several scripts for generating simulation experiments).
• Helper functions for estimator implementations.
• Application Analysis Module:
• Walmart analysis scripts in code/Walmart (scripting of data cleaning, factor selection, estimation, figure generation).
• QLD module in code/qld handling specialized estimation routines.
• Output & Reporting:
• Estimates results (estimates folder).
• Visualizations, figures, tables, papers (out folder with subfolders for figures, tables, and slides).
• External Dependencies:
• Data sources noted in the raw-data folder (Census and business patterns data) and external downloads required (e.g., efsy_panel_naics.csv).
b. Relationships and Interactions:
• Raw-data feeds into the Data Management component which processes and cleans data for both simulation and application analyses.
• The Simulation Module uses helper functions and estimator scripts to generate experimental outputs that are then stored as estimates and figures.
• The Application Analysis Module (including Walmart and QLD components) depends on the cleaned, processed data and uses various scripts to estimate treatment effects.
• All modules output results (tables, figures, reports) into the "out" and "estimates" directories.
• There is a flow from data ingestion → simulation/analysis → output generation.
c. Architectural Patterns and Design Principles:
• A modular separation of concerns is evident: simulation routines, application analysis, and specialized estimators are kept in separate folders.
• The project follows a pipeline workflow: preprocessing raw data, running estimators, and finally generating output artifacts.
• Both R and Julia are used, which indicates a polyglot coding environment with each language being used for components where they excel.
d. Relevant Technologies, Frameworks, and Libraries:
• Julia and R are the primary programming languages.
• The use of scripts to produce LaTeX and PDF outputs (evident from .tex files and .pdf outputs) suggests integration with document generation tools.
• Data manipulation and statistical estimation libraries (likely within R and Julia) are central to the estimation process.