How cse545-bda/hypothesis_testing works
This repository analyzes U.S. Census block-group income and population data to compare income-per-capita trends for the bottom 40% with overall trends. The main workflow reads exported JSON from HDFS, filters incomplete geographic histories, and reports three hypothesis-test results. Separate preliminary scripts estimate bottom-40% income and produce comparisons. The graph reflects the excerpts and README; internal steps are grouped by responsibility, and no unsampled implementation details are assumed.
Main entry points: main.py Script runner · prelim_income_per_capita.py Per-capita comparison · main1.py Class-based analysis · prelim_analysis.py Median-income estimate
Components 14
Hypothesis analysis
- Script runner
Hypothesis testing/main.py - Filter complete data
Hypothesis testing/main.py - Bottom-40 cutoff
Hypothesis testing/main.py - Overall comparison
Hypothesis testing/main.py - Bottom-40 trends
Hypothesis testing/main.py - Growth-rate test
Hypothesis testing/main.py - Class-based analysis
Hypothesis testing/main1.py - Printed test results
Hypothesis testing/main.py
Preliminary analysis
- Median-income estimate
Preliminary analysis/prelim_analysis.py - Per-capita comparison
Preliminary analysis/prelim_income_per_capita.py - Trend charts
Preliminary analysis/prelim_income_per_capita.py
Input data
- Census data
- HDFS JSON
Other components
- Analyst
Connections 17
- Census data → HDFS JSON: export JSON
README.md - Analyst → Script runner: runs script
README.md - HDFS JSON → Filter complete data: reads and filters
Hypothesis testing/main.py - Filter complete data → Bottom-40 cutoff: derives cutoff
Hypothesis testing/main.py - Script runner → Overall comparison: runs test
Hypothesis testing/main.py - Script runner → Bottom-40 trends: runs test
Hypothesis testing/main.py - Script runner → Growth-rate test: runs test
Hypothesis testing/main.py - Overall comparison → Printed test results: prints result
Hypothesis testing/main.py - Bottom-40 trends → Printed test results: prints result
Hypothesis testing/main.py - Growth-rate test → Printed test results: prints result
Hypothesis testing/main.py - Analyst → Class-based analysis: runs script
README.md - HDFS JSON → Class-based analysis: loads and filters
Hypothesis testing/main1.py - Analyst → Median-income estimate: runs analysis
Preliminary analysis/prelim_analysis.py - HDFS JSON → Median-income estimate: reads JSON
Preliminary analysis/prelim_analysis.py - Analyst → Per-capita comparison: runs analysis
Preliminary analysis/prelim_income_per_capita.py - HDFS JSON → Per-capita comparison: reads JSON
Preliminary analysis/prelim_income_per_capita.py - Per-capita comparison → Trend charts: plots trends
Preliminary analysis/prelim_income_per_capita.py