How reached-hq/edu-access-intelligence works
Education Access Intelligence is an early-stage data-engineering project for evaluating education access and capacity in the Philippines. Its current workflow profiles source datasets and cross-source comparability, then ingests approved deliveries through validation into Bronze while recording batch and quality-control state. The README describes later Silver-to-dashboard layers, but those implementations are not present in the supplied tree, so they are not represented as runtime components. Edges reflect the shown excerpts, SOURCE INDEX references, or README; some ingestion internals are only partially sampled.
Main entry points: pipeline.py Ingestion run · formats.py Format dispatch · profiling Dataset profiling · validate.py Delivery validation
Components 20
Source research
- Dataset profiling
analysis/profiling - Cross-source research
analysis/cross_source/cross_source_deped.py - XLSX reader
src/profiling/xlsx.py
Delivery ingestion
- Ingestion CLI
src/ingestion/cli.py - Ingestion run
src/ingestion/pipeline.py - Format dispatch
src/ingestion/formats.py - Source contracts
src/ingestion/contract.py - Batch discovery
src/ingestion/batch.py - Revision handling
src/ingestion/revision.py
Validation and loading
- Archive reader
src/ingestion/archive.py - Delivery validation
src/ingestion/validate.py - Workbook validation
src/ingestion/workbook.py - Table validation
src/ingestion/xlsx_table.py - Delivery reader
src/ingestion/reader.py - Bronze loading
src/ingestion/bronze.py
Run state and storage
- Control records
src/ingestion/control.py - Store adapter
src/ingestion/store.py - Ingestion errors
src/ingestion/errors.py
Other components
- Education planners
- Data publishers
Connections 23
- Education planners → Cross-source research: frames questions
README.md - Data publishers → Dataset profiling: provide datasets
- Dataset profiling → Cross-source research: compare sources
- Cross-source research → XLSX reader: reads workbooks
analysis/cross_source/cross_source_deped.py - Dataset profiling → XLSX reader: reads workbooks
analysis/profiling/profile_psa_poverty.py - Data publishers → Ingestion CLI: deliver files
- Ingestion CLI → Ingestion run: starts run
src/ingestion/cli.py - Ingestion run → Format dispatch: selects format
src/ingestion/pipeline.py - Ingestion run → Source contracts: loads contract
src/ingestion/pipeline.py - Ingestion run → Batch discovery: discovers batches
src/ingestion/pipeline.py - Ingestion run → Revision handling: handles revisions
src/ingestion/pipeline.py - Format dispatch → Delivery validation: prepares archives
src/ingestion/formats.py - Format dispatch → Workbook validation: prepares sheets
src/ingestion/formats.py - Format dispatch → Table validation: prepares tables
src/ingestion/formats.py - Delivery validation → Archive reader: checks archives
src/ingestion/validate.py - Workbook validation → Delivery reader: reads delivery
src/ingestion/workbook.py - Table validation → Delivery reader: reads delivery
src/ingestion/xlsx_table.py - Ingestion run → Bronze loading: loads rows
src/ingestion/pipeline.py - Bronze loading → Store adapter: writes Bronze
src/ingestion/bronze.py - Ingestion run → Control records: records batches
src/ingestion/pipeline.py - Control records → Store adapter: writes state
src/ingestion/control.py - Ingestion run → Ingestion errors: records failures
src/ingestion/pipeline.py - Delivery validation → Ingestion errors: rejects invalid files
src/ingestion/validate.py