How sasark-cyber/ba_thesis works
This is an early-stage DOJ document entity-counting project. Its substantive workflow is a script that reads JSON records from the raw corpus, extracts text and entities into a Polars dataframe, then normalizes and counts people, organizations, and locations into separate CSV outputs. The package entry point currently only prints a greeting and is not part of that workflow. The README is empty, and the graph is based on the sampled loader and visible tree; no broader processing stages are evidenced.
Main entry points: 01_data_loading.py JSON loader · raw Raw JSON corpus · count.people.csv People counts · count.organizations.csv Organization counts
Components 7
Data processing
- JSON loader
scripts/01_data_loading.py - Entity dataframe
scripts/01_data_loading.py
Dataset files
- Raw JSON corpus
data/raw - People counts
data/interim/count.people.csv - Organization counts
data/interim/count.organizations.csv - Location counts
data/interim/count.locations.csv
Other components
- Analyst
Connections 6
- Analyst → JSON loader: runs
- JSON loader → Raw JSON corpus: reads
- JSON loader → Entity dataframe: extracts entities
- JSON loader → People counts: writes counts
- JSON loader → Organization counts: writes counts
- JSON loader → Location counts: writes counts