How 30aniket/neural-language-models-for-biomedical-causal-inference works
This repository is a biomedical causal-inference model-comparison study over two FAERS benchmarks. Its main workflow prepares benchmark records, trains a broad set of classifiers under five-fold evaluation, evaluates predictions and calibration, then tests term-level causal outputs and audits methodological claims. The graph keeps the two benchmark-specific training suites distinct while showing shared evaluation and audit layers. Some training and evaluation internals were not excerpted; their links are included only where the README or shown source supports them.
Main entry points: common.py Check data helpers · run_evaluation.py Fold predictions · Analgesics-induced_acute_liver_failure AILF model suite · prepare_data.py Data preparation
Components 19
Benchmark data
- Data preparation
data/prepare_data.py - AILF benchmark
data/prepare_data.py - TRAM benchmark
data/prepare_data.py
Model training
- AILF model suite
src/Analgesics-induced_acute_liver_failure - TRAM model suite
src/Tramadol-related_mortalities - XGBoost features
src/Tramadol-related_mortalities/xgb_train_src.py - Text classifiers
src/Analgesics-induced_acute_liver_failure/gemma_peft_train.py
Evaluation
- Fold predictions
src/run_evaluation.py - Isotonic calibration
src/run_evaluation.py - Prediction metrics
src/run_evaluation.py - Causal term tests
src/run_evaluation.py - Significant term sets
src/run_evaluation.py
Audits
- Check data helpers
feedback_work/feedback_checks/common.py - Protocol and claim audits
feedback_work/feedback_checks/check05_pipeline_and_claims.py - Reference method audit
feedback_work/feedback_checks/check03_reference_methods.py - Term score reimplementation
feedback_work/feedback_checks/zcore.py - Endpoint leakage audit
feedback_work/feedback_checks/check01_leakage.py - Repeated case audit
feedback_work/feedback_checks/check02_duplicates.py
Other components
- Researcher
Connections 21
- Researcher → Data preparation: prepares benchmarks
README.md - Data preparation → AILF benchmark: prepares records
README.md - Data preparation → TRAM benchmark: prepares records
README.md - AILF benchmark → AILF model suite: trains on
README.md - TRAM benchmark → TRAM model suite: trains on
README.md - AILF model suite → Text classifiers: includes text models
README.md - AILF model suite → XGBoost features: includes baseline
README.md - AILF model suite → Fold predictions: writes fold scores
README.md - TRAM model suite → Fold predictions: writes fold scores
README.md - Text classifiers → Fold predictions: writes predictions
src/Analgesics-induced_acute_liver_failure/gemma_peft_train.py - Fold predictions → Isotonic calibration: calibrates scores
README.md - Fold predictions → Prediction metrics: computes metrics
src/run_evaluation.py - Prediction metrics → Causal term tests: tests term scores
README.md - Causal term tests → Significant term sets: selects terms
src/run_evaluation.py - Researcher → Protocol and claim audits: runs audits
README.md - Check data helpers → Protocol and claim audits: loads audit inputs
feedback_work/feedback_checks/check05_pipeline_and_claims.py - Check data helpers → Reference method audit: loads audit inputs
feedback_work/feedback_checks/check03_reference_methods.py - Check data helpers → Endpoint leakage audit: loads reports
feedback_work/feedback_checks/check01_leakage.py - Check data helpers → Repeated case audit: loads reports
feedback_work/feedback_checks/check02_duplicates.py - Check data helpers → Term score reimplementation: loads predictions
feedback_work/feedback_checks/zcore.py - Term score reimplementation → Reference method audit: recomputes term scores
feedback_work/feedback_checks/check03_reference_methods.py