How utrechtuniversity/hta-genai works
This repository extracts structured HTA attributes from PDF documents with configurable LLMs, then optionally evaluates the results against ground truth. The main workflow is CLI configuration → PDF text extraction and schema-guided prompting → model response parsing and saved JSON → comparison and metrics. The graph follows the sampled runtime code; evaluation internals are only partially sampled, so uncertain evaluation substeps are not expanded.
Main entry points: run.py Run workflow · extract_hta_attributes.py Extraction runner · evaluate_results.py Result evaluator · hta_extractor.py Document extractor
Components 14
Orchestration
- Run workflow
run.py - Runtime config
config/config.yaml
HTA Extraction
- Extraction runner
hta/extraction/extract_hta_attributes.py - Document extractor
hta/extraction/hta_extractor.py - Prompt template
config/prompt_template.yaml - Output schema
config/schema.json - Extraction results
data/processed
LLM Integration
- LLM client
hta/llm_client.py
Evaluation
- Ground truth
data/raw - Result evaluator
hta/evaluation/evaluate_results.py - Evaluation outputs
data/processed
Other components
- Researcher
- HTA PDFs
- LLM providers
Connections 19
- Researcher → Run workflow: starts
- Run workflow → Runtime config: loads
- Run workflow → Extraction runner: runs extraction
- Extraction runner → Runtime config: reads
- Extraction runner → HTA PDFs: discovers PDFs
- Extraction runner → Document extractor: processes each
- Document extractor → Prompt template: loads
- Document extractor → Output schema: loads
- Document extractor → LLM client: requests completion
- LLM client → LLM providers: sends prompt
- LLM providers → LLM client: returns response
- Extraction runner → Extraction results: appends JSON
- Run workflow → Ground truth: checks availability
- Run workflow → Result evaluator: runs evaluation
- Result evaluator → Extraction results: reads results
- Result evaluator → Ground truth: compares against
- Result evaluator → Output schema: reads descriptions
- Result evaluator → LLM client: requests comparison
- Run workflow → Evaluation outputs: writes comparisons and metrics