How uci-soe/fairytaleqadata works
Project type and purpose
• This is an open-source dataset repository (“FairytaleQA”) designed for NLP researchers and educators.
• It provides a rich collection of CSV files (stories, QA-pairs, sentences) plus helper code to load, split, and preprocess data, and a Huggingface integration for easy consumption.
File structure
• Top-level:
• data-by-origin/
• data-by-train-split/
• img/ (charts)
• starter.py (data loader entrypoint)
• story_meta.csv
• Under each data-by-* directory there are three parallel subtrees:
• questions/ (expert-labeled QA pairs)
• section-stories/ (paragraph-level splits)
• sentence-stories/ (sentence-level splits, via spaCy)
• Each split namespace (origin vs. train/test/val) repeats the same structure.
• Configuration: README.md describes usage; no database or web server; purely file-based.
README insights
• Explains two splitting strategies: by origin and by train/val/test.
• Mentions starter.py for retrieving and aggregating data using pandas.
• References spaCy’s en_core_web_sm for sentence segmentation.
• Mentions Huggingface Dataset loading.