How kurocaguama/kurosagol works
Kurosagol is a thesis-research toolkit for studying LLM logical reasoning and alignment. The principal workflow takes FOLIO natural-language premises through translation to first-order logic, logical inference, and retranslation; generated answers are assembled into preference datasets, used for DPO alignment, and evaluated against FOLIO with LogicSim-style metrics. The repository also contains ongoing baseline and alignment experiments. Some ongoing behavior is represented only by notebooks and artifact directories, so their internal call wiring is not inferred.
Main entry points: Ongoing Ongoing experiments · create_pref_dataset.py Preference generation · llm_testing.py Baseline evaluation · align.py Alignment runner
Components 17
Datasets and outputs
- FOLIO corpus
FOLIO/FOLIO - Preference datasets
Legacy/Kurosagol/create_pref_dataset.py - Generated answers
Ongoing/alignment_results - Aligned model outputs
Ongoing/alignment_datasets
Reasoning generation
- Ongoing experiments
Ongoing - Stage prompts
Ongoing/prompts.py - Dataset generation
Legacy/Kurosagol/Kurosagol.py - Preference generation
Legacy/Kurosagol/create_pref_dataset.py
Preference and alignment
- DPO alignment
Legacy/Kurosagol/Kurosagol.py - Alignment runner
Legacy/Kurosagol/align.py
Evaluation
- Baseline evaluation
Legacy/Kurosagol/llm_testing.py - LogicSim evaluation
Legacy/Kurosagol/logic_sim.py - LogicSim+ metrics
Ongoing/logicsim.py - Text similarity
Legacy/Kurosagol/utils.py
Other components
- Researcher
- Language models
- Hugging Face Hub
Connections 25
- Researcher → Preference generation: runs stages
- FOLIO corpus → Preference generation: loads training data
- Preference generation → Dataset generation: creates manager
- Preference generation → Stage prompts: formats prompts
- Dataset generation → Language models: generates answers
- Dataset generation → Preference datasets: builds preference data
- Preference generation → Hugging Face Hub: uploads datasets
- Preference datasets → Alignment runner: supplies training data
- Researcher → Alignment runner: runs alignment
- Alignment runner → DPO alignment: trains model
- DPO alignment → Language models: loads base model
- Alignment runner → Hugging Face Hub: publishes models
- Researcher → Baseline evaluation: runs evaluation
- Baseline evaluation → FOLIO corpus: reads validation split
- Baseline evaluation → Language models: generates stage answers
- Baseline evaluation → Hugging Face Hub: publishes answer datasets
- Researcher → LogicSim evaluation: evaluates outputs
- LogicSim evaluation → Hugging Face Hub: loads answer datasets
- LogicSim evaluation → Text similarity: compares answers
- Researcher → Ongoing experiments: runs experiments
- Ongoing experiments → Stage prompts: uses stage prompts
- Ongoing experiments → Generated answers: records results
- Ongoing experiments → Aligned model outputs: records alignment data
- Ongoing experiments → LogicSim+ metrics: evaluates generations
- LogicSim+ metrics → FOLIO corpus: compares FOLIO answers