TUdatalib : Constrained C-Test Generation via Mixed-Integer Programming (Supplementary Material)

Show simple item record

dc.contributor.author	Lee, Ji-Ung
dc.contributor.author	Pfetsch, Marc
dc.contributor.author	Gurevych, Iryna
dc.date.accessioned	2024-04-08T09:54:18Z
dc.date.available	2024-04-08T09:54:18Z
dc.date.issued	2024-04
dc.identifier.uri	https://tudatalib.ulb.tu-darmstadt.de/handle/tudatalib/4205
dc.description	This work proposes a novel method to generate C-Tests; a deviated form of cloze tests (a gap filling exercise) where only the last part of a word is turned into a gap. In contrast to previous works that only consider varying the gap size or gap placement to achieve locally optimal solutions, we propose a mixed-integer programming (MIP) approach. This allows us to consider gap size and placement simultaneously, achieving globally optimal solutions and to directly integrate state-of-the-art models for gap difficulty prediction into the optimization problem. A user study with 40 participants across four C-Tests generation strategies (including GPT-4) shows that our approach (MIP) significantly outperforms two of the baseline strategies (based on gap placement and GPT-4); and performs on-par with the third (based on gap size). Our analysis shows that GPT-4 still struggles to fulfill explicit constraints during generation and that MIP produces C-Tests that correlate best with the perceived difficulty. We publish our code, model, and collected data consisting of 32 English C-Tests with 20 gaps each (3,200 in total) under an open source license.	de_DE
dc.language.iso	en	de_DE
dc.rights	Creative Commons Attribution 4.0
dc.rights.uri	https://creativecommons.org/licenses/by/4.0/
dc.subject	C-Test	de_DE
dc.subject	NLP	de_DE
dc.subject	Language Learning	de_DE
dc.subject	Constrained Optimization	de_DE
dc.subject	Machine Learning	de_DE
dc.subject.classification	409-05 Interaktive und intelligente Systeme, Bild- und Sprachverarbeitung, Computergraphik und Visualisierung	de_DE
dc.subject.ddc	004
dc.title	Constrained C-Test Generation via Mixed-Integer Programming (Supplementary Material)	de_DE
dc.type	Dataset	de_DE
dc.type	Text	de_DE
dc.type	Software	de_DE
tud.unit	TUDa