DIP-SumEval: A Data Set of Human Summary Evaluations
Date
2019-11Description
This repository contains the summaries and evaluations from the paper 'A Dataset for the Analysis of Text Quality Dimensions in Summarization Evaluation' presented at LREC 2020. In total, the data set contains over 400 automatically generated summaries for 49 topics of an data set for multi-document summarization, 1274 judgements according to 11 text and summary quality criteria on a Likert-scale (1 to 5) performed by 26 trained annotators, and 43218 pairwise judgements according to 6 criteria performed by 64 crowd-workers.
If you want to use the data please cite the paper http://www.ke.tu-darmstadt.de/bibtex/publications/show/3200
DFG subject classification
1.14-03 Angewandte Sprachwissenschaften, Computerlinguistik4.43-04 Künstliche Intelligenz und Maschinelle Lernverfahren
4.43-05 Bild- und Sprachverarbeitung, Computergraphik und Visualisierung, Human Computer Interaction, Ubiquitous und Wearable Computing
Related third party funded projects
DFG | GRK1994 | Prof. Roth / AIPHESRelated Resources
- Is supplement to: http://www.ke.tu-darmstadt.de/bibtex/publications/show/3200
- Is original form of: https://github.com/keelm/DIP-SumEval
Collections
The following license files are associated with this item:
Except where otherwise noted, this item's license is described as Creative Commons Attribution 4.0