DIP-SumEval: A Data Set of Human Summary Evaluations
Description
This repository contains the summaries and evaluations from the paper 'A Dataset for the Analysis of Text Quality Dimensions in Summarization Evaluation' presented at LREC 2020. In total, the data set contains over 400 automatically generated summaries for 49 topics of an data set for multi-document summarization, 1274 judgements according to 11 text and summary quality criteria on a Likert-scale (1 to 5) performed by 26 trained annotators, and 43218 pairwise judgements according to 6 criteria performed by 64 crowd-workers.
If you want to use the data please cite the paper http://www.ke.tu-darmstadt.de/bibtex/publications/show/3200
DFG subject classification
104-04 Angewandte Sprachwissenschaften, Experimentelle Linguistik, Computerlinguistik409-05 Interaktive und intelligente Systeme, Bild- und Sprachverarbeitung, Computergraphik und Visualisierung
Related third party funded projects
DFG | GRK1994 | Prof. Roth / AIPHESRelated Resources
- Is supplement to: http://www.ke.tu-darmstadt.de/bibtex/publications/show/3200
- Is original form of: https://github.com/keelm/DIP-SumEval
Collections
The following license files are associated with this item: