research
∙
05/22/2023
SEAHORSE: A Multilingual, Multifaceted Dataset for Summarization Evaluation
Reliable automatic evaluation of summarization systems is challenging du...
share
research
∙
12/20/2022
mFACE: Multilingual Summarization with Factual Consistency Evaluation
Abstractive summarization has enjoyed renewed interest in recent years, ...
research
∙
12/20/2022
Needle in a Haystack: An Analysis of Finding Qualified Workers on MTurk for Summarization
The acquisition of high-quality human annotations through crowdsourcing ...
research
∙
11/02/2022
Dialect-robust Evaluation of Generated Text
Evaluation metrics that are not robust to dialect variation make it impo...
research
∙
02/14/2022
Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text
Evaluation practices in natural language generation (NLG) have many know...
research
∙
06/30/2021
All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Text
Human evaluations are typically considered the gold standard in natural ...
research
∙
06/26/2020
Evaluation of Text Generation: A Survey
The paper surveys evaluation methods of natural language generation (NLG...
research
∙
04/07/2020
Evaluating Machines by their Real-World Language Use
There is a fundamental gap between how humans understand and use languag...
research
∙
09/09/2019
Counterfactual Story Reasoning and Generation
Counterfactual reasoning requires predicting how alternative events, con...
research
∙
04/26/2018