Please use this identifier to cite or link to this item: http://hdl.handle.net/10995/4561
Title: Automatic summary evaluation. Roug e modifications
Authors: Ermakova, Liana
Issue Date: 2012
Citation: Ermakova L. Automatic summary evaluation. Roug e modifications / L. Ermakova // VI Russian Summer School in Information Retrieval, August 6–10, 2012. Proceedings of the Sixth Russian Young Scientists Conference in Information Retrieval / B. Sokolov, P. Braslavski (Eds.). — Yaroslavl, 2012. — P. 58-70.
Abstract: Nowadays there is no common approach to summary. Manual evaluation is expensive and subjective and it is not applicable in real time or on a large corpus. Widely used approaches involve little human efforts and assume comparison with a set of reference summaries. We tried to overcome drawbacks of existing metrics such as ignoring redundant information, synonyms and sentence ordering. Our method combines edit distance, ROUGE-SU and trigrams similarity measure enriched by weights for different parts of speech and synonyms. Since nouns provide the most valuable information, each sentence is mapped into a set of nouns. If the normalized intersection of any pair is greater than a predefined threshold the sentences are penalized. Doing extracts there is no need to analyze sentence structure but sentence ordering is crucial. Sometimes it is impossible to compare sentence order with a gold standard. Therefore similarity between adjacent sentences may be used as a measure of text coherence. Chronological constraint violation should be penalized. Relevance score and readability assessment may be combined in the F-measure. In order to choose the best parameter values machine learning can be applied.
Keywords: ИНФОРМАТИКА
ИНФОРМАЦИОННЫЙ ПОИСК В ИНТЕРНЕТЕ
ПОИСК ИНФОРМАЦИИ В ИНТЕРНЕТЕ
КОНФЕРЕНЦИИ
AUTOMATIC SUMMARY EVALUATION
ROUGE
SUMMARIZATION
EDIT DISTANCE
READABILITY
SENTENCE ORDERING
REDUNDANT INFORMATION
URI: http://hdl.handle.net/10995/4561
https://elar.urfu.ru/handle/10995/4561
Conference name: VI Российская летняя школа по информационному поиску (RuSSIR’2012)
VI Russian Summer School in Information Retrieval (RuSSIR’2012)
Conference date: 6.08.2012–10.08.2012
ISBN: 978-5-8397-0888-4
Origin: RuSSIR 2012
Appears in Collections:Информационный поиск

Files in This Item:
File Description SizeFormat 
RuSSIR_2012_06.pdf189,2 kBAdobe PDFView/Open


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.