Please use this identifier to cite or link to this item:
http://elar.urfu.ru/handle/10995/4561
Title: | Automatic summary evaluation. Roug e modifications |
Authors: | Ermakova, Liana |
Issue Date: | 2012 |
Citation: | Ermakova L. Automatic summary evaluation. Roug e modifications / L. Ermakova // VI Russian Summer School in Information Retrieval, August 6–10, 2012. Proceedings of the Sixth Russian Young Scientists Conference in Information Retrieval / B. Sokolov, P. Braslavski (Eds.). — Yaroslavl, 2012. — P. 58-70. |
Abstract: | Nowadays there is no common approach to summary. Manual evaluation is expensive and subjective and it is not applicable in real time or on a large corpus. Widely used approaches involve little human efforts and assume comparison with a set of reference summaries. We tried to overcome drawbacks of existing metrics such as ignoring redundant information, synonyms and sentence ordering. Our method combines edit distance, ROUGE-SU and trigrams similarity measure enriched by weights for different parts of speech and synonyms. Since nouns provide the most valuable information, each sentence is mapped into a set of nouns. If the normalized intersection of any pair is greater than a predefined threshold the sentences are penalized. Doing extracts there is no need to analyze sentence structure but sentence ordering is crucial. Sometimes it is impossible to compare sentence order with a gold standard. Therefore similarity between adjacent sentences may be used as a measure of text coherence. Chronological constraint violation should be penalized. Relevance score and readability assessment may be combined in the F-measure. In order to choose the best parameter values machine learning can be applied. |
Keywords: | ИНФОРМАТИКА ИНФОРМАЦИОННЫЙ ПОИСК В ИНТЕРНЕТЕ ПОИСК ИНФОРМАЦИИ В ИНТЕРНЕТЕ КОНФЕРЕНЦИИ AUTOMATIC SUMMARY EVALUATION ROUGE SUMMARIZATION EDIT DISTANCE READABILITY SENTENCE ORDERING REDUNDANT INFORMATION |
URI: | http://elar.urfu.ru/handle/10995/4561 |
Conference name: | VI Российская летняя школа по информационному поиску (RuSSIR’2012) VI Russian Summer School in Information Retrieval (RuSSIR’2012) |
Conference date: | 6.08.2012–10.08.2012 |
ISBN: | 978-5-8397-0888-4 |
Origin: | RuSSIR 2012 |
Appears in Collections: | Информационный поиск |
Files in This Item:
File | Description | Size | Format | |
---|---|---|---|---|
RuSSIR_2012_06.pdf | 189,2 kB | Adobe PDF | View/Open |
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.