Пожалуйста, используйте этот идентификатор, чтобы цитировать или ссылаться на этот ресурс:
http://elar.urfu.ru/handle/10995/111587
Название: | A Systematic Evaluation of Transfer Learning and Pseudo-labeling with BERT-based Ranking Models |
Авторы: | Mokrii, I. Boytsov, L. Braslavski, P. |
Дата публикации: | 2021 |
Издатель: | Association for Computing Machinery, Inc ACM |
Библиографическое описание: | Mokrii I. A Systematic Evaluation of Transfer Learning and Pseudo-labeling with BERT-based Ranking Models / I. Mokrii, L. Boytsov, P. Braslavski // SIGIR 2021 - Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. — 2021. — Vol. — P. 2081-2085. — 3463093. |
Аннотация: | Due to high annotation costs making the best use of existing human-created training data is an important research direction. We, therefore, carry out a systematic evaluation of transferability of BERT-based neural ranking models across five English datasets. Previous studies focused primarily on zero-shot and few-shot transfer from a large dataset to a dataset with a small number of queries. In contrast, each of our collections has a substantial number of queries, which enables a full-shot evaluation mode and improves reliability of our results. Furthermore, since source datasets licences often prohibit commercial use, we compare transfer learning to training on pseudo-labels generated by a BM25 scorer. We find that training on pseudo-labels - -possibly with subsequent fine-tuning using a modest number of annotated queries - -can produce a competitive or better model compared to transfer learning. Yet, it is necessary to improve the stability and/or effectiveness of the few-shot training, which, sometimes, can degrade performance of a pretrained model. © 2021 ACM. |
Ключевые слова: | NEURAL INFORMATION RETRIEVAL PSEUDO-LABELING TRANSFER LEARNING INFORMATION RETRIEVAL LARGE DATASET LEARNING SYSTEMS TRANSFER LEARNING EVALUATION MODES FINE TUNING RANKING MODEL SYSTEMATIC EVALUATION TRAINING DATA LEARNING TO RANK |
URI: | http://elar.urfu.ru/handle/10995/111587 |
Условия доступа: | info:eu-repo/semantics/openAccess |
Конференция/семинар: | 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2021 |
Дата конференции/семинара: | 11 July 2021 through 15 July 2021 |
Идентификатор SCOPUS: | 85111638459 |
Идентификатор WOS: | 000719807900251 |
Идентификатор PURE: | 22990153 |
ISBN: | 9781450380379 |
DOI: | 10.1145/3404835.3463093 |
Сведения о поддержке: | Pavel Braslavski thanks the Ministry of Science and Higher Education of the Russian Federation (“Ural Mathematical Center” project). |
Располагается в коллекциях: | Научные публикации ученых УрФУ, проиндексированные в SCOPUS и WoS CC |
Файлы этого ресурса:
Файл | Описание | Размер | Формат | |
---|---|---|---|---|
2-s2.0-85111638459.pdf | 579,34 kB | Adobe PDF | Просмотреть/Открыть |
Все ресурсы в архиве электронных ресурсов защищены авторским правом, все права сохранены.