Volltext herunterladen
(507.2 KB)
Zitationshinweis
Bitte beziehen Sie sich beim Zitieren dieses Dokumentes immer auf folgenden Persistent Identifier (PID):
https://nbn-resolving.org/urn:nbn:de:0168-ssoar-86702-6
Export für Ihre Literaturverwaltung
Evaluation Infrastructures for Academic Shared Tasks: Requirements and Concept Design for Search and Recommendation Scenarios
[Zeitschriftenartikel]
Abstract Academic search systems aid users in finding information covering specific topics of scientific interest and have evolved from early catalog-based library systems to modern web-scale systems. However, evaluating the performance of the underlying retrieval approaches remains a challenge. An increasin... mehr
Academic search systems aid users in finding information covering specific topics of scientific interest and have evolved from early catalog-based library systems to modern web-scale systems. However, evaluating the performance of the underlying retrieval approaches remains a challenge. An increasing amount of requirements for producing accurate retrieval results have to be considered, e.g., close integration of the system’s users. Due to these requirements, small to mid-size academic search systems cannot evaluate their retrieval system in-house. Evaluation infrastructures for shared tasks alleviate this situation. They allow researchers to experiment with retrieval approaches in specific search and recommendation scenarios without building their own infrastructure. In this paper, we elaborate on the benefits and shortcomings of four state-of-the-art evaluation infrastructures on search and recommendation tasks concerning the following requirements: support for online and offline evaluations, domain specificity of shared tasks, and reproducibility of experiments and results. In addition, we introduce an evaluation infrastructure concept design aiming at reducing the shortcomings in shared tasks for search and recommender systems.... weniger
Thesaurusschlagwörter
information retrieval
Klassifikation
Information und Dokumentation, Bibliotheken, Archive
Freie Schlagwörter
Domain-Specific Academic Search; Evaluation Infrastructures; Online/Offline Evaluation; Reproducibility
Sprache Dokument
Englisch
Publikationsjahr
2020
Seitenangabe
S. 29-36
Zeitschriftentitel
Datenbank-Spektrum : Zeitschrift für Datenbanktechnologien und Information Retrieval, 20 (2020) 1
Heftthema
Trends in Information Retrieval Evaluation
DOI
https://doi.org/10.1007/s13222-020-00335-x
ISSN
1610-1995
Status
Veröffentlichungsversion; begutachtet (peer reviewed)
Lizenz
Creative Commons - Namensnennung 4.0
FörderungGefördert durch die Deutsche Forschungsgemeinschaft (DFG) - Projektnummer 407518790 / Funded by the German Research Foundation (DFG) - Project number 407518790