Methods for Evaluating Interactive Information Retrieval Systems with Users PDF Download
Are you looking for read ebook online? Search for your book and save it on your Kindle device, PC, phones or tablets. Download Methods for Evaluating Interactive Information Retrieval Systems with Users PDF full book. Access full book title Methods for Evaluating Interactive Information Retrieval Systems with Users by Diane Kelly. Download full books in PDF and EPUB format.
Author: Diane Kelly
Publisher: Now Publishers Inc
ISBN: 1601982240
Category : Database management
Languages : en
Pages : 246
Get Book Here
Book Description
Provides an overview and instruction on the evaluation of interactive information retrieval systems with users.
Author: Diane Kelly
Publisher: Now Publishers Inc
ISBN: 1601982240
Category : Database management
Languages : en
Pages : 246
Get Book Here
Book Description
Provides an overview and instruction on the evaluation of interactive information retrieval systems with users.
Author: Xie, Iris
Publisher: IGI Global
ISBN: 1599042428
Category : Computers
Languages : en
Pages : 375
Get Book Here
Book Description
"This book includes the integration of existing frameworks on user-oriented information retrieval systems across multiple disciplines; the comprehensive review of empirical studies of interactive information retrieval systems for different types of users, tasks, and subtasks; and the discussion of how to evaluate interactive information retrieval systems. "--Provided by publisher.
Author: Jiqun Liu
Publisher: Springer Nature
ISBN: 3031232291
Category : Computers
Languages : en
Pages : 220
Get Book Here
Book Description
This book brings together the insights from three different areas, Information Seeking and Retrieval, Cognitive Psychology, and Behavioral Economics, and shows how this new interdisciplinary approach can advance our knowledge about users interacting with diverse search systems, especially their seemingly irrational decisions and anomalies that could not be predicted by most normative models. The first part “Foundation” of this book introduces the general notions and fundamentals of this new approach, as well as the main concepts, terminology and theories. The second part “Beyond Rational Agents” describes the systematic biases and cognitive limits confirmed by behavioral experiments of varying types and explains in detail how they contradict the assumptions and predictions of formal models in information retrieval (IR). The third part “Toward A Behavioral Economics Approach” first synthesizes the findings from existing preliminary research on bounded rationality and behavioral economics modeling in information seeking, retrieval, and recommender system communities. Then, it discusses the implications, open questions and methodological challenges of applying the behavioral economics framework to different sub-areas of IR research and practices, such as modeling users and search sessions, developing unbiased learning to rank and adaptive recommendations algorithms, implementing bias-aware intelligent task support, as well as extending the conceptualization and evaluation on IR fairness, accountability, transparency and ethics (FATE) with the knowledge regarding both human biases and algorithmic biases. This book introduces a behavioral economics framework to IR scientists seeking a new perspective on both fundamental and new emerging problems of IR as well as the development and evaluation of bias-aware intelligent information systems. It is especially intended for researchers working on IR and human-information interaction who want to learn about the potential offered by behavioral economics in their own research areas.
Author: Jiqun Liu
Publisher: Springer Nature
ISBN: 3031023196
Category : Computers
Languages : en
Pages : 75
Get Book Here
Book Description
Since user study design has been widely applied in search interactions and information retrieval (IR) systems evaluation studies, a deep reflection and meta-evaluation of interactive IR (IIR) user studies is critical for sharpening the instruments of IIR research and improving the reliability and validity of the conclusions drawn from IIR user studies. To this end, we developed a faceted framework for supporting user study design, reporting, and evaluation based on a systematic review of the state-of-the-art IIR research papers recently published in several top IR venues (n=462). Within the framework, we identify three major types of research focuses, extract and summarize facet values from specific cases, and highlight the under-reported user study components which may significantly affect the results of research. Then, we employ the faceted framework in evaluating a series of IIR user studies against their respective research questions and explain the roles and impacts of the underlying connections and "collaborations" among different facet values. Through bridging diverse combinations of facet values with the study design decisions made for addressing research problems, the faceted framework can shed light on IIR user study design, reporting, and evaluation practices and help students and young researchers design and assess their own studies.
Author: Christopher D. Manning
Publisher: Cambridge University Press
ISBN: 1139472100
Category : Computers
Languages : en
Pages :
Get Book Here
Book Description
Class-tested and coherent, this textbook teaches classical and web information retrieval, including web search and the related areas of text classification and text clustering from basic concepts. It gives an up-to-date treatment of all aspects of the design and implementation of systems for gathering, indexing, and searching documents; methods for evaluating systems; and an introduction to the use of machine learning methods on text collections. All the important ideas are explained using examples and figures, making it perfect for introductory courses in information retrieval for advanced undergraduates and graduate students in computer science. Based on feedback from extensive classroom experience, the book has been carefully structured in order to make teaching more natural and effective. Slides and additional exercises (with solutions for lecturers) are also available through the book's supporting website to help course instructors prepare their lectures.
Author: Donna Harman
Publisher: Springer Nature
ISBN: 3031022769
Category : Computers
Languages : en
Pages : 107
Get Book Here
Book Description
Evaluation has always played a major role in information retrieval, with the early pioneers such as Cyril Cleverdon and Gerard Salton laying the foundations for most of the evaluation methodologies in use today. The retrieval community has been extremely fortunate to have such a well-grounded evaluation paradigm during a period when most of the human language technologies were just developing. This lecture has the goal of explaining where these evaluation methodologies came from and how they have continued to adapt to the vastly changed environment in the search engine world today. The lecture starts with a discussion of the early evaluation of information retrieval systems, starting with the Cranfield testing in the early 1960s, continuing with the Lancaster "user" study for MEDLARS, and presenting the various test collection investigations by the SMART project and by groups in Britain. The emphasis in this chapter is on the how and the why of the various methodologies developed. The second chapter covers the more recent "batch" evaluations, examining the methodologies used in the various open evaluation campaigns such as TREC, NTCIR (emphasis on Asian languages), CLEF (emphasis on European languages), INEX (emphasis on semi-structured data), etc. Here again the focus is on the how and why, and in particular on the evolving of the older evaluation methodologies to handle new information access techniques. This includes how the test collection techniques were modified and how the metrics were changed to better reflect operational environments. The final chapters look at evaluation issues in user studies -- the interactive part of information retrieval, including a look at the search log studies mainly done by the commercial search engines. Here the goal is to show, via case studies, how the high-level issues of experimental design affect the final evaluations. Table of Contents: Introduction and Early History / "Batch" Evaluation Since 1992 / Interactive Evaluation / Conclusion
Author: Maristella Agosti
Publisher: Springer
ISBN: 3642364152
Category : Computers
Languages : en
Pages : 184
Get Book Here
Book Description
The research domains information retrieval and information visualization have always been independent from each other. However, they have the potential to be mutually beneficial. With this in mind, a writer school was organized in Zinal, Switzerland, in January 2012, within the context of the EU-funded research project PROMISE (Participative Research Laboratory for Multimedia and Multilingual Information Systems Evaluation). PROMISE aims at advancing the experimental evaluation of complex multimedia and multilingual information systems in order to support individuals, commercial entities, and communities who design, develop, employ, and improve such complex systems. The overall goal of PROMISE is to deliver a unified environment collecting data, knowledge, tools, and methodologies, and to help the user community involved in experimental evaluation. This book constitutes the outcome of the PROMISE Winter School 2012 and contains 11 invited lectures from the research domains information retrieval and information visualization. A large variety of subjects are covered, including hot topics such as crowdsourcing and social media.
Author: Ian Ruthven
Publisher: Facet Publishing
ISBN: 1856047075
Category : Computers
Languages : en
Pages : 337
Get Book Here
Book Description
Information retrieval (IR) is a complex human activity supported by sophisticated systems. Information science has contributed much to the design and evaluation of previous generations of IR system development and to our general understanding of how such systems should be designed and yet, due to the increasing success and diversity of IR systems, many recent textbooks concentrate on IR systems themselves and ignore the human side of searching for information. This book is the first text to provide an information science perspective on IR. Unique in its scope, the book covers the whole spectrum of information retrieval, including: history and background information behaviour and seeking task-based information searching and retrieval approaches to investigating information interaction and behaviour information representation access models evaluation interfaces for IR interactive techniques web retrieval, ranking and personalization recommendation, collaboration and social search multimedia: interfaces and access. Readership: Senior undergraduates and masters' level students of all information and library studies courses and practising LIS professionals who need to better appreciate how IR systems are designed, implemented and evaluated.
Author: Nazli Goharian
Publisher: Springer Nature
ISBN: 3031560698
Category :
Languages : en
Pages : 529
Get Book Here
Book Description
Author: Pavel Serdyukov
Publisher: Springer
ISBN: 3642369731
Category : Computers
Languages : en
Pages : 919
Get Book Here
Book Description
This book constitutes the proceedings of the 35th European Conference on IR Research, ECIR 2013, held in Moscow, Russia, in March 2013. The 55 full papers, 38 poster papers and 10 demonstrations presented in this volume were carefully reviewed and selected from 287 submissions. The papers are organized in the following topical sections: user aspects; multimedia and cross-media IR; data mining; IR theory and formal models; IR system architectures; classification; Web; event detection; temporal IR, and microblog search. Also included are 4 tutorial and 2 workshop presentations.
