Methods For Evaluating Interactive Information Retrieval Systems With Users

Author : Diane Kelly
Publisher : Now Publishers Inc
ISBN 13 : 1601982240
Total Pages : 246 pages
Book Rating : 4.47/5 ( download)

DOWNLOAD NOW!

Download or read book Methods for Evaluating Interactive Information Retrieval Systems with Users written by Diane Kelly and published by Now Publishers Inc. This book was released on 2009 with total page 246 pages. Available in PDF, EPUB and Kindle. Book excerpt: Provides an overview and instruction on the evaluation of interactive information retrieval systems with users.

Author : Donna K. Harman
Publisher : Morgan & Claypool Publishers
ISBN 13 : 1598299719
Total Pages : 122 pages
Book Rating : 4.17/5 ( download)

DOWNLOAD NOW!

Download or read book Information Retrieval Evaluation written by Donna K. Harman and published by Morgan & Claypool Publishers. This book was released on 2011 with total page 122 pages. Available in PDF, EPUB and Kindle. Book excerpt: Evaluation has always played a major role in information retrieval, with the early pioneers such as Cyril Cleverdon and Gerard Salton laying the foundations for most of the evaluation methodologies in use today. The retrieval community has been extremely fortunate to have such a well-grounded evaluation paradigm during a period when most of the human language technologies were just developing. This lecture has the goal of explaining where these evaluation methodologies came from and how they have continued to adapt to the vastly changed environment in the search engine world today. The lecture starts with a discussion of the early evaluation of information retrieval systems, starting with the Cranfield testing in the early 1960s, continuing with the Lancaster "user" study for MEDLARS, and presenting the various test collection investigations by the SMART project and by groups in Britain. The emphasis in this chapter is on the how and the why of the various methodologies developed. The second chapter covers the more recent "batch" evaluations, examining the methodologies used in the various open evaluation campaigns such as TREC, NTCIR (emphasis on Asian languages), CLEF (emphasis on European languages), INEX (emphasis on semi-structured data), etc. Here again the focus is on the how and why, and in particular on the evolving of the older evaluation methodologies to handle new information access techniques. This includes how the test collection techniques were modified and how the metrics were changed to better reflect operational environments. The final chapters look at evaluation issues in user studies -- the interactive part of information retrieval, including a look at the search log studies mainly done by the commercial search engines. Here the goal is to show, via case studies, how the high-level issues of experimental design affect the final evaluations. Table of Contents: Introduction and Early History / "Batch" Evaluation Since 1992 / Interactive Evaluation / Conclusion

Author : Christopher D. Manning
Publisher : Cambridge University Press
ISBN 13 : 1139472100
Total Pages : pages
Book Rating : 4.04/5 ( download)

DOWNLOAD NOW!

Download or read book Introduction to Information Retrieval written by Christopher D. Manning and published by Cambridge University Press. This book was released on 2008-07-07 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt: Class-tested and coherent, this textbook teaches classical and web information retrieval, including web search and the related areas of text classification and text clustering from basic concepts. It gives an up-to-date treatment of all aspects of the design and implementation of systems for gathering, indexing, and searching documents; methods for evaluating systems; and an introduction to the use of machine learning methods on text collections. All the important ideas are explained using examples and figures, making it perfect for introductory courses in information retrieval for advanced undergraduates and graduate students in computer science. Based on feedback from extensive classroom experience, the book has been carefully structured in order to make teaching more natural and effective. Slides and additional exercises (with solutions for lecturers) are also available through the book's supporting website to help course instructors prepare their lectures.

Author : Jiqun Liu
Publisher : Springer Nature
ISBN 13 : 3031023196
Total Pages : 75 pages
Book Rating : 4.94/5 ( download)

DOWNLOAD NOW!

Download or read book Interactive IR User Study Design, Evaluation, and Reporting written by Jiqun Liu and published by Springer Nature. This book was released on 2022-05-31 with total page 75 pages. Available in PDF, EPUB and Kindle. Book excerpt: Since user study design has been widely applied in search interactions and information retrieval (IR) systems evaluation studies, a deep reflection and meta-evaluation of interactive IR (IIR) user studies is critical for sharpening the instruments of IIR research and improving the reliability and validity of the conclusions drawn from IIR user studies. To this end, we developed a faceted framework for supporting user study design, reporting, and evaluation based on a systematic review of the state-of-the-art IIR research papers recently published in several top IR venues (n=462). Within the framework, we identify three major types of research focuses, extract and summarize facet values from specific cases, and highlight the under-reported user study components which may significantly affect the results of research. Then, we employ the faceted framework in evaluating a series of IIR user studies against their respective research questions and explain the roles and impacts of the underlying connections and "collaborations" among different facet values. Through bridging diverse combinations of facet values with the study design decisions made for addressing research problems, the faceted framework can shed light on IIR user study design, reporting, and evaluation practices and help students and young researchers design and assess their own studies.

Author : Ian Ruthven
Publisher : Facet Publishing
ISBN 13 : 1856047075
Total Pages : 337 pages
Book Rating : 4.74/5 ( download)

DOWNLOAD NOW!

Download or read book Interactive Information Seeking, Behaviour and Retrieval written by Ian Ruthven and published by Facet Publishing. This book was released on 2011 with total page 337 pages. Available in PDF, EPUB and Kindle. Book excerpt: Information retrieval (IR) is a complex human activity supported by sophisticated systems. Information science has contributed much to the design and evaluation of previous generations of IR system development and to our general understanding of how such systems should be designed and yet, due to the increasing success and diversity of IR systems, many recent textbooks concentrate on IR systems themselves and ignore the human side of searching for information. This book is the first text to provide an information science perspective on IR. Unique in its scope, the book covers the whole spectrum of information retrieval, including: history and background information behaviour and seeking task-based information searching and retrieval approaches to investigating information interaction and behaviour information representation access models evaluation interfaces for IR interactive techniques web retrieval, ranking and personalization recommendation, collaboration and social search multimedia: interfaces and access. Readership: Senior undergraduates and masters' level students of all information and library studies courses and practising LIS professionals who need to better appreciate how IR systems are designed, implemented and evaluated.

Author : Katja Hofmann
Publisher :
ISBN 13 : 9781680831634
Total Pages : 134 pages
Book Rating : 4.31/5 ( download)

DOWNLOAD NOW!

Download or read book Online Evaluation for Information Retrieval written by Katja Hofmann and published by . This book was released on 2016-06-07 with total page 134 pages. Available in PDF, EPUB and Kindle. Book excerpt: Provides a comprehensive overview of the topic. It shows how online evaluation is used for controlled experiments, segmenting them into experiment designs that allow absolute or relative quality assessments. It also includes an extensive discussion of recent work on data re-use, and experiment estimation based on historical data.

Author : Mark Sanderson
Publisher : Now Publishers Inc
ISBN 13 : 1601983603
Total Pages : 143 pages
Book Rating : 4.02/5 ( download)

DOWNLOAD NOW!

Download or read book Test Collection Based Evaluation of Information Retrieval Systems written by Mark Sanderson and published by Now Publishers Inc. This book was released on 2010-06-03 with total page 143 pages. Available in PDF, EPUB and Kindle. Book excerpt: Use of test collections and evaluation measures to assess the effectiveness of information retrieval systems has its origins in work dating back to the early 1950s. Across the nearly 60 years since that work started, use of test collections is a de facto standard of evaluation. This monograph surveys the research conducted and explains the methods and measures devised for evaluation of retrieval systems, including a detailed look at the use of statistical significance testing in retrieval experimentation. This monograph reviews more recent examinations of the validity of the test collection approach and evaluation measures as well as outlining trends in current research exploiting query logs and live labs. At its core, the modern-day test collection is little different from the structures that the pioneering researchers in the 1950s and 1960s conceived of. This tutorial and review shows that despite its age, this long-standing evaluation method is still a highly valued tool for retrieval research.

Author : Fabio Crestani
Publisher : Springer
ISBN 13 : 3319607774
Total Pages : 110 pages
Book Rating : 4.71/5 ( download)

DOWNLOAD NOW!

Download or read book Mobile Information Retrieval written by Fabio Crestani and published by Springer. This book was released on 2017-07-19 with total page 110 pages. Available in PDF, EPUB and Kindle. Book excerpt: This book offers a helpful starting point in the scattered, rich, and complex body of literature on Mobile Information Retrieval (Mobile IR), reviewing more than 200 papers in nine chapters. Highlighting the most interesting and influential contributions that have appeared in recent years, it particularly focuses on both user interaction and techniques for the perception and use of context, which, taken together, shape much of today’s research on Mobile IR. The book starts by addressing the differences between IR and Mobile IR, while also reviewing the foundations of Mobile IR research. It then examines the different kinds of documents, users, and information needs that can be found in Mobile IR, and which set it apart from standard IR. Next, it discusses the two important issues of user interfaces and context-awareness. In closing, it covers issues related to the evaluation of Mobile IR applications. Overall, the book offers a valuable tool, helping new and veteran researchers alike to navigate this exciting and highly dynamic area of research.

Author : Donna Harman
Publisher : Springer Nature
ISBN 13 : 3031022769
Total Pages : 107 pages
Book Rating : 4.60/5 ( download)

DOWNLOAD NOW!

Download or read book Information Retrieval Evaluation written by Donna Harman and published by Springer Nature. This book was released on 2022-05-31 with total page 107 pages. Available in PDF, EPUB and Kindle. Book excerpt: Evaluation has always played a major role in information retrieval, with the early pioneers such as Cyril Cleverdon and Gerard Salton laying the foundations for most of the evaluation methodologies in use today. The retrieval community has been extremely fortunate to have such a well-grounded evaluation paradigm during a period when most of the human language technologies were just developing. This lecture has the goal of explaining where these evaluation methodologies came from and how they have continued to adapt to the vastly changed environment in the search engine world today. The lecture starts with a discussion of the early evaluation of information retrieval systems, starting with the Cranfield testing in the early 1960s, continuing with the Lancaster "user" study for MEDLARS, and presenting the various test collection investigations by the SMART project and by groups in Britain. The emphasis in this chapter is on the how and the why of the various methodologies developed. The second chapter covers the more recent "batch" evaluations, examining the methodologies used in the various open evaluation campaigns such as TREC, NTCIR (emphasis on Asian languages), CLEF (emphasis on European languages), INEX (emphasis on semi-structured data), etc. Here again the focus is on the how and why, and in particular on the evolving of the older evaluation methodologies to handle new information access techniques. This includes how the test collection techniques were modified and how the metrics were changed to better reflect operational environments. The final chapters look at evaluation issues in user studies -- the interactive part of information retrieval, including a look at the search log studies mainly done by the commercial search engines. Here the goal is to show, via case studies, how the high-level issues of experimental design affect the final evaluations. Table of Contents: Introduction and Early History / "Batch" Evaluation Since 1992 / Interactive Evaluation / Conclusion

Author : Xie, Iris
Publisher : IGI Global
ISBN 13 : 1599042428
Total Pages : 376 pages
Book Rating : 4.28/5 ( download)

DOWNLOAD NOW!

Download or read book Interactive Information Retrieval in Digital Environments written by Xie, Iris and published by IGI Global. This book was released on 2008-04-30 with total page 376 pages. Available in PDF, EPUB and Kindle. Book excerpt: "This book includes the integration of existing frameworks on user-oriented information retrieval systems across multiple disciplines; the comprehensive review of empirical studies of interactive information retrieval systems for different types of users, tasks, and subtasks; and the discussion of how to evaluate interactive information retrieval systems. "--Provided by publisher.