A formal framework for optimizing and evaluating interactive retrieval system

The past decades have seen dramatic increase in the amount of information available to us, and the research area of information retrieval has served to help us access the tiny subset of information relevant to each of us more efficiently and more effectively. This thesis studies how to formally optimize as well as evaluate an interactive information retrieval system. First, we propose a formal general framework, the Interface Card Model, for optimizing interactive retrieval interface. We frame the task of an interactive retrieval system as to choose a sequence of interface cards to present to the user that can maximize the expected gain of relevant information for the user while minimizing the effort of the user, with consideration of the user's action model and any desired constraints on the interface card. We show that such a formal Interface Card Model can not only cover the classic Probability Ranking Principle as a special case by making multiple simplification assumptions, but also be used to derive a novel formal interface model for adaptively optimizing navigational interfaces in a retrieval system. Second, we propose a novel formulation of the Interface Card Model, the Interface Card Model with User States, for solving concrete interface optimization problems. The formulation is based on sequential decision theory, leading to a general framework for formal modeling of user states and stopping actions. Simulation and user study experiments demonstrate the effectiveness of the proposed model in automatically adjusting the interface layout in adaptation to inferred user stopping tendencies in addition to user interaction and screen size. Third, as a specific example of applying our proposed interface optimization framework in a larger scale real world application, we propose a Bayesian framework for user preference modeling and dynamically optimizing a faceted browsing system based on users' facet selection interactions. Finally, we propose a general formal framework for evaluating IR systems based on search session simulation that can be used to perform reproducible experiments for evaluating any IR system, including interactive systems and systems with sophisticated interfaces. We show that the traditional Cranfield evaluation method can be regarded as a special instantiation of the proposed framework where the simulated search session is a user sequentially browsing the presented search results. We further show that the proposed framework enables us to evaluate a set of tag-based search interfaces, a generalization of faceted browsing interfaces, producing results consistent with real user experiments and revealing interesting findings about effectiveness of the interfaces for different types of users.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2017

Author dc:creator

• Zhang, Yinan

Contributors dc:contributor

• Zhai, ChengXiang
• Han, Jiawei
• Parameswaran, Aditya
• Fuhr, Norbert

• Information retrieval
• Probability ranking principle
• Interface card model

Statement dc:rights

• Copyright 2017 Yinan Zhang

Language dc:language

en

Handle dc:identifier

http://hdl.handle.net/2142/98262

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/98262