Exploring the impact of conversational strategies on user search experience in goal-oriented tasks in a voice-only domain

Conversational search has recently become established as a new information search paradigm. Increasing numbers of people are using smart speakers and virtual personal assistants to access information online. Given the substantial increase in the usage of devices that feature conversational agents, it is timely to explore the implications that they have on user search behaviour.;This thesis explores how conversational search agents impact on user search experience and outcomes for goal-oriented tasks. Our enquiry is inspired by the premise that a fully conversational agent equipped with memory could lead to a better user search experience than an agent that is based on a slot-filling paradigm which represents the current state of the art. This dissertation presents three Wizard of Oz, lab-based studies that explore a series of conversational agents that differ in their ability to preserve conversational state and employ a different degree of conversational initiative. The goal of this research is to get a better understanding of how different ways of presenting information impact on user search experience in terms of cognitive load, satisfaction with the agent, time required to complete the task and overall performance.;In the first study, we address the problem of an agent's statefulness, which is the ability to preserve and account for the information presented by the user without the need to repeat it. Specifically, through a series of interactive search tasks, we evaluate how agent's memory improves user search experience. In the second study, we focus on mixed-initiative in conversation, by evaluating how different ways of eliciting information and presenting it back to the user (revealment) impact upon user search experience. In the third study, we further explore the role of mixed initiative by evaluating agent's 'proactivity' (i.e. the search involvement that goes beyond the original scope of the query) on user search experience. Finally, we compare the insights from all of the three studies and assess how differences in search task design and the mode of delivering the studies (lab-based vs. deployed online) impact upon experimental outcomes.;Overall, the contribution of the research presented in the current thesis is threefold. Firstly, it provides a validation of theoretical frameworks of conversational search by empirically investigating the impact of agent's memory and mixed initiative on user search experience. Secondly, it provides insights into the efficacy of alternative ways of eliciting information and presenting search results during conversational search. Thirdly, it contributes towards a more robust evaluation of conversational agents by developing a spectrum of conversational strategies and reflecting on the impact of the design of goal-oriented search tasks.

Name dc:type.qualificationname

phd

Level dc:type.qualificationlevel

doctoral-pg

Grantor dc:publisher.institution

University of Strathclyde

Year dc:date.issued

2020

Author dc:creator

• Dubiel, Mateusz

Advisors dc:contributor.advisor

• Halvey, Martin
• Azzopardi, Leif

DOI

10.48730/vgfp-ah87

Identifier

T15835

Author Identifier

201660549

OAI identifier oai:identifier

oai:strathclyde:rx913p93b