Characterising user engagement of parents with the Aurora chatbot

Chatbots have the potential to enhance everyday life by providing information, customer support, personal assistance, and more across various sectors, making them versatile tools for modern living. In childcare, they have the potential to provide parents with relevant and acceptable childcare information. This potential arises from the growing disparity between the abundance of childcare information available and parents' ability to access information tailored to their specific needs [5]. Aurora, a rule-based Facebook Messenger chatbot, was developed to support parents and caregivers in caring for their children by providing accessible and comprehensible childcare information, targeted for Portuguese-speaking parents [6]. This research analysed the Aurora chatbot's chatlogs dating back to October 2018 up until September 2021 to scrutinise the interaction dynamics between Aurora and its users. Through this research, the objective is to delineate user engagement patterns, identify topics discussed, highlight topics outside the chatbot's knowledge domain, and assess the dynamics and quality of the conversations. The methodology used to achieve this objective encompassed text pre-processing, engagement metric extraction, topic analysis, content analysis, and sentiment analysis. The analysis of 1043 Aurora chatlogs indicated that only 718 (69%) users actively interacted with the system. These interactions predominantly occurred during lunchtime and late at night. The data showed that approximately 80% of conversations centred around baby sleep, 13% pertained to breastfeeding, and 7% focused on healthcare topics. While Aurora responded appropriately to in-domain questions, challenges arose when users' questions contained multiple topics, such as questions about the ability to breastfeed while taking certain medicines. User feedback was positive, with an average star rating of 4.37/5 (continuous scale), despite the modest sentiment score of 0.119 in the rating comments. The research classified users into four groups, based on paid and free subscriptions, each highlighting specific engagement patterns. Users who had the paid subscription showed a 243% increase in interactions and a 162% increase in extended use of the chatbot. These insights serve as guidance for Aurora's next iteration, highlighting the importance of recognising different user types and refining areas of shortfall. Additionally, this research contributes to expanding the scholarly corpus on how parents interact with chatbots.

Grantor dc:publisher.institution

Department of Computer Science

Year dc:date.issued

2024

Author dc:creator

• Liebetrau, Diana Rangel Lopes de Campos

Advisors dc:contributor.advisor

• Densmore, Melissa
• Nunes, Francisco

• Information Technology

Handle dc:identifier.uri

http://hdl.handle.net/11427/40836

OAI identifier oai:identifier

oai:open.uct.ac.za:11427/40836