Utilization of Natural Language Processing for Extracting Smart Cities Requirements from Large Social Media Text

Major organizations such as urban centers worldwide face challenges from rapid population growth and evolving demands, requiring innovative approaches to stay responsive to residents' needs. This challenge is exemplified by the city of Calgary, where an automated system for aggregating and categorizing resident feedback could improve city planning. What people find important and useful can be seen in the articles they post on social media. One method for determining the performance of urban services and assets for citizens is paying attention to these data generated by the residents. In this regard, we need to examine datasets wherein writing is the primary form of citizen engagement (direct messages, requests, comments, complaints, etc.). To interpret this data, it is necessary to use appropriate tools and techniques for data processing and analysis of large volumes of unstructured text. Some of the most effective tools used by researchers nowadays falls into the scope of computational linguistics, specifically Natural language processing (NLP). Furthermore, Twitter is one of the primary platforms where individuals freely voice their opinions and concerns. In this study, we develop an automated workflow that can scrape, classify, and display tweets in a simplistic view. With the help of this system, local officials will be able to speed up the decision-making process when considering citizens' current problems. Following our research question, we look into the optimal scraping criteria, explore a variety of methods for topic and emotions analysis, and validate these methods both using automatic evaluation and manual assessment. As a result, we are able to identify issues related to city development, senior citizens, taxes, and unemployment using our best performing models (BERTopic for topic modeling and few-shot learning using Setfit for emotion analysis.) Afterward, we collect city employees' opinion regarding our research to determine the usefulness and applicability of this approach. Overall, we demonstrate how delving into these analyses can complement the current systems in place for urban planning.

Name thesis:degree_name

Master of Science (MSc)

Discipline thesis:degree_discipline

Engineering – Electrical & Computer

Grantor dc:publisher.institution

Graduate Studies

Year dc:date.issued

2024

Author dc:creator

• Mirshafiee Khoozani, Mitra Sadat

Advisors dc:contributor.advisor

• Barcomb, Ann
• Tan, Benjamin

Committee members dc:contributor.committeemember

• Messier, Geoffrey
• Fapojuwo, Abraham

Statement dc:rights

• University of Calgary graduate students retain copyright ownership and moral rights for their thesis. You may use this material in any way that is permitted by the Copyright Act or through licensing that has been assigned to the document. For uses that are not allowable under copyright legislation or licensing, you are required to seek permission.

Language dc:language.iso

en

DOI

https://doi.org/10.11575/PRISM/46444

OAI identifier oai:identifier

oai:ucalgary.scholaris.ca:1880/118847