Sentiment Analysis with Language Models on Finnish Workplace Well-Being Surveys

Improving employee well-being is a key part of pension agency Keva’s mission statement. Recently, Keva has launched a tool for conducting repeated small-scale employee well-being surveys called ”Pulssi”. With the number of responses reaching thousands Keva has identified processing and organizing this data as a part of this process that could be improved using machine learning methods. In this thesis, we conducted a comprehensive investigation into using language models and sentiment classifications as a solution. We tested three different methodologies for this purpose, traditional machine learning with learned embeddings, generative language methods, and fine-tuned BERT models. To our knowledge, this is the first study evaluating the use of language models on the Finnish sentiment analysis task. Additionally, we evaluated the feasibility of implementing these methods based on their operating costs and the time it took to create classifications. We found that the traditional machine learning trained on learned embeddings performed surprisingly well, achieving an accuracy of 91%. These models offer a fast and cost-effective alternative to the more cumbersome language models. Our fine-tuned BERT model the ”KevaBERT” achieved an impressive accuracy of 93.6%, when trained on GPT-4 generated predictions, suggesting a potential pathway for training data creation. Overall our best performance was achieved by the ”GPT-4 few-shot with context” model at 93.9% accuracy. Our accuracies rival or even surpass the state-of-the-art accuracies achieved on other datasets. Despite the near human-level performance, this model was slow and expensive to operate. Based on these findings we recommend the use of our ”KevaBERT” model for sentiment classifications and a separate GPT-4 based model for text summarization.

Grantor dc:publisher

Helsingin yliopisto

Year dc:date.issued

2024

Author dc:creator

• Kortesalmi, Ville

Contributors dc:contributor

• Helsingin yliopisto, Matemaattis-luonnontieteellinen tiedekunta
• University of Helsinki, Faculty of Science
• Helsingfors universitet, Matematisk-naturvetenskapliga fakulteten

• Sentiment analysis
• Natural language processing
• Language models
• Survey data

Language dc:language.iso

eng

Identifier URI

URN:NBN:fi:hulib-202406253322

OAI identifier oai:identifier

oai:helda.helsinki.fi:10138/577817