{"id":{"repo_id":"vt","oai_identifier":"oai:vtechworks.lib.vt.edu:10919/118974"},"canonical_url":"https://search.dev.ndltd.org/etd/vt/oai:vtechworks.lib.vt.edu:10919/118974","repository":{"repo_id":"vt","name":"Virginia Tech","base_url":"https://vtechworks.lib.vt.edu/oai/request"},"display":{"title":"Utilizing Machine Learning Methods for Usability Evaluation in Learning Management Systems","abstract":"The concept of usability refers to a user's capability to interact with a system to fulfill goals in terms of task completion (effectiveness), time measurement (efficiency), and positive attitude (satisfaction). The strategy for usability evaluation in software systems usually involves questionnaires, user testing, and heuristics. Although these methods have been widely used due to several benefits, there are challenges related to time consumption and embedded bias. In response to these challenges, this work proposes a hybrid approach based on usability questionnaire answers and machine learning algorithms to predict usability scores. We describe three different experiments with features extracted from a Learning Management System. These features were applied in the Machine Learning algorithms Linear Regression, Decision Trees, Random Forest, and Neural Networks in three experiments. Random Forest produces the best performance of average mean square error and root mean square error among machine learning algorithms. The results are promising, though there are alternatives for improvements for better performance of the System Usability Scale and UseLearn scores prediction. This approach has potential as a reliable predictive tool for usability scores, which would help create software systems that better satisfy users' needs.","abstract_html":"The concept of usability refers to a user's capability to interact with a system to fulfill goals in terms of task completion (effectiveness), time measurement (efficiency), and positive attitude (satisfaction). The strategy for usability evaluation in software systems usually involves questionnaires, user testing, and heuristics. Although these methods have been widely used due to several benefits, there are challenges related to time consumption and embedded bias. In response to these challenges, this work proposes a hybrid approach based on usability questionnaire answers and machine learning algorithms to predict usability scores. We describe three different experiments with features extracted from a Learning Management System. These features were applied in the Machine Learning algorithms Linear Regression, Decision Trees, Random Forest, and Neural Networks in three experiments. Random Forest produces the best performance of average mean square error and root mean square error among machine learning algorithms. The results are promising, though there are alternatives for improvements for better performance of the System Usability Scale and UseLearn scores prediction. This approach has potential as a reliable predictive tool for usability scores, which would help create software systems that better satisfy users' needs.","abstract_has_math":false,"creators":["Torres Molina, Richard Andres"],"institution":"Virginia Tech","degree_name":"Master of Science","degree_level":"masters","degree_discipline":"Computer Science & Applications","degree_department":"Computer Science and Applications","school":null,"contributors":[],"advisors":[],"committee_chairs":["Seyam, Mohammed Saad Mohamed Elmahdy"],"committee_members":["Fox, Edward A.","McCrickard, Donald Scott"],"year":2024,"date_issued":"2024-05-14","date_published":"2024-05-14","updated_at":"2026-07-22T22:19:44Z","subjects":["usability evaluation","machine learning","learning management systems"],"languages":["en"],"rights":["In Copyright"],"rights_urls":["http://rightsstatements.org/vocab/InC/1.0/"],"identifier_entries":[{"key":"dc:identifier.other","label":"Dc Identifier Other","values":["vt_gsexam:40067"],"render_values":[{"text":"vt_gsexam:40067","href":null,"code":true}]}]},"links":{"outbound_url":"https://hdl.handle.net/10919/118974","outbound_label":"Handle","outbound_source":"dc:identifier.uri"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor.committeechair","label":"Committee Chair","values":["Seyam, Mohammed Saad Mohamed Elmahdy"]},{"key":"dc:contributor.committeemember","label":"Committee Member","values":["Fox, Edward A.","McCrickard, Donald Scott"]},{"key":"dc:contributor.department","label":"Department","values":["Computer Science and Applications"]},{"key":"dc:creator","label":"Author","values":["Torres Molina, Richard Andres"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.accessioned","label":"Dc Date Accessioned","values":["2024-05-15T08:00:34Z"]},{"key":"dc:date.available","label":"Dc Date Available","values":["2024-05-15T08:00:34Z"]},{"key":"dc:date.issued","label":"Date","values":["2024-05-14"]},{"key":"dc:publisher","label":"Institution","values":["Virginia Tech"]},{"key":"dc:type","label":"Dc Type","values":["Thesis"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Computer Science & Applications"]},{"key":"thesis:degree_level","label":"Degree Level","values":["masters"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Master of Science"]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["Virginia Polytechnic Institute and State University"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["usability evaluation","machine learning","learning management systems"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language.iso","label":"Language (ISO)","values":["en"]},{"key":"dc:rights","label":"Dc Rights","values":["In Copyright"]},{"key":"dc:rights.uri","label":"Rights URI","values":["http://rightsstatements.org/vocab/InC/1.0/"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier.other","label":"Dc Identifier Other","values":["vt_gsexam:40067"]},{"key":"dc:identifier.uri","label":"Identifier URI","values":["https://hdl.handle.net/10919/118974"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["The concept of usability refers to a user's capability to interact with a system to fulfill goals in terms of task completion (effectiveness), time measurement (efficiency), and positive attitude (satisfaction). The strategy for usability evaluation in software systems usually involves questionnaires, user testing, and heuristics. Although these methods have been widely used due to several benefits, there are challenges related to time consumption and embedded bias. In response to these challenges, this work proposes a hybrid approach based on usability questionnaire answers and machine learning algorithms to predict usability scores. We describe three different experiments with features extracted from a Learning Management System. These features were applied in the Machine Learning algorithms Linear Regression, Decision Trees, Random Forest, and Neural Networks in three experiments. Random Forest produces the best performance of average mean square error and root mean square error among machine learning algorithms. The results are promising, though there are alternatives for improvements for better performance of the System Usability Scale and UseLearn scores prediction. This approach has potential as a reliable predictive tool for usability scores, which would help create software systems that better satisfy users' needs."]},{"key":"dc:description.abstractgeneral","label":"General Abstract","values":["Instructors and students have used online platforms known as Learning Management Systems (LMSs) to improve learning and satisfaction. Students need to achieve their learning goals by interacting with these systems. To achieve these goals, usability evaluation involves ensuring that LMSs attain effectiveness (task completion), efficiency (time measurement), and satisfaction (positive attitude). Usability evaluation usually follows questionnaires, user testing of the LMS, and expert reviews. Although these methods are widely used due to several benefits, they face challenges related to trying these software systems multiple times until the system satisfies student needs and human subjectivity perception. To face these challenges, promote student engagement with the system, and create a better design in the LMS courses, we propose a hybrid approach based on data, questionnaire answers, and machine learning algorithms to predict usability scores. We evaluated this approach through a case study with data collected from undergraduate students at Virginia Tech. The results showed different advantages and drawbacks of machine learning performance. The approach contributes to the engineering and computing education field by providing a reliable predictive tool for usability scores to improve the student learning experience and the features of the LMS."]},{"key":"dc:description.degree","label":"Dc Description Degree","values":["Master of Science"]},{"key":"dc:format.medium","label":"Dc Format Medium","values":["ETD"]},{"key":"dc:title","label":"Title","values":["Utilizing Machine Learning Methods for Usability Evaluation in Learning Management Systems"]}]}],"canonical_facts":{"dc:contributor.committeechair":["Seyam, Mohammed Saad Mohamed Elmahdy"],"dc:contributor.committeemember":["Fox, Edward A.","McCrickard, Donald Scott"],"dc:contributor.department":["Computer Science and Applications"],"dc:creator":["Torres Molina, Richard Andres"],"dc:date.accessioned":["2024-05-15T08:00:34Z"],"dc:date.available":["2024-05-15T08:00:34Z"],"dc:date.issued":["2024-05-14"],"dc:description.abstract":["The concept of usability refers to a user's capability to interact with a system to fulfill goals in terms of task completion (effectiveness), time measurement (efficiency), and positive attitude (satisfaction). The strategy for usability evaluation in software systems usually involves questionnaires, user testing, and heuristics. Although these methods have been widely used due to several benefits, there are challenges related to time consumption and embedded bias. In response to these challenges, this work proposes a hybrid approach based on usability questionnaire answers and machine learning algorithms to predict usability scores. We describe three different experiments with features extracted from a Learning Management System. These features were applied in the Machine Learning algorithms Linear Regression, Decision Trees, Random Forest, and Neural Networks in three experiments. Random Forest produces the best performance of average mean square error and root mean square error among machine learning algorithms. The results are promising, though there are alternatives for improvements for better performance of the System Usability Scale and UseLearn scores prediction. This approach has potential as a reliable predictive tool for usability scores, which would help create software systems that better satisfy users' needs."],"dc:description.abstractgeneral":["Instructors and students have used online platforms known as Learning Management Systems (LMSs) to improve learning and satisfaction. Students need to achieve their learning goals by interacting with these systems. To achieve these goals, usability evaluation involves ensuring that LMSs attain effectiveness (task completion), efficiency (time measurement), and satisfaction (positive attitude). Usability evaluation usually follows questionnaires, user testing of the LMS, and expert reviews. Although these methods are widely used due to several benefits, they face challenges related to trying these software systems multiple times until the system satisfies student needs and human subjectivity perception. To face these challenges, promote student engagement with the system, and create a better design in the LMS courses, we propose a hybrid approach based on data, questionnaire answers, and machine learning algorithms to predict usability scores. We evaluated this approach through a case study with data collected from undergraduate students at Virginia Tech. The results showed different advantages and drawbacks of machine learning performance. The approach contributes to the engineering and computing education field by providing a reliable predictive tool for usability scores to improve the student learning experience and the features of the LMS."],"dc:description.degree":["Master of Science"],"dc:format.medium":["ETD"],"dc:identifier.other":["vt_gsexam:40067"],"dc:identifier.uri":["https://hdl.handle.net/10919/118974"],"dc:language.iso":["en"],"dc:publisher":["Virginia Tech"],"dc:rights":["In Copyright"],"dc:rights.uri":["http://rightsstatements.org/vocab/InC/1.0/"],"dc:subject":["usability evaluation","machine learning","learning management systems"],"dc:title":["Utilizing Machine Learning Methods for Usability Evaluation in Learning Management Systems"],"dc:type":["Thesis"],"thesis:degree_discipline":["Computer Science & Applications"],"thesis:degree_level":["masters"],"thesis:degree_name":["Master of Science"],"thesis:institution_name":["Virginia Polytechnic Institute and State University"]},"updated_at":"2026-07-22T22:19:44Z"}