{"id":{"repo_id":"milano","oai_identifier":"oai:air.unimi.it:2434/1157335"},"canonical_url":"https://search.dev.ndltd.org/etd/milano/oai:air.unimi.it:2434/1157335","repository":{"repo_id":"milano","name":"Università degli Studi di Milano","base_url":"https://air.unimi.it/oai/request"},"display":{"title":"INTERFACE EVALUATION AND OPERATOR PERFORMANCE PREDICTION, APPLIED TO A TELEROBOTIC CASE-STUDY.","abstract":"With the increased use of automation in Industry, the role of the operator is shifting to a monitoring and supervising role. The design of the Human-Machine Interfaces has a critical impact on operator situational awareness and ability to manually control the system when facing automation failures. The current work investigates how human performance assessment methods can be leveraged to support the design of more effective, user-centered and intelligent Human-Machine Interfaces and Interactions (HMIIs) in Industry, that can keep the human in the loop. This investigation is guided by a specific case-study, the use of a telerobotic system for a medical device manufacturing application. Teleoperation has gained traction, particularly for industrial use, with the fast development of new interaction modalities and extended reality technology. However, it is still challenging for non-expert users, as human performance is hindered by the lack of feedback, limited awareness of the remote environment and the control complexity. Literature on teleoperation interface design has focused mostly on solving technological challenges and typically employs subjective operator measures for its development. As a result, two main research questions are established in this thesis: 1) identify what human performance measures are better suited to support the design and evaluation of telerobotic interfaces, 2) assess whether human performance can be reliably predicted with ML, to potentially inform the system and adapt the human-machine interface to the operator needs. To address the former, a telerobotics user study was conducted, where 27 subjects performed a teleoperated cutting task, while performance, physiological and behavioral data was collected. Four telerobotic interfaces with different control and visual designs were developed and evaluated, aiming to advance our understanding of how the interface impacts the operator and how that impact can be assessed. To address the latter, traditional machine learning methods were employed for human performance prediction on the collected dataset. In alternative, the author also explores the use of deep learning techniques for the recognition of performance-related states, such as cognitive workload and task engagement, from raw multimodal physiological signals. Taking into consideration the industrial focus, the work provides in addition insights into the safety, ethical and regulatory requirements and challenges of applying these systems in Industry, as a guideline for the future research needs.","abstract_html":"With the increased use of automation in Industry, the role of the operator is shifting to a monitoring and supervising role. The design of the Human-Machine Interfaces has a critical impact on operator situational awareness and ability to manually control the system when facing automation failures. The current work investigates how human performance assessment methods can be leveraged to support the design of more effective, user-centered and intelligent Human-Machine Interfaces and Interactions (HMIIs) in Industry, that can keep the human in the loop. This investigation is guided by a specific case-study, the use of a telerobotic system for a medical device manufacturing application. Teleoperation has gained traction, particularly for industrial use, with the fast development of new interaction modalities and extended reality technology. However, it is still challenging for non-expert users, as human performance is hindered by the lack of feedback, limited awareness of the remote environment and the control complexity. Literature on teleoperation interface design has focused mostly on solving technological challenges and typically employs subjective operator measures for its development. As a result, two main research questions are established in this thesis: 1) identify what human performance measures are better suited to support the design and evaluation of telerobotic interfaces, 2) assess whether human performance can be reliably predicted with ML, to potentially inform the system and adapt the human-machine interface to the operator needs. To address the former, a telerobotics user study was conducted, where 27 subjects performed a teleoperated cutting task, while performance, physiological and behavioral data was collected. Four telerobotic interfaces with different control and visual designs were developed and evaluated, aiming to advance our understanding of how the interface impacts the operator and how that impact can be assessed. To address the latter, traditional machine learning methods were employed for human performance prediction on the collected dataset. In alternative, the author also explores the use of deep learning techniques for the recognition of performance-related states, such as cognitive workload and task engagement, from raw multimodal physiological signals. Taking into consideration the industrial focus, the work provides in addition insights into the safety, ethical and regulatory requirements and challenges of applying these systems in Industry, as a guideline for the future research needs.","abstract_has_math":false,"creators":["FERNANDES RAMOS, INES FILIPA"],"institution":"Università degli Studi di Milano","degree_name":null,"degree_level":null,"degree_discipline":null,"degree_department":null,"school":null,"contributors":["supervisor: G. Gianini ; co-supervisor: E. Damiani ; coordinator: R. Sassi","I.F. Fernandes Ramos","SASSI, ROBERTO"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2025,"date_issued":"2025-04-14","date_published":"2025-04-14","updated_at":"2026-07-27T20:18:58Z","subjects":["Human-Machine Interface","Human-Machine Interaction","Human Performance","Telerobotic","Machine Learning Model","Industry 5.0","Settore INFO-01/A - Informatica"],"languages":["eng"],"rights":["info:eu-repo/semantics/openAccess","license:Creative commons","license uri:http://creativecommons.org/licenses/by-sa/4.0/"],"rights_urls":[],"identifier_entries":[{"key":"dc:identifier","label":"Identifier","values":["http://dx.doi.org/10.13130/fernandes-ramos-ines-filipa_phd2025-04-14","10.13130/fernandes-ramos-ines-filipa_phd2025-04-14"],"render_values":[{"text":"http://dx.doi.org/10.13130/fernandes-ramos-ines-filipa_phd2025-04-14","href":"http://dx.doi.org/10.13130/fernandes-ramos-ines-filipa_phd2025-04-14","code":true},{"text":"10.13130/fernandes-ramos-ines-filipa_phd2025-04-14","href":"https://doi.org/10.13130/fernandes-ramos-ines-filipa_phd2025-04-14","code":true}]}]},"links":{"outbound_url":"https://hdl.handle.net/2434/1157335","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["supervisor: G. Gianini ; co-supervisor: E. Damiani ; coordinator: R. Sassi","I.F. Fernandes Ramos","SASSI, ROBERTO"]},{"key":"dc:creator","label":"Author","values":["FERNANDES RAMOS, INES FILIPA"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2025-04-14"]},{"key":"dc:publisher","label":"Institution","values":["Università degli Studi di Milano"]},{"key":"dc:relation","label":"Dc Relation","values":["numberofpages:272"]},{"key":"dc:type","label":"Dc Type","values":["info:eu-repo/semantics/doctoralThesis"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Human-Machine Interface","Human-Machine Interaction","Human Performance","Telerobotic","Machine Learning Model","Industry 5.0","Settore INFO-01/A - Informatica"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["eng"]},{"key":"dc:rights","label":"Dc Rights","values":["info:eu-repo/semantics/openAccess","license:Creative commons","license uri:http://creativecommons.org/licenses/by-sa/4.0/"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://hdl.handle.net/2434/1157335","http://dx.doi.org/10.13130/fernandes-ramos-ines-filipa_phd2025-04-14","10.13130/fernandes-ramos-ines-filipa_phd2025-04-14"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["With the increased use of automation in Industry, the role of the operator is shifting to a monitoring and supervising role. The design of the Human-Machine Interfaces has a critical impact on operator situational awareness and ability to manually control the system when facing automation failures. The current work investigates how human performance assessment methods can be leveraged to support the design of more effective, user-centered and intelligent Human-Machine Interfaces and Interactions (HMIIs) in Industry, that can keep the human in the loop. This investigation is guided by a specific case-study, the use of a telerobotic system for a medical device manufacturing application. Teleoperation has gained traction, particularly for industrial use, with the fast development of new interaction modalities and extended reality technology. However, it is still challenging for non-expert users, as human performance is hindered by the lack of feedback, limited awareness of the remote environment and the control complexity. Literature on teleoperation interface design has focused mostly on solving technological challenges and typically employs subjective operator measures for its development. As a result, two main research questions are established in this thesis: 1) identify what human performance measures are better suited to support the design and evaluation of telerobotic interfaces, 2) assess whether human performance can be reliably predicted with ML, to potentially inform the system and adapt the human-machine interface to the operator needs. To address the former, a telerobotics user study was conducted, where 27 subjects performed a teleoperated cutting task, while performance, physiological and behavioral data was collected. Four telerobotic interfaces with different control and visual designs were developed and evaluated, aiming to advance our understanding of how the interface impacts the operator and how that impact can be assessed. To address the latter, traditional machine learning methods were employed for human performance prediction on the collected dataset. In alternative, the author also explores the use of deep learning techniques for the recognition of performance-related states, such as cognitive workload and task engagement, from raw multimodal physiological signals. Taking into consideration the industrial focus, the work provides in addition insights into the safety, ethical and regulatory requirements and challenges of applying these systems in Industry, as a guideline for the future research needs."]},{"key":"dc:title","label":"Title","values":["INTERFACE EVALUATION AND OPERATOR PERFORMANCE PREDICTION, APPLIED TO A TELEROBOTIC CASE-STUDY."]}]}],"canonical_facts":{"dc:contributor":["supervisor: G. Gianini ; co-supervisor: E. Damiani ; coordinator: R. Sassi","I.F. Fernandes Ramos","SASSI, ROBERTO"],"dc:creator":["FERNANDES RAMOS, INES FILIPA"],"dc:date":["2025-04-14"],"dc:description":["With the increased use of automation in Industry, the role of the operator is shifting to a monitoring and supervising role. The design of the Human-Machine Interfaces has a critical impact on operator situational awareness and ability to manually control the system when facing automation failures. The current work investigates how human performance assessment methods can be leveraged to support the design of more effective, user-centered and intelligent Human-Machine Interfaces and Interactions (HMIIs) in Industry, that can keep the human in the loop. This investigation is guided by a specific case-study, the use of a telerobotic system for a medical device manufacturing application. Teleoperation has gained traction, particularly for industrial use, with the fast development of new interaction modalities and extended reality technology. However, it is still challenging for non-expert users, as human performance is hindered by the lack of feedback, limited awareness of the remote environment and the control complexity. Literature on teleoperation interface design has focused mostly on solving technological challenges and typically employs subjective operator measures for its development. As a result, two main research questions are established in this thesis: 1) identify what human performance measures are better suited to support the design and evaluation of telerobotic interfaces, 2) assess whether human performance can be reliably predicted with ML, to potentially inform the system and adapt the human-machine interface to the operator needs. To address the former, a telerobotics user study was conducted, where 27 subjects performed a teleoperated cutting task, while performance, physiological and behavioral data was collected. Four telerobotic interfaces with different control and visual designs were developed and evaluated, aiming to advance our understanding of how the interface impacts the operator and how that impact can be assessed. To address the latter, traditional machine learning methods were employed for human performance prediction on the collected dataset. In alternative, the author also explores the use of deep learning techniques for the recognition of performance-related states, such as cognitive workload and task engagement, from raw multimodal physiological signals. Taking into consideration the industrial focus, the work provides in addition insights into the safety, ethical and regulatory requirements and challenges of applying these systems in Industry, as a guideline for the future research needs."],"dc:identifier":["https://hdl.handle.net/2434/1157335","http://dx.doi.org/10.13130/fernandes-ramos-ines-filipa_phd2025-04-14","10.13130/fernandes-ramos-ines-filipa_phd2025-04-14"],"dc:language":["eng"],"dc:publisher":["Università degli Studi di Milano"],"dc:relation":["numberofpages:272"],"dc:rights":["info:eu-repo/semantics/openAccess","license:Creative commons","license uri:http://creativecommons.org/licenses/by-sa/4.0/"],"dc:subject":["Human-Machine Interface","Human-Machine Interaction","Human Performance","Telerobotic","Machine Learning Model","Industry 5.0","Settore INFO-01/A - Informatica"],"dc:title":["INTERFACE EVALUATION AND OPERATOR PERFORMANCE PREDICTION, APPLIED TO A TELEROBOTIC CASE-STUDY."],"dc:type":["info:eu-repo/semantics/doctoralThesis"]},"updated_at":"2026-07-27T20:18:58Z"}