{"id":{"repo_id":"cambridge","oai_identifier":"oai:www.repository.cam.ac.uk:1810/382251"},"canonical_url":"https://search.dev.ndltd.org/etd/cambridge/oai:www.repository.cam.ac.uk:1810/382251","repository":{"repo_id":"cambridge","name":"Cambridge University","base_url":"https://api.repository.cam.ac.uk/server/oai/request"},"display":{"title":"Multimodal Interaction for Text Entry and Control in Mixed Reality","abstract":"This thesis explores the integration of multimodal interaction methods in Mixed Reality (MR) environments for text entry control. The central hypothesis is that these multimodal approaches can provide an expressive and efficient solution for (1) text input; (2) text editing; and (3) command retrieval, complementing the limitations of traditional input methods in immersive environments. The research addresses four key areas: text caret control, dwell-free eye typing, command search systems through dwell-free eye typing, and eye-hand coordination in mid-air gesture typing. Through iterative prototyping and controlled empirical studies, the research demonstrates that a multimodal interaction method, combining gaze for coarse positioning and hand gestures for fine control, provides significant advantages in terms of performance and reducing physical strain. Moreover, the introduction of dwell-free eye typing methods greatly enhances typing speed without sacrificing accuracy, offering an efficient solution for hands-free input in MR environments. Additionally, this dwell-free method was later utilized for a command search system which outperform traditional menu-based approaches, improving both usability and task efficiency. The thesis also investigates the challenges of eye-hand coordination in mid-air gesture typing, revealing that the absence of tactile feedback necessitates greater visual attention and introduces a tighter synchronization between eye and hand movements. Despite these challenges, participants adapted to achieve stable typing performance after practice. These findings contribute to the design of more user-friendly and effective MR interfaces, offering practical insights for improving the user experience in multimodal MR applications. The thesis concludes by discussing limitations and proposing avenues for future research.","abstract_html":"This thesis explores the integration of multimodal interaction methods in Mixed Reality (MR) environments for text entry control. The central hypothesis is that these multimodal approaches can provide an expressive and efficient solution for (1) text input; (2) text editing; and (3) command retrieval, complementing the limitations of traditional input methods in immersive environments. The research addresses four key areas: text caret control, dwell-free eye typing, command search systems through dwell-free eye typing, and eye-hand coordination in mid-air gesture typing. Through iterative prototyping and controlled empirical studies, the research demonstrates that a multimodal interaction method, combining gaze for coarse positioning and hand gestures for fine control, provides significant advantages in terms of performance and reducing physical strain. Moreover, the introduction of dwell-free eye typing methods greatly enhances typing speed without sacrificing accuracy, offering an efficient solution for hands-free input in MR environments. Additionally, this dwell-free method was later utilized for a command search system which outperform traditional menu-based approaches, improving both usability and task efficiency. The thesis also investigates the challenges of eye-hand coordination in mid-air gesture typing, revealing that the absence of tactile feedback necessitates greater visual attention and introduces a tighter synchronization between eye and hand movements. Despite these challenges, participants adapted to achieve stable typing performance after practice. These findings contribute to the design of more user-friendly and effective MR interfaces, offering practical insights for improving the user experience in multimodal MR applications. The thesis concludes by discussing limitations and proposing avenues for future research.","abstract_has_math":false,"creators":["Hu, Jinghui"],"institution":"University of Cambridge","degree_name":"Doctor of Philosophy (PhD)","degree_level":"Doctoral","degree_discipline":null,"degree_department":null,"school":null,"contributors":[],"advisors":["Kristensson, Per Ola"],"committee_chairs":[],"committee_members":[],"year":2024,"date_issued":"2024-09-30","date_published":"2024-09-30","updated_at":"2026-07-22T22:24:03Z","subjects":["Human computer interaction","Mixed Reality","Text Entry","Multimodality interaction","Augmented Reality","Gesture Typing","Eye Typing","Eye hand coordination"],"languages":["eng"],"rights":[],"rights_urls":["https://apollo8-f-pro.lib.cam.ac.uk/bitstreams/005bb2d1-d894-40a7-8c50-46816f64e758/download","http://purl.org/NET/rdflicense/allrightsreserved"],"identifier_entries":[{"key":"dc:creator.authoridentifier","label":"Author Identifier","values":["0000000239652474"],"render_values":[{"text":"0000-0002-3965-2474","href":"https://orcid.org/0000-0002-3965-2474","code":true}]}]},"links":{"outbound_url":"https://doi.org/10.17863/CAM.117147","outbound_label":"DOI","outbound_source":"dc:identifier.doi"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor.advisor","label":"Advisor","values":["Kristensson, Per Ola"]},{"key":"dc:creator","label":"Author","values":["Hu, Jinghui"]},{"key":"dc:creator.authoridentifier","label":"Author Identifier","values":["0000000239652474"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.issued","label":"Date","values":["2024-09-30"]},{"key":"dc:publisher.institution","label":"Dc Publisher Institution","values":["University of Cambridge"]},{"key":"dc:relation.isreferencedby.uri","label":"Dc Relation Isreferencedby URI","values":["https://www.repository.cam.ac.uk/handle/1810/382251"]},{"key":"dc:type","label":"Dc Type","values":["Thesis"]},{"key":"dc:type.qualificationlevel","label":"Dc Type Qualificationlevel","values":["Doctoral"]},{"key":"dc:type.qualificationname","label":"Dc Type Qualificationname","values":["Doctor of Philosophy (PhD)"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Human computer interaction","Mixed Reality","Text Entry","Multimodality interaction","Augmented Reality","Gesture Typing","Eye Typing","Eye hand coordination"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["eng"]},{"key":"dc:rights","label":"Dc Rights","values":["https://apollo8-f-pro.lib.cam.ac.uk/bitstreams/005bb2d1-d894-40a7-8c50-46816f64e758/download","http://purl.org/NET/rdflicense/allrightsreserved"]},{"key":"dc:rights.embargodate","label":"Dc Rights Embargodate","values":["2026-04-04"]},{"key":"dc:rights.embargotype","label":"Dc Rights Embargotype","values":["embargo"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier.doi","label":"DOI","values":["https://doi.org/10.17863/CAM.117147"]},{"key":"dc:identifier.uri","label":"Identifier URI","values":["https://apollo8-f-pro.lib.cam.ac.uk/bitstreams/aaf20bc5-a488-4ffa-aabb-bbe3464c9aa4/download"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["This thesis explores the integration of multimodal interaction methods in Mixed Reality (MR) environments for text entry control. The central hypothesis is that these multimodal approaches can provide an expressive and efficient solution for (1) text input; (2) text editing; and (3) command retrieval, complementing the limitations of traditional input methods in immersive environments. The research addresses four key areas: text caret control, dwell-free eye typing, command search systems through dwell-free eye typing, and eye-hand coordination in mid-air gesture typing. Through iterative prototyping and controlled empirical studies, the research demonstrates that a multimodal interaction method, combining gaze for coarse positioning and hand gestures for fine control, provides significant advantages in terms of performance and reducing physical strain. Moreover, the introduction of dwell-free eye typing methods greatly enhances typing speed without sacrificing accuracy, offering an efficient solution for hands-free input in MR environments. Additionally, this dwell-free method was later utilized for a command search system which outperform traditional menu-based approaches, improving both usability and task efficiency. The thesis also investigates the challenges of eye-hand coordination in mid-air gesture typing, revealing that the absence of tactile feedback necessitates greater visual attention and introduces a tighter synchronization between eye and hand movements. Despite these challenges, participants adapted to achieve stable typing performance after practice. These findings contribute to the design of more user-friendly and effective MR interfaces, offering practical insights for improving the user experience in multimodal MR applications. The thesis concludes by discussing limitations and proposing avenues for future research."]},{"key":"dc:format.checksum.md5","label":"Dc Format Checksum Md5","values":["1ebe94479783bf77e3764d1e3a5fb97f","87eda9de84448d1f82354d60eee3eb5f"]},{"key":"dc:title","label":"Title","values":["Multimodal Interaction for Text Entry and Control in Mixed Reality"]}]}],"canonical_facts":{"dc:contributor.advisor":["Kristensson, Per Ola"],"dc:creator":["Hu, Jinghui"],"dc:creator.authoridentifier":["0000000239652474"],"dc:date.issued":["2024-09-30"],"dc:description.abstract":["This thesis explores the integration of multimodal interaction methods in Mixed Reality (MR) environments for text entry control. The central hypothesis is that these multimodal approaches can provide an expressive and efficient solution for (1) text input; (2) text editing; and (3) command retrieval, complementing the limitations of traditional input methods in immersive environments. The research addresses four key areas: text caret control, dwell-free eye typing, command search systems through dwell-free eye typing, and eye-hand coordination in mid-air gesture typing. Through iterative prototyping and controlled empirical studies, the research demonstrates that a multimodal interaction method, combining gaze for coarse positioning and hand gestures for fine control, provides significant advantages in terms of performance and reducing physical strain. Moreover, the introduction of dwell-free eye typing methods greatly enhances typing speed without sacrificing accuracy, offering an efficient solution for hands-free input in MR environments. Additionally, this dwell-free method was later utilized for a command search system which outperform traditional menu-based approaches, improving both usability and task efficiency. The thesis also investigates the challenges of eye-hand coordination in mid-air gesture typing, revealing that the absence of tactile feedback necessitates greater visual attention and introduces a tighter synchronization between eye and hand movements. Despite these challenges, participants adapted to achieve stable typing performance after practice. These findings contribute to the design of more user-friendly and effective MR interfaces, offering practical insights for improving the user experience in multimodal MR applications. The thesis concludes by discussing limitations and proposing avenues for future research."],"dc:format.checksum.md5":["1ebe94479783bf77e3764d1e3a5fb97f","87eda9de84448d1f82354d60eee3eb5f"],"dc:identifier.doi":["https://doi.org/10.17863/CAM.117147"],"dc:identifier.uri":["https://apollo8-f-pro.lib.cam.ac.uk/bitstreams/aaf20bc5-a488-4ffa-aabb-bbe3464c9aa4/download"],"dc:language":["eng"],"dc:publisher.institution":["University of Cambridge"],"dc:relation.isreferencedby.uri":["https://www.repository.cam.ac.uk/handle/1810/382251"],"dc:rights":["https://apollo8-f-pro.lib.cam.ac.uk/bitstreams/005bb2d1-d894-40a7-8c50-46816f64e758/download","http://purl.org/NET/rdflicense/allrightsreserved"],"dc:rights.embargodate":["2026-04-04"],"dc:rights.embargotype":["embargo"],"dc:subject":["Human computer interaction","Mixed Reality","Text Entry","Multimodality interaction","Augmented Reality","Gesture Typing","Eye Typing","Eye hand coordination"],"dc:title":["Multimodal Interaction for Text Entry and Control in Mixed Reality"],"dc:type":["Thesis"],"dc:type.qualificationlevel":["Doctoral"],"dc:type.qualificationname":["Doctor of Philosophy (PhD)"]},"updated_at":"2026-07-22T22:24:03Z"}