{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/120164"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/120164","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Towards real-time robotics perception with continual adaptation","abstract":"Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2023-09-01 without embargo terms","abstract_html":"Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2023-09-01 without embargo terms","abstract_has_math":false,"creators":["Qiu, Rizhao"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"M.S.","degree_level":"Thesis","degree_discipline":"Computer Science","degree_department":null,"school":null,"contributors":["Hauser, Kris","Wang, Yuxiong"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2023,"date_issued":"2023-05","date_published":"2023-05","updated_at":"2026-07-22T22:24:56Z","subjects":["Robotics","Segmentation","Computer Vision","Artificial Intelligence"],"languages":["en","eng"],"rights":["Copyright 2023 Rizhao Qiu"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/2142/120164","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Hauser, Kris","Wang, Yuxiong"]},{"key":"dc:creator","label":"Author","values":["Qiu, Rizhao"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2023-05","2023-05-03"]},{"key":"dc:type","label":"Dc Type","values":["text"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Computer Science"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["M.S."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Robotics","Segmentation","Computer Vision","Artificial Intelligence"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en","eng"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2023 Rizhao Qiu"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://hdl.handle.net/2142/120164"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2023-09-01 without embargo terms","The student, Rizhao Qiu, accepted the attached license on 2023-05-02 at 15:39.","The student, Rizhao Qiu, submitted this Thesis for approval on 2023-05-02 at 15:46.","This Thesis was approved for publication on 2023-05-03 at 12:31.","DSpace SAF Submission Ingestion Package generated from Vireo submission #19301 on 2023-09-01 at 16:56:10","One of the key characteristics of intelligent beings is the ability to perceive and interact with the surrounding environment. Increasingly many learning-based methods have been proposed in attempts to equip robots with such perception power. However, most existing robotics perception algorithms are still inappropriate for practical robotics applications because they 1) do not consider computational efficiency and require a lot of computational resources or 2) assume a fixed and closed set of objects to recognize and hence fail when robots encounter previously unseen objects or repeat the same failure patterns. This thesis aims to address these issues between perception algorithms and robotics systems by developing robotics perception algorithms that are real-time for downstream tasks and can adapt during deployment time for open-world robotics. Specifically, the co-designing of 3D semantic mapping and a downstream planner and continual adaptation from few data for 2D semantic segmentation are discussed. The first work, RA-SLAM (RA-Simultaneous Localization And Mapping), designs a real-time GPU-based volumetric semantic mapping system that understands scenes geometrically and semantically. It operates at 50Hz on a consumer-grade GPU - approximately 5 times faster than CPU-based TSDF semantic reconstruction methods. Two studies are then presented for RA-SLAM: first, it can be combined with a 2D high-touch affordance segmentation model to improve disinfection trajectory planning with human-like semantic awareness. Second, different multi-class semantic representations in voxels are investigated, which leads to a variant of RA-SLAM for multi-class semantic reconstruction that balances computation-memory usage. The second work to be presented, GAPS (Guided copy-And-Paste Synthesis), enables robots to learn continually with few data via training data synthesis. GAPS dramatically boosts the novel IoU of baseline methods on established few-shot continual segmentation benchmarks by up to 80%, and maintains good performance in even more impoverished annotation settings, where only single instances of novel objects are annotated. To investigate its potential for perception in human-in-the-loop robotics application such as tele-operating robotics, a study is then carried out to simulate continual few-shot learning on real robots, which shows that existing algorithms fail dramatically when the limited training budget on robots are considered. The thesis then describes a system approach that balances training-inference resources and allows continual learning algorithms to run on the onboard computers of robots without hindering their perception capability."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Towards real-time robotics perception with continual adaptation"]}]}],"canonical_facts":{"dc:contributor":["Hauser, Kris","Wang, Yuxiong"],"dc:creator":["Qiu, Rizhao"],"dc:date":["2023-05","2023-05-03"],"dc:description":["Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2023-09-01 without embargo terms","The student, Rizhao Qiu, accepted the attached license on 2023-05-02 at 15:39.","The student, Rizhao Qiu, submitted this Thesis for approval on 2023-05-02 at 15:46.","This Thesis was approved for publication on 2023-05-03 at 12:31.","DSpace SAF Submission Ingestion Package generated from Vireo submission #19301 on 2023-09-01 at 16:56:10","One of the key characteristics of intelligent beings is the ability to perceive and interact with the surrounding environment. Increasingly many learning-based methods have been proposed in attempts to equip robots with such perception power. However, most existing robotics perception algorithms are still inappropriate for practical robotics applications because they 1) do not consider computational efficiency and require a lot of computational resources or 2) assume a fixed and closed set of objects to recognize and hence fail when robots encounter previously unseen objects or repeat the same failure patterns. This thesis aims to address these issues between perception algorithms and robotics systems by developing robotics perception algorithms that are real-time for downstream tasks and can adapt during deployment time for open-world robotics. Specifically, the co-designing of 3D semantic mapping and a downstream planner and continual adaptation from few data for 2D semantic segmentation are discussed. The first work, RA-SLAM (RA-Simultaneous Localization And Mapping), designs a real-time GPU-based volumetric semantic mapping system that understands scenes geometrically and semantically. It operates at 50Hz on a consumer-grade GPU - approximately 5 times faster than CPU-based TSDF semantic reconstruction methods. Two studies are then presented for RA-SLAM: first, it can be combined with a 2D high-touch affordance segmentation model to improve disinfection trajectory planning with human-like semantic awareness. Second, different multi-class semantic representations in voxels are investigated, which leads to a variant of RA-SLAM for multi-class semantic reconstruction that balances computation-memory usage. The second work to be presented, GAPS (Guided copy-And-Paste Synthesis), enables robots to learn continually with few data via training data synthesis. GAPS dramatically boosts the novel IoU of baseline methods on established few-shot continual segmentation benchmarks by up to 80%, and maintains good performance in even more impoverished annotation settings, where only single instances of novel objects are annotated. To investigate its potential for perception in human-in-the-loop robotics application such as tele-operating robotics, a study is then carried out to simulate continual few-shot learning on real robots, which shows that existing algorithms fail dramatically when the limited training budget on robots are considered. The thesis then describes a system approach that balances training-inference resources and allows continual learning algorithms to run on the onboard computers of robots without hindering their perception capability."],"dc:format":["application/pdf"],"dc:identifier":["https://hdl.handle.net/2142/120164"],"dc:language":["en","eng"],"dc:rights":["Copyright 2023 Rizhao Qiu"],"dc:subject":["Robotics","Segmentation","Computer Vision","Artificial Intelligence"],"dc:title":["Towards real-time robotics perception with continual adaptation"],"dc:type":["text"],"thesis:degree_discipline":["Computer Science"],"thesis:degree_level":["Thesis"],"thesis:degree_name":["M.S."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:24:56Z"}