{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/129754"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/129754","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"CNNFORMER interpolation: a demosaicing framework for single-chip color–NIR1 imaging in fluorescence-guided cancer surgery","abstract":"Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2027-05-01","abstract_html":"Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2027-05-01","abstract_has_math":false,"creators":["Yang, Jiankun"],"institution":"University of Illinois Urbana-Champaign","degree_name":"M.S.","degree_level":"Thesis","degree_discipline":"Electrical & Computer Engr","degree_department":null,"school":null,"contributors":["Gruev, Viktor"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2025,"date_issued":"2025-05-09","date_published":"2025-05-09","updated_at":"2026-07-22T22:25:05Z","subjects":["Machine Learning","In-vivo Imaging","Computer Vision"],"languages":["en","eng"],"rights":["Copyright 2025 Jiankun Yang"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/2142/129754","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Gruev, Viktor"]},{"key":"dc:creator","label":"Author","values":["Yang, Jiankun"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2025-05-09","2025-05"]},{"key":"dc:type","label":"Dc Type","values":["text"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Electrical & Computer Engr"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["M.S."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Machine Learning","In-vivo Imaging","Computer Vision"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en","eng"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2025 Jiankun Yang"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://hdl.handle.net/2142/129754"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2027-05-01","The student, Jiankun Yang, accepted the attached license on 2025-05-05 at 11:45.","The student, Jiankun Yang, submitted this Thesis for approval on 2025-05-05 at 11:54.","This Thesis was approved for publication on 2025-05-09 at 10:56.","DSpace SAF Submission Ingestion Package generated from Vireo submission #22098 on 2025-10-19 at 19:54:44","Single-chip multispectral imaging sensors that integrate vertically stacked photodiodes with pixelated spectral filters offer significant promise for advanced, real-time visualization during image-guided cancer surgery. However, the inherent design trade-offs of these sensors lead to a substantial reduction in spatial resolution, which can limit their clinical utility. Conventional demosaicing algorithms, such as bilinear interpolation and convolutional neural network (CNN)-based approaches, have been developed to mitigate these limitations by improving the effective resolution and reducing common image artifacts. While these methods have achieved meaningful improvements, they still exhibit deficiencies in accuracy and often introduce smaller, yet clinically relevant, reconstruction artifacts, particularly in regions with high spatial frequency content. To address these persistent challenges, this work presents a CNN–Transformer-based demosaicing algorithm specifically optimized for reconstructing high-resolution color and near-infrared (NIR) images acquired by a hexachromatic multispectral imaging sensor. Transformer architectures, originally developed for natural language processing to capture long-range dependencies within sequential data, have recently shown remarkable success when adapted to vision tasks due to their ability to model complex, global relationships across an image. By integrating convolutional layers for local feature extraction with Transformer modules for capturing long-range context, the proposed hybrid model was developed and trained on extensive color-image datasets and rigorously evaluated on both color and NIR imagery. Quantitative evaluation demonstrates that the CNN–Transformer demosaicing method achieves an average mean squared error (MSE) reduction of approximately 85% for color images and 76% for NIR images, along with improvements in structural dissimilarity (DSSIM) of roughly 72% and 79%, respectively, when compared to state-of-the-art CNN-based demosaicing algorithms on preclinical datasets. In clinical datasets, the model similarly achieves significant reductions in MSE and DSSIM, particularly excelling at reconstructing fine image details critical for accurate intraoperative visualization. These results underscore the capability of Transformer-based models to surpass the limitations of traditional CNN-only approaches by capturing more comprehensive spatial information. Leveraging recent advances in GPU computing, the proposed CNN–Transformer demosaicing framework offers a practical, real-time solution for enhancing spatial resolution and image fidelity in multispectral imaging applications, ultimately advancing the effectiveness of image-guided cancer surgery."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["CNNFORMER interpolation: a demosaicing framework for single-chip color–NIR1 imaging in fluorescence-guided cancer surgery"]}]}],"canonical_facts":{"dc:contributor":["Gruev, Viktor"],"dc:creator":["Yang, Jiankun"],"dc:date":["2025-05-09","2025-05"],"dc:description":["Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2027-05-01","The student, Jiankun Yang, accepted the attached license on 2025-05-05 at 11:45.","The student, Jiankun Yang, submitted this Thesis for approval on 2025-05-05 at 11:54.","This Thesis was approved for publication on 2025-05-09 at 10:56.","DSpace SAF Submission Ingestion Package generated from Vireo submission #22098 on 2025-10-19 at 19:54:44","Single-chip multispectral imaging sensors that integrate vertically stacked photodiodes with pixelated spectral filters offer significant promise for advanced, real-time visualization during image-guided cancer surgery. However, the inherent design trade-offs of these sensors lead to a substantial reduction in spatial resolution, which can limit their clinical utility. Conventional demosaicing algorithms, such as bilinear interpolation and convolutional neural network (CNN)-based approaches, have been developed to mitigate these limitations by improving the effective resolution and reducing common image artifacts. While these methods have achieved meaningful improvements, they still exhibit deficiencies in accuracy and often introduce smaller, yet clinically relevant, reconstruction artifacts, particularly in regions with high spatial frequency content. To address these persistent challenges, this work presents a CNN–Transformer-based demosaicing algorithm specifically optimized for reconstructing high-resolution color and near-infrared (NIR) images acquired by a hexachromatic multispectral imaging sensor. Transformer architectures, originally developed for natural language processing to capture long-range dependencies within sequential data, have recently shown remarkable success when adapted to vision tasks due to their ability to model complex, global relationships across an image. By integrating convolutional layers for local feature extraction with Transformer modules for capturing long-range context, the proposed hybrid model was developed and trained on extensive color-image datasets and rigorously evaluated on both color and NIR imagery. Quantitative evaluation demonstrates that the CNN–Transformer demosaicing method achieves an average mean squared error (MSE) reduction of approximately 85% for color images and 76% for NIR images, along with improvements in structural dissimilarity (DSSIM) of roughly 72% and 79%, respectively, when compared to state-of-the-art CNN-based demosaicing algorithms on preclinical datasets. In clinical datasets, the model similarly achieves significant reductions in MSE and DSSIM, particularly excelling at reconstructing fine image details critical for accurate intraoperative visualization. These results underscore the capability of Transformer-based models to surpass the limitations of traditional CNN-only approaches by capturing more comprehensive spatial information. Leveraging recent advances in GPU computing, the proposed CNN–Transformer demosaicing framework offers a practical, real-time solution for enhancing spatial resolution and image fidelity in multispectral imaging applications, ultimately advancing the effectiveness of image-guided cancer surgery."],"dc:format":["application/pdf"],"dc:identifier":["https://hdl.handle.net/2142/129754"],"dc:language":["en","eng"],"dc:rights":["Copyright 2025 Jiankun Yang"],"dc:subject":["Machine Learning","In-vivo Imaging","Computer Vision"],"dc:title":["CNNFORMER interpolation: a demosaicing framework for single-chip color–NIR1 imaging in fluorescence-guided cancer surgery"],"dc:type":["text"],"thesis:degree_discipline":["Electrical & Computer Engr"],"thesis:degree_level":["Thesis"],"thesis:degree_name":["M.S."],"thesis:institution_name":["University of Illinois Urbana-Champaign"]},"updated_at":"2026-07-22T22:25:05Z"}