{"id":{"repo_id":"bu","oai_identifier":"oai:open.bu.edu:2144/50458"},"canonical_url":"https://search.dev.ndltd.org/etd/bu/oai:open.bu.edu:2144/50458","repository":{"repo_id":"bu","name":"Boston University","base_url":"https://open.bu.edu/oai/request"},"display":{"title":"Low-power, low-complexity image classification with optical sensors","abstract":"Convolutional Neural Networks (CNNs) have become a very effective tool in image estimation and inference (e.g., noise reduction, segmentation, object recognition). However, CNNs are difficult to deploy in resource-constrained scenarios, such as edge devices, IoT sensors, mobile embedded systems, because of high computational requirements and large memory footprint. Clearly, there is a growing need to investigate lightweight neural-network architectures designed to provide high performance at significantly reduced computational cost, memory usage, and power consumption. In this thesis, we investigate a low-power, low-complexity hybrid optical-digital neural network that leverages a novel metasurface sensor recently developed in Professor Paiella’s lab. Unlike typical image sensors, this sensor outputs an edge-like map of the scene akin to the output of the first convolutional layer of a CNN. We simulate this physical sensor in software and combine it with just a few digital layers to assure low computational load and power consumption. Since different pixels of the sensor capture different edge orientations, we organize the sensor array into groups of 2-by-2 or 3-by-3 pixels capturing either 4 or 9 edge orientations. This leads to either 4-channel or 9-channel convolutional layer simulation. We jointly optimize the optical parameters of this layer and digital parameters of the remaining layers for image classification of low-resolution images. Our best-performing designs approach classification performance of equivalent fully-digital network within 2% points, while reducing computational complexity and power consumption by a factor of 7.","abstract_html":"Convolutional Neural Networks (CNNs) have become a very effective tool in image estimation and inference (e.g., noise reduction, segmentation, object recognition). However, CNNs are difficult to deploy in resource-constrained scenarios, such as edge devices, IoT sensors, mobile embedded systems, because of high computational requirements and large memory footprint. Clearly, there is a growing need to investigate lightweight neural-network architectures designed to provide high performance at significantly reduced computational cost, memory usage, and power consumption. In this thesis, we investigate a low-power, low-complexity hybrid optical-digital neural network that leverages a novel metasurface sensor recently developed in Professor Paiella’s lab. Unlike typical image sensors, this sensor outputs an edge-like map of the scene akin to the output of the first convolutional layer of a CNN. We simulate this physical sensor in software and combine it with just a few digital layers to assure low computational load and power consumption. Since different pixels of the sensor capture different edge orientations, we organize the sensor array into groups of 2-by-2 or 3-by-3 pixels capturing either 4 or 9 edge orientations. This leads to either 4-channel or 9-channel convolutional layer simulation. We jointly optimize the optical parameters of this layer and digital parameters of the remaining layers for image classification of low-resolution images. Our best-performing designs approach classification performance of equivalent fully-digital network within 2% points, while reducing computational complexity and power consumption by a factor of 7.","abstract_has_math":false,"creators":["Huang, Xulun"],"institution":null,"degree_name":null,"degree_level":null,"degree_discipline":null,"degree_department":null,"school":null,"contributors":[],"advisors":["Konrad, Janusz"],"committee_chairs":[],"committee_members":[],"year":2025,"date_issued":"2025","date_published":"2025","updated_at":"2026-07-24T01:24:28Z","subjects":["Artificial intelligence","Optics"],"languages":["en_US"],"rights":[],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/2144/50458","outbound_label":"Handle","outbound_source":"dc:identifier.uri"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor.advisor","label":"Advisor","values":["Konrad, Janusz"]},{"key":"dc:creator","label":"Author","values":["Huang, Xulun"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.accessioned","label":"Dc Date Accessioned","values":["2025-05-22T15:03:15Z"]},{"key":"dc:date.available","label":"Dc Date Available","values":["2025-05-22T15:03:15Z"]},{"key":"dc:date.issued","label":"Date","values":["2025"]},{"key":"dc:type","label":"Dc Type","values":["Thesis/Dissertation"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Artificial intelligence","Optics"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language.iso","label":"Language (ISO)","values":["en_US"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier.uri","label":"Identifier URI","values":["https://hdl.handle.net/2144/50458"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["2025"]},{"key":"dc:description.abstract","label":"Abstract","values":["Convolutional Neural Networks (CNNs) have become a very effective tool in image estimation and inference (e.g., noise reduction, segmentation, object recognition). However, CNNs are difficult to deploy in resource-constrained scenarios, such as edge devices, IoT sensors, mobile embedded systems, because of high computational requirements and large memory footprint. Clearly, there is a growing need to investigate lightweight neural-network architectures designed to provide high performance at significantly reduced computational cost, memory usage, and power consumption. In this thesis, we investigate a low-power, low-complexity hybrid optical-digital neural network that leverages a novel metasurface sensor recently developed in Professor Paiella’s lab. Unlike typical image sensors, this sensor outputs an edge-like map of the scene akin to the output of the first convolutional layer of a CNN. We simulate this physical sensor in software and combine it with just a few digital layers to assure low computational load and power consumption. Since different pixels of the sensor capture different edge orientations, we organize the sensor array into groups of 2-by-2 or 3-by-3 pixels capturing either 4 or 9 edge orientations. This leads to either 4-channel or 9-channel convolutional layer simulation. We jointly optimize the optical parameters of this layer and digital parameters of the remaining layers for image classification of low-resolution images. Our best-performing designs approach classification performance of equivalent fully-digital network within 2% points, while reducing computational complexity and power consumption by a factor of 7."]},{"key":"dc:title","label":"Title","values":["Low-power, low-complexity image classification with optical sensors"]}]}],"canonical_facts":{"dc:contributor.advisor":["Konrad, Janusz"],"dc:creator":["Huang, Xulun"],"dc:date.accessioned":["2025-05-22T15:03:15Z"],"dc:date.available":["2025-05-22T15:03:15Z"],"dc:date.issued":["2025"],"dc:description":["2025"],"dc:description.abstract":["Convolutional Neural Networks (CNNs) have become a very effective tool in image estimation and inference (e.g., noise reduction, segmentation, object recognition). However, CNNs are difficult to deploy in resource-constrained scenarios, such as edge devices, IoT sensors, mobile embedded systems, because of high computational requirements and large memory footprint. Clearly, there is a growing need to investigate lightweight neural-network architectures designed to provide high performance at significantly reduced computational cost, memory usage, and power consumption. In this thesis, we investigate a low-power, low-complexity hybrid optical-digital neural network that leverages a novel metasurface sensor recently developed in Professor Paiella’s lab. Unlike typical image sensors, this sensor outputs an edge-like map of the scene akin to the output of the first convolutional layer of a CNN. We simulate this physical sensor in software and combine it with just a few digital layers to assure low computational load and power consumption. Since different pixels of the sensor capture different edge orientations, we organize the sensor array into groups of 2-by-2 or 3-by-3 pixels capturing either 4 or 9 edge orientations. This leads to either 4-channel or 9-channel convolutional layer simulation. We jointly optimize the optical parameters of this layer and digital parameters of the remaining layers for image classification of low-resolution images. Our best-performing designs approach classification performance of equivalent fully-digital network within 2% points, while reducing computational complexity and power consumption by a factor of 7."],"dc:identifier.uri":["https://hdl.handle.net/2144/50458"],"dc:language.iso":["en_US"],"dc:subject":["Artificial intelligence","Optics"],"dc:title":["Low-power, low-complexity image classification with optical sensors"],"dc:type":["Thesis/Dissertation"]},"updated_at":"2026-07-24T01:24:28Z"}