Low-Complexity Structured Neural Networks and Their Usage in Image and Signal Processing

<p>Conventional neural networks face significant challenges due to high computational costs, large parameter counts, and reliance on backpropagation, which restricts their application in resource-constrained and real-time settings. To address these challenges, this thesis proposes three structured neural network (NN) architectures grounded in the theories of sparse and self-contained factorizations of transforms, with applications to image compression, reconstruction, classification, encryption, and also adaptive wideband multi-beam beamforming. The first neural network architecture, named DCTrix-Net, replaces conventional spatial con- volution with highly sparse factorization of the discrete Cosine transform (DCT) complemented by Toeplitz-structured weight initialization, achieving at least 97% FLOP reduction over CNNs, R-CNN, Faster R-CNN, DCT-Net, and YOLOv11s, and 50% parameter reduction when compared to CNNs, R-CNN, and Faster R-CNN, alongside the lowest inference time of order 10⁻⁴ seconds on the DigiFace1M dataset, compared among all benchmark NNs. The second neural network architecture, referred to as DSTrans-Net, performs image classification and encryption entirely in the frequency domain via learned pointwise discrete Sine transform (DST) filters based on the highly sparse factorization of the DST. This network achieves the lowest FLOPs at 0.02399 G and the least number of parameters at 6.75M com- pared to other benchmarked networks, including CNN, R-CNN, Faster R-CNN, EfficientNet- B0, and ViT-B/16 on the STL-10 dataset at 128 × 128 resolution, marking a reduction of at least 90% in FLOPs relative to all benchmark NNs and at least 62% in comparison with CNN, R-CNN, Faster R-CNN, and EfficientNet-B0. The third neural network, called the Forward-Forward Beamformer (FFB) algorithm, is designed to learn adaptive True-Time- Delay (TTD) wideband multi-beam beamformers. This network utilizes Delay Vandermonde Matrix (DVM)-based weight matrices, eliminating the need for backpropagation and achieving training times 99% faster than conventional neural networks across 7–15 GHz range, with antenna array elements of 8, 16, and 32, having an accuracy of beamformed signals in the order of 10⁻². In all three NN architectures, structured transform-based weight and sub-weight matrices unequivocally present a principled pathway in realizing low-complexity, lightweight structured neural network architectures tailored for applications in image and signal processing.</p>

Name thesis:degree_name

Master of Science in Data Science

Level thesis:degree_level

Thesis - Open Access

Discipline thesis:degree_discipline

Mathematics

Year

2026

Author dc:creator

• Kuzmicki, Adam

• Convolutional Neural Networks
• DCTrix-Net
• Complexity and Performance of Algorithms
• Image Recognition
• Structured Weight Matrices
• Discrete Sine Transform
• DSTrans-Net
• Wideband Multibeam Beamformers
• True-Time Delays
• Delay Vandermonde Matrix
• Adaptive Beamformers
• Forward-Forward Algorithm
• Artificial Intelligence and Robotics
• Data Science
• Numerical Analysis and Computation
• Numerical Analysis and Scientific Computing

Repository record dc:identifier

https://commons.erau.edu/edt/984

OAI identifier oai:identifier

oai:commons.erau.edu:edt-2031