Integration of Model- and Learning-based Methods in Image Restoration

Following the spectacular success of deep learning algorithms in image restoration tasks, there is growing interest in exploring how to combine the practical advantages of learning-based methods with the theoretical understanding that comes from model-based approaches. In fact, while deep learning methods often provide state-of-the-art performance, they usually have no performance guarantees and one cannot predict how well they would perform on new data. It is generally expected that the integration of such methods with ideas from model-based strategies could lead to a new generation of more powerful algorithms. Here we focus especially on problems motivated by image restoration tasks that include image inpainting, denoising and deblurring. After reviewing the most prominent strategies proposed in the literature to combine model- and learning-driven methods for problems in image restoration, we introduce two new algorithms designed to combine the interpretability of model-driven methods and the efficiency of learning-driven methods. Our first approach proposes a novel training strategy for deep convolutional neural networks (CNNs) that leverages theoretically concepts from transform domain and sparse representations. Its main idea consists of representing the convolutional kernels as linear combinations of predefined filter dictionaries where the coefficients of the linear combination are the only learnable parameters. We show that, by choosing the dictionaries carefully and imposing sparsity-based constraints motivated by the theory of sparse multiscale representations, this approach offers several potential advantages with respect to conventional CNNs, especially in the situation of small sample size. Our second approach exploits the recently introduced framework of the Implicit Neural Representation (INR), a coordinate-based neural network for imaging problems, which parametrizes images or shapes using a Multi Layer Perception (MLP). We propose a novel INR with a prolate spheroidal wave function (PSWF) as an activation function, taking advantage of ideas from the theory of band-limit function to derive an improved and highly competitive method for image representation and restoration.

Name thesis:degree_name

Doctor of Philosophy

Level thesis:degree_level

Doctoral

Discipline thesis:degree_discipline

Mathematics

Grantor

University of Houston

Year dc:date.issued

2024

Author dc:creator

• Zhao, Heng

Advisor dc:contributor.advisor

• Labate, Demetrio

Committee members dc:contributor.committeemember

• Papadakis, Emanuel
• Mang, Andreas
• Zhang, Ruda

• neural networks
• image restoration
• image inpainting
• image denoising
• implicit neural representation

Language dc:language.iso

en

Handle dc:identifier.uri

https://hdl.handle.net/10657/17751

OAI identifier oai:identifier

oai:uh-ir.tdl.org:10657/17751