Generative models for problems in imaging science

In recent years, generative models have risen to the forefront of machine learning research. Modern generative models such as generative adversarial networks (GANs) and invertible generative models (IGMs) are capable of approximating high-dimensional image distributions and synthesizing images with high perceptual quality. In imaging science, they are being investigated for several potential applications, such as inverse problems and image reconstruction, image-to-image translation, and dataset augmentation. In the first part of this thesis, generative models are investigated for solving inverse problems in imaging. Specifically, we first developed a new image reconstruction method using a multiscale IGM as a prior, which demonstrated high performance on image reconstruction from stylized, simulated magnetic resonance imaging measurements, and was robust to test-time distribution shifts. Next, a style-based GAN was employed as a prior in a framework for estimating an object of interest that is closely related to a known prior image. The approach accurately captured difficult-to-model semantic differences between the sought-after and prior images and estimated the object accurately in terms of conventional metrics. Third, variational Bayesian methods were employed to learn an IGM of objects directly from a dataset of noisy and incomplete images. The second part of this thesis focuses on evaluating generative models and data-driven priors in imaging. Specifically, the concept of "hallucinations" in the context of image reconstruction was formally defined and utilized to illustrate the effects of an incorrect data-driven prior on the image estimate. Lastly, a framework for evaluating GANs in terms of medically relevant statistics was developed. Perceptual measures for evaluating the GAN did not always correlate with the relevant measures developed, highlighting the urgent need to assess generative models in terms of relevant, task-informed statistics. Our findings directly inspired an ongoing large-scale competition on deep generative modeling for learning medical image statistics.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Electrical & Computer Engr

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2023

Author dc:creator

• Kelkar, Varun Ajit

Contributors dc:contributor

• Anastasio, Mark A
• Do, Minh N
• Zhao, Zhizhen
• Lam, Fan

• Generative Adversarial Networks
• Invertible Networks
• Tomography
• Diffusion Models
• Imaging Science
• Generative Models
• Inverse Problems

Statement dc:rights

• Copyright 2023 Varun Ajit Kelkar

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/121447

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F121447