Advancing Blind Face Restoration: Robustness and Identity Preservation with Integrated GAN and Codebook Prior Architectures

Blind Face Restoration (BFR) is a challenging task in computer vision, which aims to reconstruct High-Quality (HQ) facial images from Low-Quality (LQ) inputs. BFR presents as a challenging ill-posed problem, necessitating auxiliary information to constrain the solution space. While geometric and generative facial priors provide some support in BFR, their effectiveness wanes under intense degradation. Discrete codebook priors, though promising, grapple with the difficulty of associating intensely degraded images with their corresponding codes. To effectively address these limitations, this research introduces a two-stage restoration approach, termed Identity-embedded GAN and Codebook Priors (IGCP), which synergistically combines the strengths of both generative and codebook priors. In the first stage, our approach employs a Generative Prior Restorer (GPR) network for initial image restoration. Distinct from existing methods that apply identity-based losses to the final restored image, our work innovates by embedding identity information directly into the style vectors of the StyleGAN2 network during the generation process. This is achieved through the introduction of an \emph{identity-in-style} loss, ensuring superior fidelity and identity preservation even in severely degraded images Proceeding to the second stage, the approach utilizes a two-component framework known as the Codebook Prior Restorer (CPR) network. This framework comprises a Vector Quantized AutoEncoder (VQAE) for artifact mitigation and to add a final touch of quality, complemented by introducing a Feature Transfer Module (FTM) that is demonstrated to be necessary to ensure fidelity and identity preservation. Extensive experimental evaluations were conducted across five datasets, including our newly introduced CelebA-IntenseTest dataset. The results from these experiments demonstrate the remarkable efficacy of the IGCP approach. Notably, IGCP has shown exceptional performance in handling various degradation levels, setting new benchmarks in the domain of BFR.

Author dc:creator

• Tayarani Bathaie, Seyed Nima

Advisor dc:contributor.advisor

• An, Aijun

Statement dc:rights

• Author owns copyright, except where explicitly noted. Please contact the author directly with licensing requests.

Language dc:language

en

Handle dc:identifier.uri

https://hdl.handle.net/10315/41866

OAI identifier oai:identifier

oai:yorkspace.library.yorku.ca:10315/41866