Leveraging the Latent Space for Model Understanding and Optimization

In recent years, the field of machine learning has seen massive growth in both the size and quality of models and performance on tasks such as classification or image generation. However, these models are typically limited by two key factors. First, models such as those used in tasks of text-to-image generation lack interpretation. Second, models that leverage the latent space to represent data struggle to capture high-level details. This often results in reconstructions which do not accurately represent the original data. First, to address the issue of interpretability in text-to-image models, we introduce WINOVIS, a novel dataset designed to probe models in their ability to interpret textual prompts. This approach reframes the task of pronoun disambiguation from a single mode of natural language to a multi-model problem involving both visual and textual understanding. Second, we turn our focus to models in image generation such as the VQ-VAE which often struggle to reconstruct images capturing the finer details of the original input image. By introducing lightweight and straightforward modifications to the VQ-VAE’s loss function and dictionary selection process, we enable the reconstruction of images that retain high-level details often absent from the reconstructions produced by the traditional VQ-VAE.

Name thesis:degree_name

M.Sc. Computer Science

Level thesis:degree_level

Masters

Discipline thesis:degree_discipline

Faculty of Mathematics and Science

Department dc:contributor.department

Department of Computer Science

Grantor

Brock University

Year dc:date.issued

2025

Author dc:creator

• Park, Brendan

Advisor dc:contributor.advisor

• Li, Yifeng

• TECHNOLOGY::Information technology::Computer science::Computer science

Language dc:language.iso

eng

Handle dc:identifier.uri

https://hdl.handle.net/10464/19106

OAI identifier oai:identifier

oai:brocku.scholaris.ca:10464/19106