Improving Controllability in Diffusion-Based Image Inpainting through Structured Workflows and Preference-Based Model Adaptation

This thesis presents two complementary approaches for improving controllability in diffusion-based image generation. The first addresses limitations of existing inpainting-based virtual staging methods, which can produce objects that are semantically inappropriate, visually unconvincing, or poorly matched to the surrounding environment. To address this, an end-to-end prompt-based contextual virtual staging framework is developed that fine-tunes a diffusion model for controlled object generation and integrates it into an agentic pipeline for context-aware scene inpainting. Building on this, the second approach addresses the limitation that workflow-level control alone does not ensure that the underlying inpainting model fully internalizes prompt and scene context. To address this, a pretrained diffusion inpainting model is directly fine-tuned using preference-based optimization, together with a composite reward model and a region-aware evaluation system. Together, these approaches improve controllability at both the workflow level and the model level.

Name thesis:degree_name

M Eng Sci

Discipline thesis:degree_discipline

Electrical and Computer Engineering

Grantor dc:publisher

The University of Western Ontario

Year dc:date.issued

2026

Author dc:creator

• Murray, Scott Jason

Advisor dc:contributor.advisor

• Yang, Yimin

• Diffusion Models
• Image Inpainting
• Controllable Image Editing
• Preference Optimization
• Reinforcement Learning
• Virtual Staging
• Generative AI
• Stable Diffusion
• LoRA Fine-Tuning
• Context-Aware Generation

Statement dc:rights

• Attribution-NonCommercial-ShareAlike 4.0 International

Language dc:language.iso

en

Handle dc:identifier.uri

https://hdl.handle.net/20.500.14721/39622

OAI identifier oai:identifier

oai:uwo.scholaris.ca:20.500.14721/39622