ImageSI: Interactive Deep Learning for Image Semantic Interaction

Interactive deep learning frameworks are crucial for effectively exploring and analyzing complex image datasets in visual analytics. However, existing approaches often face challenges related to inference accuracy and adaptability. To address these issues, we propose ImageSI, a framework integrating deep learning models with semantic interaction techniques for interactive image data analysis. Unlike traditional methods, ImageSI directly incorporates user feedback into the image model, updating underlying embeddings through customized loss functions, thereby enhancing the performance of dimension reduction tasks. We introduce three variations of ImageSI, ImageSI_text{MDS}^-1, prioritizing explicit pairwise relationships from user interaction, and ImageSI_{text{DRTriplet}} and ImageSI_{text{PHTriplet}}, emphasizing clustering by defining groups of images based on user input. Through usage scenarios and quantitative analyses centered on algorithms, we demonstrate the superior performance of ImageSI_{text{DRTriplet}} and ImageSI_text{MDS}^-1 in terms of inference accuracy and interaction efficiency. Moreover, ImageSI_{text{PHTriplet}} shows competitive results. The baseline model, WMDS^-1, generally exhibits lower performance metrics.

Name thesis:degree_name

Master of Science

Level thesis:degree_level

masters

Discipline thesis:degree_discipline

Computer Science & Applications

Department dc:contributor.department

Computer Science and#38; Applications

Grantor dc:publisher

Virginia Tech

Year dc:date.issued

2024

Author dc:creator

• Lin, Jiayue

Chair dc:contributor.committeechair

• North, Christopher L.

Committee members dc:contributor.committeemember

• Faust, Rebecca Jane
• Huang, Lifu

• Semantic Interaction
• Deep Learning
• Dimension Reduction
• Images

Statement dc:rights

• In Copyright

Licence dc:rights.uri

• http://rightsstatements.org/vocab/InC/1.0/

Language dc:language.iso

en

Dc Identifier Other

vt_gsexam:40777

OAI identifier oai:identifier

oai:vtechworks.lib.vt.edu:10919/119283