Harnessing Synthetic Data for Robust and Reliable Vision

Progress in computer vision has been driven by models trained on large amounts of exemplar data for different tasks. These exemplar data sources intend to capture task-specific information and instance-level variations that a trained model will likely encounter in the wild. However, for conditions where curating lots of labeled real-world data is prohibitively expensive, synthetic data can serve as a cost-effective alternative. Synthetic data sources offer a few key benefits: fast access to labeled task-specific data at scale, labels across varying task complexities, and curation of labeled data across diverse conditions in a controlled manner. This thesis demonstrates how “controlled variations” in synthetic data can be used to develop robust and reliable vision models. Controlled variations refer to intentional, systematic modifications to synthetic data, designed to either explore specific aspects of model behavior or improve model transfer across distributions. In Chapters 3 and 4, we discuss applying controlled variations internally at the data-engine (simulator) stage to create diverse instances to systematically investigate the robustness of trained vision models. In Chapters 5 and 6, we discuss how applying controlled variations externally as perturbations or data augmentations (to intermediate features or input images) can enable model transfer across changing visual distributions. Finally, in Chapter 7, we discuss how controlled variations applied externally on synthetic data can ensure reliability of predictions made on real data distributions. We conclude by summarizing takeaways and outlining potential future research directions.

Level thesis:degree_level

Doctoral

Department dc:contributor.department

Interactive Computing

Grantor dc:publisher

Georgia Institute of Technology

Year dc:date.issued

2024

Author dc:creator

• Chattopadhyay, Prithvijit

Advisor dc:contributor.advisor

• Hoffman, Judy

Committee members dc:contributor.committeemember

• Batra, Dhruv
• Hays, James
• Garg, Animesh
• Mottaghi, Roozbeh

• Synthetic Data
• Robustness
• Generalization
• Reliability
• Computer Vision

Language dc:language.iso

en_US

Handle dc:identifier.uri

https://hdl.handle.net/1853/75661

OAI identifier oai:identifier

oai:repository.gatech.edu:1853/75661