Generative models for predictive UI design tools

User interface (UI) design is a central part of the mobile app creation process, which involves specifying the elements that should be placed on a screen, and how they should be arranged and styled. This paper introduces a generative model approach to predictive design for mobile UI layouts. Given a partial UI design, the model predicts the next UI element that should be added to the layout. Moreover, the model can be used queried multiple times in succession to autocomplete an entire UI screen. To power this design interaction, we present two types of models: generative adversarial networks (GANs) [7] and variational auto-encoders (VAEs) [15]. We train the GAN and VAE models over 1949 mobile UIs that represent a variety of screen types (e.g. Login, Onboarding), and compare both models along standard and design-based metrics, identifying key tradeoffs. Finally, we present a mobile UI mockup tool that leverages the GAN-based model to support a predictive design workflow.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2019

Author dc:creator

• Situ, Jason Jun

Contributors dc:contributor

• Kumar, Ranjitha

• mobile design
• generative models
• gan
• vae
• design workflow

Statement dc:rights

• Copyright 2019 Jason Situ

Language dc:language

en

Handle dc:identifier

http://hdl.handle.net/2142/105107

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/105107