A low latency remote rendering system for interactive mobile graphics

Mobile devices are gradually changing people’s computing behaviors. However, due to the limitations of physical size and power consumption, they are not capable of delivering a 3D graphics rendering experience comparable to traditional desktops. Many applications with intensive graphics rendering workloads are unable to run on mobile platforms directly. This issue can be addressed with the idea of remote rendering: the heavy 3D graphics rendering computation runs on a powerful server and the rendering results are transmitted to the mobile client for display. However, the simple remote rendering solution inevitably suffers from the large interaction latency caused by wireless networks, and is not acceptable for the applications having very strict latency requirements. In this thesis, I present an advanced low-latency remote rendering system that assists mobile devices to render interactive 3D graphics in real-time. My design takes advantage of an image based rendering technique: 3D image warping, to synthesize the final display image on the client from the multiple depth image references generated on the server. This Multi Depth Image approach can successfully reduce the interaction latency for any user interaction that changes the rendering viewpoint and at the same time maintain high rendering quality. The success of the proposed system depends on two key components: how to select the reference viewpoints and how to compress multiple depth image references efficiently. I propose a Reference Prediction Algorithm for real-time reference selection and a high performance warping-assisted video coding method to compress both image and depth frames in real-time. Furthermore, I also discuss about how to evaluate the real interactive performance of the proposed system. A prototype system that runs several visualization and game applications has been implemented to evaluate and demonstrate the proposed design.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2012

Author dc:creator

• Shi, Shu

Contributors dc:contributor

• Campbell, Roy H.
• Nahrstedt, Klara
• Hart, John C.
• Zhang, Zhengyou

• Remote Rendering
• Interaction Latency
• Mobile Graphics
• Real-time Video Coding
• 3D Image Warping

Statement dc:rights

• Copyright 2012 Shu Shi

Language dc:language

en

Handle dc:identifier

http://hdl.handle.net/2142/30899

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/30899