Streaming low-bandwidth real-time video using video super-resolution

Jalapeño is a real-time video streaming platform designed primarily for tele-operations. The platform uses a traditional video compression approach (H264) and pairs that with unique networking optimizations and artificial super-scaling to increase reliability and decrease bandwidth consumption. Jalapeño assumes the vehicle (client) is running the platform on a device with low computational capabilities. Conversely, the operator is controlling the vehicle on a system with very high resources. Additionally, the client network is assumed to be transmitting over a lossy wireless channel. Real-time streaming also has tighter constraints than traditional live-streaming, hence some networking algorithms that rely on introducing artificial stream delays are infeasible. Jalapeño must alleviate the issues of video streaming under heavy loss conditions with a low-powered client under the challenging real-time constraints. The networking layer closely monitors the quality of the stream and adjusts the parameters of the compression and transmission dynamically. We leverage the temporal scalable video codec capabilities of the open-source OpenH264 codec by Cisco to design a reconstruction algorithm that works under real-time constraints to mitigate packet loss. The prime feature is video super-resolution, a GAN based generator that is able to upscale a low-resolution stream from the client to high-definition frames while maintaining temporal consistency. This thesis also outlines an approach to build a large video dataset for unsupervised video learning and discusses compression-aware super-resolution to advance the framework to the next level.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Electrical & Computer Engr

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2022

Author dc:creator

• Vohra, Sanchit

Contributors dc:contributor

• Patel, Sanjay

• video
• compression
• H264
• super-resolution
• super resolution
• video super-resolution
• real-time
• streaming
• teleoperations
• autonomy

Statement dc:rights

• Copyright 2022 Sanchit Vohra

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/115801

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F115801