Edge processing for remote operation of automated vehicles in variable network environments

The remote operation of road vehicles is a vital link towards enabling fully autonomous vehicles to operate on public roads. Current generation mobile networks have the theoretical throughput capacity to support the streaming of high quality imagery from multiple remotely operated vehicles, but in practice are often limited due to insufficient range and network saturation. As a result, actual network throughput is often limited, posing a developmental barrier towards the implementation of remotely operated vehicles outside zones of good connectivity. Being able to adjust to varying amounts of network throughput, while providing enough information for a remote operator to maintain situational awareness, represents a key challenge which is to be addressed in this research by the use of edge processing to pre-interpret and simplify imagery using computer vision before it is sent to a remote operator. This study proposes the novel use of computer vision to transmit low data rate salient information about detected road users, instead of high data rate imagery, to overcome the constraints associated with operating a remotely controlled vehicle on a low throughput 4G mobile network connection, enabling a continued glass-to-glass latency of under 230ms even when the available network throughput is below 500kb/s. The first approach augments highly compressed imagery by encoding the shapes of objects detected by YOLO11x computer vision model so that they can be displayed to a remote operator as colour coded highlights, reducing the required throughput by 50%. The second approach utilises Fourier descriptors to send only the outline of road users and infrastructure extracted from a computer vision segmentation, requiring only 15% of the throughput of conventional imagery streaming methods. Applications for the Fourier descriptor segmentation technique potentially exceed far beyond just remote operation, as being able to transmit a spatial representation of detected objects in real time could facilitate the exchange of data between robotic agents for the purpose of collaborative environmental perception or assist in the efficient exchange of information where network quality has been compromised.

Grantor dc:publisher

Oxford Brookes University

Author dc:creator

• Donnelly, Sebastian

Contributors dc:contributor

• Bradley, Andrew
• Rast, Alexander
• Ball, Peter

Statement dc:rights

• All rights reserved

Language dc:language

en

DOI dc:identifier

https://doi.org/10.24384/wq6w-t322

OAI identifier oai:identifier

tle:e525c769-f439-4735-b13c-4041dbb09433:d6bd9758-527a-46cd-bfe2-c433766e8fca:1