Teleoperation, a seemingly simple capability, involves a multitude of technologies and systems in order to be implemented safely. In the first article from this series, we established what teleoperation is and why it is critical for the future of autonomous vehicles (AV). In the second article, we showed the legislative traction and emphasis gained for this technology. In this article, we will explain the first of the major technical challenges needed to be overcome to enable remote vehicle assistance and operation.
What is Stopping You
As humans, the sense we rely on the most is sight. Of any sense impairment, blindness is considered the most challenging one, and for good reason. This holds true for most of our daily activities and driving is certainly no exception. So if you want to drive a car from a remote location you need to be able to see the vehicle’s surroundings. If you were doing this with a device in your close proximity you might use an HDMI cable or your personal WiFi. However, AVs are moving over great distances and must rely on whatever public network is available.
5G is around the corner, sort of. It is being slowly deployed and is far from saturated. This is further slowed because tower range, as with every generational increase, is shorter than the previous generation. Even when 5G does reach full deployment, there will still be pockets of 4G networks for a long time to come. Currently, we have had 4G networks for over a decade and still find locations with only a 3G connection.
Once you accept that we cannot and should not wait for 5G you understand that we must work with what currently exists. Unfortunately, 4G is far from perfect. 4G LTE, or even WiFi for that matter, was not designed to support such high bandwidth and ultra-low latency communication from a moving vehicle across a vast geography. On top of that, there is major network inconsistency. Even if you avoid 3G areas, some 4G connections are eerily similar to their predecessor.
There is no single solution to the challenges of teleoperation with cellular networks, be they 4G or even 5G. You need a mix of technologies and a lot of innovation to enable safe and reliable vehicle teleoperation over cellular networks.
How to Cope
First, there will be unreliable network connections. Moving around, especially at speed, will bring with it variance in connectivity and, by extension, changes in network latency. It is imperative to establish a good connection and utilize the available modems to their highest potential. Each vehicle should be equipped with at least two cellular modems to work in unison and provide improved throughput, redundancy, and network coverage by minimizing packet loss and latency. Then, an AI-based algorithm (also known as a “packet scheduler”) can optimize channel usage to route packets via the most efficient channels in real-time. This is made possible through analysis of network signals, historical data, and considering each specific cellular tower and its characteristics.
Second, you are going to lose packets. It is the nature of any wireless network, especially cellular ones. However, when remotely operating a vehicle, you cannot afford to lose frames in your video as it can be the difference between avoiding a collision and not. With Dynamic Forward Error Correction (FEC) lost packets are reconstructed at the receiving end without re-transmission of data. This minimizes latency while enhancing the reliability of every channel. The “dynamic” part of the FEC algorithm minimizes the overhead throughput needed for packet reconstruction.
Third, there is going to be a jumble. Packets arrive out of order, there will be shifts in transmission, and there needs to be a delay in order to rectify that. Buffering ensures minimal video jitter (or: maximal video smoothness) and therefore there is always going to be a minimal need for it. The industry standard for quality video transmission normally requires a receive buffer greater than 300 millisecond. With the right technologies, among them an intelligent packet scheduler algorithm and a sophisticated FEC mechanism, general buffer time can be reduced by at least 75%. On top of that, based on network conditions, a dynamic receive buffer can additionally reduce buffer time. When network conditions are good, buffer size can be as small as 5 millisecond, and when conditions are suboptimal, it can max out at 50 millisecond. If the buffer system is only static then in order to ensure there is always enough time added one must use the maximum amount even if this is often wasteful.
Finally, all of the above is only possible if you know what is happening with the network at every given moment, and knowing that is only half the battle. Once network status is ascertained, the data needs to be translated for the different modules to use. The system must know the received power in each antenna, network latency, packet loss, throughput and more. Network quality is measured through analysis of several millisecond windows and comparing the number of packets sent with the amount received and combining that information with other network parameters. This data collector has to refresh many times per second to keep up with network variance. For every completed cycle, the results are analyzed and converted through complex processes into actionable information for the packet scheduler, FEC and dynamic buffer. Only then can the system respond to these measurements, every dynamic system adjusts accordingly, and usage is made even more efficient.
Four different technologies are needed just for this one part of teleoperation, namely packet scheduler, FEC, dynamic buffering and continuous network monitoring. However, like the other technologies to be explained in this series, not one can be skipped or its importance minimized. Sophisticated network technology which enables smart utilization of connectivity is essential to teleoperation. The other components will be laid out and explained in the future articles of this series.
