Teleoperation: How It’s Taking Over The Autonomous Vehicle Industry
How Do We Define Teleoperation In Autonomous Vehicles?
Starting at the very beginning, the easiest way to understand teleoperation in autonomous vehicles is to think of a person holding an umbrella in the rain.
The umbrella itself is ‘teleoperation.’ This umbrella encases the 3 individual sections that make up teleoperation: monitoring, indirect control / tele assistance, and direct control. The person that’s holding the umbrella is the teleoperator themself. As long as the human holds the umbrella, the person will stay dry. Likewise, as long as there is a human in the loop, even from a remote location, autonomous vehicles can be safely deployed.
This is the idea behind teleoperation in autonomous vehicles. Having a human in the loop is key to ensuring the vehicle can be monitored, assisted, or controlled at any time.
Obviously, the metaphor isn’t perfect. While the human needs to physically hold the umbrella to be dry, the teleoperator does not need to be physically in or near the vehicle to control it. But, you can understand that having a human in the loop in autonomous vehicles is necessary to ensure the AVs are safely deployed, and teleoperation is the method of doing this efficiently.
What Issues Can Teleoperation Solve?
- Trust amongst the passengers. There is a lack of trust amongst the general public when it comes to getting into an autonomous vehicle. Having a human in the loop can ease general unease.
- Emergencies. How does a vehicle know when to pull over if there are sirens? A teleoperator can understand things autonomy cannot.
- Facilities and construction zones. How can a vehicle in off-road facilities know where to go if they are being directed somewhere or being told to complete a task? The intuitive hand gestures are simple for a human to understand, even if they are not physically there.
- Dangerous conditions. If a vehicle needs to go somewhere that’s potentially dangerous for humans, such as a mining shaft. Having the operator sit in a remote location eliminates any bodily risk.
Technology Involved With Teleoperation In AVs
Some of the various types of technology that are required for the teleoperation of autonomously driven vehicles.
- Cameras
- LiDAR
- GPS
- Direction Sensors
- Radar
- Sensor Fusion
- 4/5G Networks
Teleoperation cannot rely on cameras alone. Instead many sensor types are required.
These technologies are among those that work together to ensure that an autonomous vehicle can drive on its own and be connected with a teleoperator at any point to be monitored, assisted, or remotely controlled.
Isn’t Teleoperation The Same As Remote Driving?
No, and this is a common mistake that is often made. Remote driving is indeed one part of teleoperation, but remote driving is not equivalent to teleoperation. Teleoperation is composed of monitoring, indirect control, and remote driving.
The 3 Parts To Teleoperation In AVs
As established, teleoperation is the umbrella term that is made up of 3 separate parts. Understanding how each one works is necessary to understand how teleoperation functions as a whole.
Monitoring
This is the basic function of teleoperation, and it is available in all teleoperated vehicles. Put simply, a teleoperator is able to monitor a vehicle from afar via cameras and sensors that are placed in the vehicle. The cameras and sensors transfer data to the compute stack which assesses what is happening inside and outside the vehicle. The data gathered is then sent back to the operator for review so they can decide if external intervention is necessary.
One example where monitoring a vehicle may be useful is if a passenger is in distress. A teleoperator may need to speak to the passenger or send emergency services to the vehicle.
One operator can monitor and service many vehicles.
Tele-assistance / Indirect Control
The second part of teleoperation is slightly more complex. It applies to every autonomously driven vehicle, including robotaxis and roboshuttles that drive on public roads as well as vehicles being driven in airports, mines, and agricultural areas. These vehicles are equipped with technology that allows a teleoperator to assist via tools including path choosing, path drawing, or policy override.
Should an autonomous vehicle signal to a teleoperator for assistance, the operator can only assist and cannot physically control the vehicle directly as in steering or braking. The vehicle is still fully in control of how the task is carried out.
Path Choosing
Should a vehicle come across an unusual path or object, it may trigger a request for a teleoperator with path options that the vehicle may take. The operator can then choose one of the correct paths, and the vehicle will continue driving with the new guidance.
Path Drawing
Should the vehicle not present any feasible options of path choices, the teleoperator can draw their own path for the autonomous vehicle to take. This may be to guide the vehicle around an obstacle or to guide the vehicle around unusually placed traffic cones or holes in the road. Once the new route is drawn, the vehicle carries out the driving itself as usual.
Policy Override
Two methods of intervention with Teleoperation
A teleoperator can also override policies that are in place in the vehicle. Should there be an object in the road and the vehicle triggers a request, the teleoperator can decide whether this object is simply able to be driven over or not. Suppose it is a piece of cardboard for example. In that case, the teleoperator can override the policy of “don’t drive over any objects”, allowing the vehicle to keep on driving as usual.
Remote Driving
A vehicle can be controlled entirely via a teleoperator with remote driving, including steering, braking, and acceleration. This is only available for small delivery bots and vehicles that do not drive on public roads, such as in airports, logistic centers, warehouses, shipping ports, mining, agriculture, and construction.
We All Make Mistakes. Is Remote Driving Via A Teleoperator Truly Safe?
One of the most significant issues with remote driving in teleoperation is the risk of driver error. This may be due partly to human mistakes, or down to fluctuation in the network connection. At Ottopia, we overcome the risk of human error with our Advanced Teleoperator Assistance System (ATAS®).
ATAS® works with teleoperation drivers to prevent collisions and accidents. Suppose a teleoperator doesn’t see a hole in the road during a moment of bad reception. ATAS® can stop the vehicle, alert the teleoperator of an issue, and allow them to take over from there.
Can A Private Vehicle Be Teleoperated As Well?
No. A private vehicle is at the sole discretion of the owner, and there is no benefit for a private vehicle to be teleoperated.
The only vehicles on public roads that will have any form of teleoperation installed on them in the near future are robotaxis, roboshuttles, autonomous trucks, and last-mile delivery robots. The vehicle will be able to drive with the benefit of teleassistance on all roads, getting the help it needs from a teleoperator should it need assistance in choosing the right path to go on.
Benefits Of Teleoperation In Autonomous Vehicles
These are some of the many benefits of teleoperation in autonomous vehicles:
Teleoperation Decreases Costs
This is true for both tele assisted vehicles on public roads and remotely driven vehicles on private roads. Tele Assisted vehicles, such as robotaxis, have no need for a driver inside the vehicle who sits and waits and drives from place to place. Having a remotely situated operator enables a decreased cost for taxi drivers, as one teleoperator can look after 5-10 vehicles at once.
With remotely driven vehicles, there is a significant reduction in costs. Take shipping ports for example. Most trucks, tractors, and forklifts are diesel-powered. Teleoperation reduces the dwell time amongst the trucks and terminal tractors that are waiting to pick up or unload their cargo. This reduces the amount of energy needed to power the heating or air conditioning, and it also reduces the need to pay a salary for the drivers, as one teleoperator will be monitoring several trucks from far away.
Minimizes Injury And Death
Autonomous vehicle technology can spot a hazard or dangerous situation quicker than a human can. Humans only have one set of eyes. An autonomous vehicle has tens of sets of eyes, with the installation of cameras both inside and outside the vehicles, LiDAR, sensors, and GPS tracking. There is a massive reduction in injury and death thanks to these technologies.
Furthermore, vehicles that are remotely operated reduce the risk of injury and death by physically removing the human from the situation. One such example is a dangerous mine or cave where it is a risk for a human to enter. This situation can simply be avoided by sending in a teleoperated vehicle and controlling it from a remote location with no humans nearby, should a dangerous situation occur. The vehicle can also carry out the job with high accuracy due to the enhanced cameras, accurate GPS signals, and sensor capabilities.
Decrease Pollution
Teleoperation leads to a decreased amount of gas used in diesel or gas-powered vehicles, as the dwell time of these vehicles is decreased. This leads to a decrease in the amount of pollution that is caused by such vehicles. In the climate crisis we’re in, any technology that can prevent pollution from being emitted into the air is a must, especially if that technology can carry out the same job as before.
Heavy Industries vehicles are not likely to go electric in the near future.
Sitting idly is bad for the environment.
So, Why Is Teleoperation Not Adopted Globally? Is There A Hidden Issue?
There are no hidden issues, one of the most important aspects of teleoperation is giving full transparency between the teleoperator and the end-user. The main reason that teleoperation is not yet adopted globally is due to a lack of legislation being passed, and a general lack of trust in autonomous vehicles. But, this mistrust lies heavily in self-driving machines, not in teleoperation. As the idea of having a ‘human in the loop’ spreads, governments and citizens are gaining trust in AVs through the power of teleoperation and the benefits it will bring in terms of both safety, accuracy, and efficiency to autonomous vehicles.
To give some examples, Germany is currently leading the way, having passed legislation that allows for tele assistance of vehicles on a public road. The US has several states that have legalized teleoperation, such as Arizona, California, Michigan, Ohio, Texas, and Florida. Other countries such as Canada (Ontario), the U.K, Japan, Sweden, Finland, and the Netherlands also require teleoperation to be a part of autonomous vehicle regulations. Although no concrete legislation has passed just yet in these countries, there is a clear trust in teleoperation and it’s only a matter of time before they follow in Germany’s footsteps..
As more countries gain awareness and trust of teleoperation technology, there is no reason why tele assisted and remotely driven vehicles won’t be part of the vehicles of the future.
FAQ
- What Is Teleoperation?
Teleoperation is the act of operating an autonomous machine or robot from a distance via a wireless connection. Nowadays, teleoperation most commonly applies to autonomous vehicles. This can range from delivery drones, robotaxis, shipping ports, mining, airports, and more. Teleoperation in autonomous vehicles requires an autonomous vehicle operator, also known as a teleoperator, who can be placed in a remote location, to be able to have access to the vehicle via a wireless network and to control it indirectly or directly. - Why Do We Need Teleoperation For Autonomous Vehicles?
Teleoperation is needed to get governments’ approval, solve situations currently too complex for autonomy, and gain the general public’s trust to enter a vehicle with no driver inside. The teleoperator is the missing link between testing and actually getting autonomous vehicles on the road. - Where Is Teleoperation Used?
Teleoperation is used on both public and private roads. For public roads, teleoperation technology includes both monitoring and indirect assistance of a vehicle, such as a robotaxi. The teleoperator does not control the acceleration or brakes, they merely assist in high-level commands such as path choice, or drawing a new path for the vehicle to take. On private roads, such as airports, mines, and agriculture, a vehicle can not only be monitored, and assisted but also remotely controlled by a teleoperator. This includes acceleration, braking, steering and manipulation of specialty appendages.
