“You can always tell how serious a company is about unmanned by how seriously they talk about teleop” a vendor once told me. Nevertheless, we found an incredible amount of industry and investor resistance to our teleop-dependent approach. Source
The dream of full autonomy is a compelling one for us roboticists, but I wonder whether it is a practical one if you are looking for medium-term impact. Teleoperation solves many of the same economic and practical problems as full autonomy in many use cases of robotics while being easier to actually bring to users:
1 Teleoperation solves the safety problem
As discussed previously , safety is very hard in robotics. But if you have humans as providing the high level supervision or control input and monitoring systems in-the-loop, you don’t need to prove a priori that the agent will not have catastrophic failures to 1-in-a-billion accuracy. Yes there are concerns about hacking and the reliability of the hardware, but all of these same concerns apply to full autonomy.
2 We can still reap economic benefits
The founder of the now-defunct self-driving truck startup Starsky did a great writeup of why exactly it was that they failed. However, I found the far less popular follow up post covering the economics of their industry even more interesting:
Decoupling the driver from the truck is the easiest way to cut labor costs. American truck drivers are paid only for the miles they haul freight — the hours spent waiting to be loaded or unloaded and taking mandatory breaks are all unpaid. As a result, the $200/day drivers typically earn is really only for the 7 hours/day they move freight and not for the 14 total hours they’re on-duty. This delta is big — it means that the trucking company feels like they pay drivers $28/hr while drivers feel like they only get $14/hr (or $8/hr if you consider the 24 hours/day drivers spend in a truck).
Starsky's graphs are informative, showing asymptotic gains as you decrease the level of human supervision from human in-the-cabin to 1:1 human teleoperation and 1:many teleoperation, nearing full autonomy:
The lesson on the economics of labour may apply to many more industries than trucking. Decoupling physical labour from the need for physical presence allows for elimination of switching costs. This could increasing dynamism in sectors that require physical labour but have as of yet not been revolutionized by software, but without requiring an algorithm and interface as complex as the self-driving, and the safety and reliability verification to go along with it in every single industry. The internet didn’t remove the need for writers, but it did fundamentally the way information consumption works.
Basic economic efficiency will increase when you don’t need to physically move humans to the site of labour. This is already happening in mining, where semi-autonomous mining vehicles are continuously monitored from thousands of miles away.
3 You get human level reasoning today, instead of in “10 years”
State of the art language models operating in the domain of text generation, while very good, fail to show true understanding and instead generate what are essentially maximum likelihood estimates of what should be said next. Such problems of understanding are even harder in the relatively more complex environment of the physical world. We can teach very complex motor-skills, but there is arguably only a tenuous path to getting agents to generalize reliably in unstructured physical environments (consider the fact that researchers are still attempting to define some of the challenges in this space).
While humans are weak, get fatigued and struggle with fine motor control, they find actual supervision of tasks at a high level very easy. It is possible human supervision is the best way for us to specify what we want of robots. For example, coming up with the exact order of operations of moving material and equipment around on a construction site, and what happens under every scenario if something doesn’t go according to plan is a problem humans can handle through supervision, but no current heuristic or RL based system could handle.
With teleoperation, we can get the best of both worlds - domain adaptation and decision making from a human, and tireless muscles and perfect motor control from robotic embodiments.
4 The resulting data can bootstrap full autonomy
Efforts such as Scale AI and Roboturk are very promising for data collection for supervised learning and even training robotic systems. However, the amount of data able to be collected thorough financing collection upfront is likely going to be capped as the collection process itself is not economically productive. With teleoperation you can end up with a positive feedback loop - with doing a task leading to more data leading to doing the task better, all the while being paid to improve as you start off with a useful system. This could actually be a viable path to full autonomy in some domains, if the quantity of data eventually becomes large enough that you can provide safety and capability guarantees and take the human out of the equation entirely.
5 Teleop can be cool, too
A silly but I think important often overlooked final point. There is sometimes a resistance to embracing more practical but near term applications because of the perceived lack of aesthetic qualities. But teleop can definitely be awesome, too - just check out DexPilot: