Books - Read and Enjoyed

Living With Robots

What Every Anxious Human Needs to Know

Ruth Aylett and Patricia A. Vargas

MIT Press, 2021

Machine learning and artificial intelligence has made impressive progress in recent years and many success stories have also been reported in the general press. In the reports of success cases however, it is rarely mentioned how narrow the task is where the AI excelled. When considering the domains of robotics and the tasks that robots have to deal with, it becomes quickly apparent, that today’s robots cannot survive “in the wild”. For specific, simplified tasks in a well defined environment robots can impress with accuracy, speed, endurance and quality. That is why industrial robots have conquered manufacturing plants the world over where they do well defined, highly specialized tasks with amazing speed and combining high force with high precision and high reliability. However, when the task is more general, less well defined, when the environment and situation varies a lot and changes during the task, today’s robots often fail spectacularly.

In this very accessible book two robotics researchers from Edinburgh review the state of the art in robotics in a range of features and capabilities, such as movement, appearance, sensing, vision, location awareness, touch and handling, learning, interaction, speech and language. It turns out that in basically all areas the capabilities fall short of what humans can do, and it is fascinating to learn how they fail and what exactly the shortcomings are.

In 1996 the IBM computer DeepBlue has won a chess tournament against the then ruling world champion and in 2016 Google’s AlphaGo beat the reigning Go champion. But neither DeepBlue nor AlphaGo actually played these board games. They printed the next move on a video screen, then a human assistant executed the move on the board, observed the move of the human opponent and typed it into a keyboard. Manipulating chess and go pieces in a real-world environment turns out to be a tremendously difficult task; more difficult it seems than modeling the games in computer memory and calculating winning moves. First it involves vision to recognize the board and all the pieces. Then it requires an arm and a gripper to gently pick up the piece and move it to the new location without destroying it or touching and moving any of the other pieces. It is certainly possible today to build a chess playing robot arm when the situation is well defined and many limiting assumptions are made. The size of board and pieces have to be set within narrow limits, the lightning has to be defined and kept stable, other arms, humans’ or robots’ must not interfere in the movements, etc. Once built, the robot arm will do a decent job for the well defined task of moving chess pieces, but it will not work for Go or Backgammon, it will not work for travel chess cases with pieces less than 1cm or with boards in a park with pieces of the size of 1m. It will not work when the light gets dim and varies a lot or when active members of the audience point their fingers close to the pieces when discussing the wisdom of a strategy. The more general and unrestricted a task is, the more difficult it becomes to design and build a robot for solving it. We can build special purpose robot arms for moving chess pieces, but we are very far from building a robot that can play any board game after a short instruction, like a human could.

This observation, that narrow tasks are feasible but general tasks are not, holds true for all the capabilities discussed in the book: movement, vision, sensing, object manipulation, language, etc. In some cases the reason is that we do not have the hardware, in other cases we do not know how to endow a robot with world understanding and general intelligence.

It came as a surprise to me that also the current state of the art of available hardware is wanting, but it is witness to how tremendously sophisticated, versatile and effective our body is. When touching and grasping a chess piece, not only our vision system provides the necessary information, there are sensors in the skin, the fingers, hand and the arm, and feedback from the muscles that facilitate appropriate force and movement. The skin sensors notice the surface of the piece, its temperature, softness and fragility to apply the right force and to ensure a safe grip without destroying the piece. The sensors in the joints of fingers, hands and arms inform the brain of the relative positions of all limbs involved in the movement, allowing for fine tuning the motion beyond what is possible through the vision system. The muscles report how much force is applied and how much resistance is met, again allowing for fine tuning the motion taking into account the weight of the piece and the arm. This rich sensory information with multiple regulatory feedback loops, some local in the fingers, some involving the spinal cord, up to the high level planning in the neocortex, is necessary for adapting gracefully to the infinite variety of real world situations.

The book gives other fascinating examples from the domain of interaction. Robots that interact with other robots and with humans will need a “theory of mind” of these other actors in order to successfully cooperate. A theory of mind means that the robot has to have at least a limited understanding of the goals, interests, and values of the other actors. If the robot does not know the goals of another actor, it will only be of very limited help because then it can do only what it is explicitly and precisely told to do. It cannot do something which is obviously useful, helpful and necessary, but not explicitly expressed. All cooperation among humans is based on a sophisticated understanding of the goals of others, and if robots should graduate from tools with narrow competence to more general purpose helpers in the home, the hospital or the workplace, they need to be equipped with a model of humans, what they want, what they abhor and some basics about the human value system. Even more, robots will have to be able to adapt to individuals and learn on the task, what the humans want and what they do not like. Moreover, when robots interact with other robots, they will also need to learn a theory of mind of those other robots, in order to understand their actions and motivations. Because we are still far away from such technology, robots at home, in nursing homes or at the workplace are still confined to mere tools with narrow, well defined tasks.

The book is an excellent guide through this landscape and with many vivid examples it provides a qualified understanding of the feasible today and what we may expect in the next few years.

(AJ April 2022)