MIT uses brain signals and hand gestures to control robots

The team harnessed the power of brain signals called “error-related potentials” (ErrPs), which naturally occur when people notice a mistake. The system monitors the brain activity of a person observing robotic work, and if an ErrP occurs — because the robot has made an error — the robot pauses its activity so the user can correct it. This happens via an interface that measures muscle activity — the person makes hand gestures to select the correct option for the robot.

In one trial, the team used “Baxter”, a robot from Rethink Robotics, to move a power drill to one of three possible targets on the body of a mock plane. With human supervision, Baxter went from choosing the correct target 70 percent of the time to more than 97 percent of the time. Critically, the system works with people it’s never seen before, so organizations could deploy it in real-world settings without needing to train it on new users.

The project’s lead author, Joseph DelPreto, said that the invention is particularly important because unlike traditional robotic management, users do not need to think in a prescribed way. “The machine adapts to you, and not the other way around,” he said, adding that the system “makes communicating with a robot more like communicating with another person.”

Being able to control robots in this way opens up new possibilities for how humans could manage teams of robot workers, but longer term, it could be useful for the elderly, or workers with language disorders or limited mobility. “We’d like to move away from a world where people have to adapt to the constraints of machines,” said project supervisor Daniela Rus. “Approaches like this show that it’s very much possible to develop robotic systems that are a more natural and intuitive extension of us.”