Don’t be afraid of these hallucinating MIT drones

While plenty of people have speculated about what virtual reality might mean for everyone from video gamers to office workers, scientists at the Massachusetts Institute of Technology have built a virtual reality environment for another audience: drones.

The system, which the researchers have dubbed “Flight Goggles,” enables unmanned aircraft to fly around a safe, open-testing environment while their cameras receive simulated visual input from another environment entirely. The goal is to train autonomous drones to safely fly in a variety of environments without having to build elaborate sets for them to experiment in–and stop to repair damage to the environments or drones as they learn to safely work in those environments.

“The drone will be flying in an empty room, but will be ‘hallucinating’ a completely different environment, and will learn in that environment,” said Sertac Karaman, an associate professor of aeronautics and astronautics at MIT and one of the scientists on the team, in a statement.

An on-campus drone-testing site is rigged with motion-capture equipment that tracks where a drone is actually flying, while a computer quickly feeds it details from where it’s located in a virtual environment. So far, they’ve trained a drone to fly safely through the window of a virtual living room, then built a real-world environment equipped with a similar window within the testing facility.

In those tests, the drone piloted itself through the actual window 119 times, only crashing or requiring human assistance six times, according to MIT.

The research team plans to present its research at the IEEE International Conference on Robotics and Automation in Brisbane, Australia, next week.

The team plans future research to test autonomous drones to safely compete against human-controlled ones in drone-racing events. Other experiments may train drones to safely fly around humans by partitioning the test area into one area where a drone can explore in virtual reality and another where someone in a motion-capture suit can move around. The drone would experience the human’s motion as if they were in the same area, enabling it to practice sharing a space without actually endangering any human lives.

“One day, when you’re really confident, you can do it in reality, and have a drone flying around a person as they’re running, in a safe way,” Karaman said. “There are a lot of mind-bending experiments you can do in this whole virtual reality thing. Over time, we will showcase all the things you can do.”