Microsoft has introduced an open source virtual reality toolkit for the training of autonomous drones. Part of Microsoft’s Aerial Informatics and Robotics Platform, the beta software became available on GitHub last week.
The toolkit is designed to allow developers to “teach” drones how to navigate the real world by recreating conditions such as shadows, reflections and even objects that might confuse a device’s on-board sensors.
The software allows researchers to write code for aerial robots such as drones, as well as other gadgets, and to test the devices in a highly realistic simulator. Users can collect data while testing devices before deploying them in real world scenarios or situations.
“The aspirational goal is really to build systems that can operate in the real world,” said Ashish Kapoor, the Microsoft researcher who is leading the project.
The hope is that these training tools could spur development of artificial intelligence-based gadgets that eventually could be trusted to drive cars, deliver packages, and even handle rudimentary chores in the home, added Kapoor.
Advanced VR
Testing in a VR environment could mean lower costs as well.
Simulators long have been used in testing scenarios, but until recently the software-based simulations lacked the accuracy of the real world and thus didn’t reflect real-world complexities. Microsoft’s system — which is based on emerging VR technologies that take advantage of advances in graphics hardware, computing power and algorithms — enables a much more realistic re-creation of a real-world environment.
Based on the latest photorealistic technologies, it can render shadows, reflections and other subtle things much better. Although humans take such things for granted, they can pose problems for computerized sensors.
Microsoft’s simulator “will help researchers to develop, debug and test their drone navigation software by enabling them to recreate a variety of operational scenarios on their desktop computers in the lab,” said Michael Braasch, professor of electrical engineering and computer science at Ohio University’s Avionics Engineering Center.
“Simulations help to reduce development costs by reducing the amount of actual flight testing required, but the catch is that the simulation must be high fidelity — that is, sufficiently realistic,” he told LinuxInsider.
“Microsoft’s simulator appears to meet this requirement for camera-based or vision sensors, but it is not yet clear if the simulator accurately depicts very small-scale obstacles such as the thin twigs at the end of tree branches,” Braasch added. “Such obstacles are nearly invisible — even with HD cameras and even at close distances. It is also unclear if Microsoft’s simulator supports non-camera-based sensors such as LIDAR and radar.”
Learning to Fly
Although it targets the development of autonomous drones, Microsoft’s technology could find applications with human operators as well. Consumer drones have been steady sellers in the past few years, but newbies likely experience a crash or two. Learning to fly in a simulator could solve some of the problems with learning to fly.
“First, it isn’t easy to fly a drone,” said Michael Blades, senior industry analyst at Frost & Sullivan.
“This is therefore a good way not only to learn how to fly a new platform, but for some operators would allow the testing of new sensors and devices before risking [flight] in the real world, where a crash could be expensive,” he told LinuxInsider.
“In many cases, as modular drones for specific uses become commonplace, the added sensors and other devices could be more expensive than the drone itself,” Blades added. “Microsoft has a lot of experience with flight simulators, so this is a natural extension of what it has created already and opens it up to a wider audience as an open source platform.”
Crowded Skies
Microsoft’s open source simulator could be a welcome development at a time when drone regulators are struggling to catch up with the dramatic increase in commercial and recreational drone use.
“With so many new operators joining the ranks every day, the need for training and safety testing has never been greater,” said Bill Walsh, co-chair of the aviation industry practice group at Cozen O’Connor.
An added benefit of the Microsoft open source simulator is that it could warrant consideration of a revised regulatory requirement that all drone users complete simulator training, Walsh told LinuxInsider.
“That may only come about if we begin to experience multiple crash events that cause injury,” he acknowledged, “but if the system proves effective at training drone operators as well as product developers, simulator training requirements could eventually help solve a significant problem facing current drone regulators: people who do not know what they are doing and are not taking the time to learn.”
Even for more experienced operators, VR technology could address flying a drone in unfamiliar locations and situations.
“The simulator may provide a more realistic testing environment to the extent that it incorporates more realistic environmental factors,” said Walsh. “Flying a drone over the farmlands of Cambridgeshire at 200 feet does not adequately represent the challenges a drone would have flying at 400-500 feet in Seattle in February or Phoenix in July.”
Open Skies
Microsoft’s open source drone and VR simulator also could be an important step forward in making drones and drone-related services more accessible.
“Developers of drone flight control systems can use Microsoft’s new simulator to extensively test their obstacle avoidance and awareness features,” said Mike Israel, CEO of AirVuz.
“Since there are an infinite number of possible scenarios a flight control system could have to resolve to avoid collisions, a simulator must be able to be continually modified to deal with new scenarios,” he told LinuxInsider.
“Microsoft probably realized the complexity of this issue and the impossibility of accounting for every possible obstacle-avoidance situation in an out-of-the-box simulator,” Israel suggested.
“Having an open source approach makes sense, because anyone who uses it will be able to make improvements and share with others in the flight control software development community,” he added. “Drones will play an important role in revolutionizing many industries — industries way beyond transportation and entertainment — but there have been significant concerns about their safety and reliability. This technology will help mitigate those concerns and flesh out potential bugs before a product hits the market.”
Loading ...
