The 7-foot-tall, 2,000-pound rover appeared miniscule compared with the vastness of Mars.
The Martian landscape that surrounded Curiosity stretched for miles, with distant mountains circling the robot in every direction.
“That’s the edge of the crater,” said Parker Abercrombie, a software engineer and the person assigned to keep me from running into anything valuable in the Jet Propulsion Laboratory’s Theodore von Kármán Auditorium.
• Photos: Virtual reality at NASA’s Jet Propulsion Laboratory
I had lost my spatial awareness because I was traveling to Gale Crater on Mars through a pair of mixed reality goggles called OnSight.
The rocky terrain, re-created in high definition, was formed in a video game engine automatically from images captured by Curiosity only weeks ago.
The technology is powered by Microsoft HoloLens, an augmented-reality headset that adds overlays to your field of vision.
NASA’s JPL provided a demo of OnSight and another internal HoloLens’ application, Protospace, during an unveiling of the technology May 17. Earlier this year, NASA revealed its “Destination: Mars” experience at the Kennedy Space Center Visitor Complex. The interactive exhibit will use a modified version of OnSight to take guests on a tour of Mars alongside holograms of Buzz Aldrin and Curiosity driver Erisa Hines.
The augmented reality created by OnSight
As I stood looking at Curiosity from an outcropping the robot had recorded during its Mars trek, it was easy to forget that, in reality, I was awkwardly clicking my finger into thin air to make menu selections and wandering around an auditorium with little awareness of my surroundings.
• Video: JPL, Microsoft collaboration allows scientists to work on Mars in virtual reality
I was completely immersed, until a scientist leaning at an inhuman 45-degree angle appeared and sped across the landscape.
The glitched scientist turned out to be Luther Beegle, a member of the Mars Science Laboratory team and a frequent user of OnSight. For years, Beegle and the other team members looked at two-dimensional images to find targets for Curiosity to investigate, but now they see everything just as Curiosity does.
Through OnSight, Beegle can walk next to Curiosity and determine if the robot’s arm could fit through a gap between rocks simply by looking at it.
Each time Beegle looked at a rock, a reticle appeared to show the focus of his attention.
NASA says that’s the appeal of the technology. Beegle not only has a better understanding of the shape and size of the terrain around Curiosity, but he also can share his observations in an instant with anyone else wearing the headset.
“You have everybody on the same page immediately,” he said.
OnSight is currently being used in a limited pilot program. But Beegle and his team members can’t wait until it’s integrated into their daily planning.
“We want it now; we wanted it six months ago,” he said.
‘We’re exploring a new science right now’
Jeff Norris leads the Ops Lab, the JPL team that created OnSight and Protospace, a mixed-reality tool that overlays 3D renderings of designs at their full size. The Ops Labs envisions future space travel using the mixed-reality headsets from beginning to end. Engineers will build future space systems and robotics using Protospace. They’ll plan out moves using OnSight. Astronauts on long-distance trips and on the space station will pop a headset on when they need to quickly learn how to complete complex tasks.
“I feel like we’re exploring a new science right now,” Norris said.
Using Protospace to design the next rover
In Protospace, Norris lifted a piece-by-piece replica of the future Mars 2020 Rover, Curiosity’s newer sibling, into the air during a demonstration. The life-sized parts fade and disappear as you move in close, allowing designers and engineers to see each and every component regardless of how far into the whole the part is installed.
Protospace builds the model using the actual Computer Assisted Design files used to make it. Like OnSight, it provides a different perspective that Norris says could help spot problems ahead of launch.
I found myself staying away from the virtual robot as Andy Etters, a mechanical engineer, explained his love for the technology. Though you can walk through the robot like a ghost passing through a wall, I couldn’t shake the feeling that I would bump into it and break something, especially since the Mars 2020 mission is estimated to cost more than $2 billion.
Etters laughed when I told him.
“It takes some getting used to,” he said. During a meeting during which his team used Protospace around a conference table, Etters said he brought his hand down hard while gesturing at a design and smashed it into a socket in the center of the table.
Etters sees it as another tool to ensure everything is ready by launch day. Where his team might have previously crowded around a monitor, or shared files to inspect the design, they might someday just crowd around a hologram.
Protospace is still very much a prototype. While engineers can see a life- sized version of their work, the interactions are somewhat limited beyond inspection.
When Etters and his team want to use it, they have to contact a technician in Norris’ lab, specify what they need it for and schedule a time for it. But Etters is hoping that will change as it gets further into development.
“I hope to just have it at my desk,” Etters said.
