How NASA’s Using Unreal Engine 5 to Prep for its Next Moonshot
Epic's game engine is unlocking innovation in fields far beyond games. We spoke with a NASA VR lead about how they're using tech like Unreal to simulate the lunar environment and train astronauts.
Video games have always been inspired by space travel.
The 1969 Apollo moon landing inspired some of the earliest computer games ever in the form of Lunar Lander games. And the first ever 3D game, Maze War, was famously developed by a few high school kids studying at the NASA Ames Research Center.1
In a neat historical parallel, NASA is using Unreal Engine 5 as part of its training and mission planning process for its upcoming plans to explore the lunar south pole.
This week we spoke to Eddie Paddock, Johnson Space Center’s Engineering VR Technical Discipline Lead, about how NASA is creating realistic virtual environments of the lunar surface for training purposes.
That story below. First, this week’s most futuristic news links:
💥 The AI x Game Dev Survey is LIVE!
Have your voice counted:
Tell us how your game studio is (or isn’t) using AI
Results will be posted in a few weeks
Fill out the survey here.
News From the Future
🇺🇸 The US Drives Mobile Games Sector Back to Growth (Pocket Gamer)
According to data gathered by AppMagic, the United States is leading global growth in mobile player spending, with revenue up to $13.26bn for the first half of 2024, a 12.29% year-over-year increase. Compare that to the overall global market, which is only up 2.2%. Monopoly GO!, Royal Match and Roblox are the big category winners.
🎥 Report: Warner Bros. is Considering Selling a Stake in Games Biz (VGC)
Sources told the Financial Times that WB is looking to offload smaller assets to help reverse a near 70% decline in its stock price since WarnerMedia and Discovery merged in 2022. This comes a month after WB announced it had acquired Player First Games, the studio behind F2P platform fighter MultiVersus, for an undisclosed sum.
🤖 College Football 25 Wouldn’t Be Possible Without AI, EA boss Says (Polygon)
EA head Andrew Wilson said that the team behind CF25 leveraged generative AI tech when creating the over 11,000 “heads” of players in the game. Wilson said the that human artists then supplemented that algorithmic work by touching up and enhancing the players likenesses.
📰 The Video Game Industry Mourns the Loss of Game Informer (The Verge)
It was a blow we all knew was coming, as print media and game magazines in particular have fallen into irrelevancy over the last decade. But Game Informer was something special for the games industry. At the height of its popularity in 2011, it was the third-largest magazine in the United States, with over 8 million copies circulated.
How NASA’s Using Unreal Engine 5 to Prep for its Next Moonshot
NASA is almost ready to send humans back to the moon.
Currently planned for “no earlier than” September 2026, Artemis III2 will send the first humans to explore the lunar south pole region. The astronauts will ride NASA’s Orion spacecraft atop the SLS (Space Launch System) rocket to get from Earth to lunar orbit, where they will dock with SpaceX’s Starship Human Landing System3, which is capable of taking them to and from the surface of the moon.
Once they’re kicking moon rocks, the astronauts will spend about a week exploring and conducting experiments, including (hopefully) on ice hidden in the depths of lunar south pole’s craters.
It’s a high-stakes mission, requiring the best technology available. And to help prepare astronauts for the rugged lunar surface, NASA is using video games. The agency is building real-time, VR-capable simulations of the moon, one of which uses Unreal Engine.
Where We Dropping?
The story of humanity’s planned return to the moon is decades in the making. The use of Unreal Engine to render the moon is only possible thanks to a 2009 mission that saw NASA launch the Lunar Reconnaissance Orbiter, a robotic spacecraft designed to create detailed 3D maps of the moon as it zips around in endless orbits.
The device is absolutely covered in measuring equipment, including a neutron detector, a cosmic ray telescope, and a laser altimeter.
In layman’s terms, it’s a robot that shoots lasers at the moon. The result is a massive trove of data about the lunar surface.
“We can actually calculate the diameter and depth of craters and we have some tools where we can synthetically create height map data from the images,” says VR Lead Eddie Paddock of Johnson Space Center. “We basically have a complete environment simulated of the moon at this point, but it’s all at various resolutions.”
The resolution data is measured in meters, with 5 meter data offering a good-quality view of the lunar surface and 1 meter data offering higher-resolution information. NASA’s highest-resolution maps focus on the particular areas around the lunar south pole where the upcoming Artemis missions will explore, and use a technique called “shape from shading” that calculates height of the lunar surface based on a mix of laser altimeter data and images showing the shapes of shadows.
“We put that data in a geospatial format,” Paddock says, “and then you can ingest that into game engines and add textures, and create meshes of terrain.”
With the game engine-version of the moon thus constructed, NASA is then able to build simulations that allow astronauts to practice flying landers or driving lunar rovers over the craggy surface.
Earthshine and Long Shadows
Of particular importance for astronaut training is the ability to realistically render the lunar south pole’s long shadows, which move so slowly that they can be treated as “handrails” for navigating the lunar landscape. This is where Unreal comes in.
“We try to create lighting conditions with the sun and synthetic lights so we have the correct shadows at the correct angles,” Paddock says. This includes the need to simulate light reflected by the earth—called earthshine—which due to the size and reflectance of the earth and the lack of atmosphere on the moon, can be up to 50 times greater than the moonshine we see on earth.
“Once you start adding multiple light sources and reflection and shadows, the renderer has to support that in a real time, as close to real time as it can.” Paddock says.
NASA’s insistence on simulating lunar lighting accurately isn’t simply a matter of making an immersive VR sim for astronauts to train in—though that is part of its appeal. There’s a practical element as well: Paddock says that astronauts need to use shadows to navigate the lunar surface.
“They may use a shadow along a ridge as a handrail to navigate to a certain permanently shaded region,” Paddock says. “And we can create that and create the maps for it. It's about helping them go to particular science areas of interest and get back safely. Things like, we need to put a beacon on the top of the lander to see it. When does the lander go out of sight? Can we go to this area? So it's about more than just making it feel like being on the moon.”
Sidebar: Rivian’s Unreal Engine-Powered Dashboard
Unreal Engine isn’t only showing up in high-powered technical environments like NASA’s astronaut training programs.
The real-time 3D game engine tech is also appearing in places you’d never expect, like the dashboard of Rivian trucks and SUVs. With its second-generation R1 vehicles, Rivian introduced a new Unreal Engine-powered update to its dash-integrated infotainment system that shows off your vehicle using a slick, cel-shaded art style that we love.
Eddy Reyes, Rivian’s Senior Manager of Embedded Software In-Vehicle Experiences, says that the vehicle-maker hired from both the games industry and Hollywood to create the Unreal-powered dashboard experience. “Some have been working in automotive 3D for a decade, some are from the AAA game world (like Call of Duty), and others who worked in VFX film (like Transformers).”
According to Reyes, the tech is lightweight, running on a Snapdragon 640 processor. “We knew we wanted to evoke an emotional reaction, and from there the visual style was built around the engine and our hardware,” Reyes says.
When asked whether the car company has any plans to create a playable experience using the tech, Reyes’s response was “stay tuned!”
Lunar Lander Goes Next Gen
Walking around the moon in VR is one use for NASA’s sims, but the real fun comes with the vehicle sims. The team has built both a lunar landing simulator and a very fun-looking rover sim for astronauts to practice driving around and dodging craters. These use a mix of virtual environments and real hardware to create a sim that’s as close to the real-deal as possible.
“We have motion-based systems, so it has the right cues for all of the bumps and the motions, the pitch, yaw, and roll, and use table washout to simulate acceleration,” Paddock says. “But then everything else is virtual, so I can operate with real controls that are hooked up to a high-fidelity simulation.”
Once the astronauts actually go to the moon, of course, they’ll be feeling all the bumps for real. But Paddock says the team is exploring ways to get them powerful devices so they can refer to their simulation software even while on the lunar surface. The goal would be to empower astronauts with “a handheld map that shows all my landmarks and craters and mountains, with real-time shadows,” says Paddock. A useful navigation tool since there’s no GPS on the moon.
The difficulty, of course, is that the tech is a bit on the demanding side. “They have tablets. So one of the challenges is to get our stuff to run on that,” says Paddock. “Now we’re running on an expensive personal computer with a top-notch graphics card..”
Upon learning this, our natural follow up question was whether NASA will be sending gamer laptops to the moon for the astronauts.
“We can't even get a gamer laptop on the International Space Station,” says Paddock with a laugh. “They have laptops, but it takes so long to flight certify and by the time you get them up there, they're out of date.”
At least Lunar Lander will still work.
A16Z GAMES thanks Eddie Paddock of Johnson Space Center and the JSC media team for their help with this interview.
🤝 The A16Z GAMES Marketing Team is Hiring!
We’re hiring a Marketing Partner to support the A16Z GAMES team!
Check the listing for the full details.
🔦 An Interview with Alan Wake 2 Director Kyle Rowley
In this episode of "Win Conditions" we sit down with Kyle Rowley, Game Director of Alan Wake 2 about how he went from being a fan of Max Payne to working on titles like Runescape and Cyberpunk?
And we ask: how does one sustain a 20-year career in game dev, an industry notorious for its ups and downs? Enjoy the full episode below:
🧊 A16Z GAMES SPEEDRUN 003 Kicks Off
Last week we kicked off in-person events for the third round of SPEEDRUN, our Games X Tech startup accelerator, at our brand-new Santa Monica a16z office.
Over the next 12 weeks, founders at 37 companies will gather, collaborate, and hear from industry-leading mentors as they prepare for Demo Day in October.
💼 Jobs Jobs Jobs
There are currently over 100 open jobs listings across both the A16Z GAMES portfolio and our SPEEDRUN portfolio. For the freshest games industry jobs postings, be sure to follow our own Caitlin Cooke and Jordan Mazer on LinkedIn.
Join our talent network for more opportunities. If we see a fit for you, we'll intro you to relevant founders in the portfolio.
You are receiving this newsletter since you opted in earlier; if you would like to opt out of future newsletters, you can unsubscribe immediately.
This newsletter is provided for informational purposes only, and should not be relied upon as legal, business, investment, or tax advice. You should consult your own advisers as to those matters. This newsletter may link to other websites and certain information contained herein has been obtained from third-party sources. While taken from sources believed to be reliable, a16z has not independently verified such information and makes no representations about the enduring accuracy of the information or its appropriateness for a given situation.
References to any companies, securities, or digital assets are for illustrative purposes only and do not constitute an investment recommendation or offer to provide investment advisory services. Any references to companies are for illustrative purposes only; please see a16z.com/investments. Furthermore, this content is not directed at nor intended for use by any investors or prospective investors, and may not under any circumstances be relied upon when making a decision to invest in any fund managed by a16z. (An offering to invest in an a16z fund will be made only by the private placement memorandum, subscription agreement, and other relevant documentation of any such fund which should be read in their entirety.) Past performance is not indicative of future results.
Charts and graphs provided within are for informational purposes solely and should not be relied upon when making any investment decision. Content in this newsletter speaks only as of the date indicated. Any projections, estimates, forecasts, targets, prospects and/or opinions expressed in these materials are subject to change without notice and may differ or be contrary to opinions expressed by others. Please see a16z.com/disclosures for additional important details.
In 2015 Polygon published an excellent feature story about the creation of Maze War.
Check out more information about the Artemis III mission on the official NASA website.
The “Human Landing System” is also called the HLS. NASA absolutely adores acronyms.