Meet the Team Bringing AI Agents to Minecraft

Introducing the MIT PhDs building next-generation AI agents for multiplayer games

You can use in-game chat to convince Altera’s autonomous Minecraft agents to fight each other.

Hey everyone.

We’ve sent out occasional A16Z GAMES newsletters before, but we're starting something new. Starting today you can expect regular drops of insider games industry content from our team and beyond. Hope you enjoy!

–Andrew Chen, General Partner at A16Z GAMES

For those reading via an email client, one quick ask before you drop in: Respond to this email. Let us know you're not an NPC so we can beat the spam filters.

In today’s edition, we’ll introduce you to the gigabrain MIT tech wizards making AI-powered Minecraft agents that are already causing buzz with players.

But first, the news roundup.

News From the Future

(Image credit: Nvidia)

🤖 Jensen Predicts Fully AI-Generated Games in 5-10 years (Tom’s Hardware)

An investor at last week’s Nvidia GPU Technology Conference asked founder Jensen Huang to predict when he thinks we’ll be “in a world where every pixel is generated at real-time frame rates.” Jensen’s take: the S-curve has already kicked in.

⛏️ Everything Epic Revealed at The State of Unreal 2024 (UnrealEngine.com)

Epic showed off a ton of stuff at GDC: the Epic Games Store on mobile, creator tools coming to other games (including Fall Guys), and dogfooding their own product with this promise: "By the end of 2025, we will ship our first season of Battle Royale developed entirely in UEFN."

🐉 Tencent Plans Fewer Big Foreign Franchise Games, More In-House (Reuters)

Tencent has long pursued big partnerships to bring western IP to mobile, but after some high-profile setbacks—most notably the now-shuttered Apex Legends Mobile—is now focused on investing in its own IP. This report raised a lot of eyebrows with a throwaway mention of failed game Undawn, which apparently cost $140 million.

🧑‍🌾 8 Years After Release, Stardew Valley is Smashing Player Records (Polygon)

After last week’s long-awaited release of the game’s 1.6 update, the solo-developed megahit Stardew Valley topped 236,000 concurrent players on Steam. The update added hundreds of items, new quests, and the ability to gather unlimited pets. Accordingly, one Stardew enjoyer on Reddit showed off his home filled with 250 cats.

Meet the Team Bringing AI Agents to Minecraft

Recently a video featuring some advanced AI tech hit the number one slot on Reddit’s r/feedthebeast, a sub focused on mods for the Java edition of Minecraft.

“We put 100 AI Agents in a game,” reads the post, which got over 1,000 upvotes and 175 comments, with many Redditors expressing curiosity about the tech behind the agents. “You need any more testers?” one Redditor wrote. 

This is the kind of reaction you dream of when you’re making a gaming prototype. And the creators of the Minecraft agents, A16Z GAMES SPEEDRUN 002 participants Altera, say they’re just getting started.

The Pitch

Altera’s vision is “Building digital human beings that live, love, and grow with us.” 

Altera’s CEO, Robert Yang, says he wants to give players access to persistent, customizable multiplayer agents they can play with across any number of different games—a sort of “ultimate party member” who remembers your adventures together, and who you can take from Roblox to Minecraft and back again across many different sessions.

Yang is a computational neuroscientist and former Assistant Professor at MIT, where he led a group of scientists and researchers focused on building neural network models of the brain. Yang has long been interested, he says, in modeling cognitive systems that govern thinking.

From left to right, Altera co-founders Shuying Luo (CTO), Robert Yang (CEO), Nico Christie (CBO), and Andrew Ahn (CSO).

Yang says he and his co-founders—a group including an MIT math PhD (Andrew Ahn), an ex-Google AI engineer (Shuying Luo), and a previously successful startup founder (Nico Christie)—are laser-focused on “not just impressing academia, but on making something that people can directly play with.” 

The company raised a $2 million pre-seed (led by SPEEDRUN) in December of 2023, and moved to San Francisco in January.

The Roadmap

To achieve Altera’s lofty vision, says Yang, the company is attempting to build AI agents with capabilities not seen in popular AI tools like large language models. The human brain, he points out, is capable of many forms of cognition that don’t involve language.

So to build “digital human beings,” Altera is working on capabilities like autonomous decision-making and the ability for agents to set their own goals. But the ambition goes even further. “They also need grounded sensation and actions so that they can directly interact with the world,” Yang says. “We want our agents to have episodic memory. We want them to be able to remember doing things with you a month ago.”

Animated GIF courtesy of Altera

Of course, getting there will take work. As of now, Altera’s agents are limited. They don’t always listen when players ask them for help, because they have their own goals that they’re interested in, Yang says with a laugh. But progress is moving quickly. “These are already agents with autonomy, who are able to interact with a virtual world and that have some memory of what they did.”

And Minecraft is the perfect playground for these early experiments. “It’s an open world with plenty of possible sophisticated actions that can be taken,” Yang says. 

Already, Altera’s early playtests are bearing fruit. Yang and his team were surprised when one of their first playtesters kept playing with their new AI friend for hours after the playtest officially ended. “Of course, you could have always played with your friend,” Yang says. “But you can’t customize your friend, and most of your friends aren’t going to be able to play with you at all hours like that. They’ve gotta get off and grab dinner.”

What’s Next for Altera

In the short-to-medium term, Altera is working on making their Minecraft agents more interesting and capable. Yang says the idea is that if Altera can improve the agents’ ability to form coherent long-term identities and memories, they’ll unlock more interesting social behavior between the agents. “Imagine if you can spawn in a bunch of agents and watch them collaborate to build a little society together,” Yang says. 

After that, the team plans to expand the agents to other games. For obvious reasons, Roblox is at the top of the priority list. Ultimately, Yang says, his team intends to integrate their tech with game engine SDKs so other developers can make use of their work. 

You can find more about Altera on its official website, and find Robert Yang on X.

More News From A16Z GAMES

📣 Calling All Founders: SPEEDRUN (SR003) Is Coming

A16Z GAMES SPEEDRUN is back.

SPEEDRUN is our early-stage accelerator for startups at the intersection of Tech x Games. This time around, we’re investing $750K in each selected company.

Our intensive 12-week program is highly selective, with ~ 1% of applicants accepted in our last cohort.

Participants are supported not just with capital, but also by a highly curated set of industry coaches, mentors, and a community of ambitious founders. 80% of companies from SPEEDRUN’s first cohort secured funding from investors following Demo Day.

Applications will be open from April 1st through May 19th, 2024:

• 🏦 Investment: $750K

• ⏰ When: 12-weeks starting July 29th

• 🌎 Where: Los Angeles, California

• 👯 Activities: In-person kickoff, live speakers and panels, dinners, and mixers, invite-only demo day

Interested? Click the button below for more details and a detailed FAQ for founders:

💰 A16Z GAMES Invests in Kaedim

Kaedim, led by CEO & founder Konstantina Psoma, is a new type of co-development platform providing AI-powered art outsourcing. Kaedim’s initial product is turning 2D images into ready-to-use 3D assets. The company is already serving over 20k new creators each month, and 250 enterprise developers, including many of the largest AAA game studios, film production companies, and e-commerce platforms.

Thanks to Konstantina’s relentless focus on customer value, we believe Kaedim will play a critical role as a next generation co-development platform for many of our favorite games and apps to come. See the full announcement here.

🪧 From the A16Z GAMES Team

Justin Paine’s Five Takeaways from GDC:

Lester Chen on YouTube’s strategy going into 2024:

💼 Jobs Jobs Jobs

There are currently around 150 open jobs listings across both the A16Z GAMES portfolio and our SPEEDRUN portfolio. For the freshest games industry jobs postings, be sure to follow our own Jordan Mazer and Caitlin Cooke on LinkedIn.

Join our talent network for more opportunities. If we see a fit for you, we'll intro you to relevant founders in the portfolio.

---

You are receiving this newsletter since you opted in earlier; if you would like to opt out of future newsletters, you can unsubscribe immediately.

This newsletter is provided for informational purposes only, and should not be relied upon as legal, business, investment, or tax advice. You should consult your own advisers as to those matters. This newsletter may link to other websites and certain information contained herein has been obtained from third-party sources. While taken from sources believed to be reliable, a16z has not independently verified such information and makes no representations about the enduring accuracy of the information or its appropriateness for a given situation.

References to any companies, securities, or digital assets are for illustrative purposes only and do not constitute an investment recommendation or offer to provide investment advisory services. Furthermore, this content is not directed at nor intended for use by any investors or prospective investors, and may not under any circumstances be relied upon when making a decision to invest in any fund managed by a16z. (An offering to invest in an a16z fund will be made only by the private placement memorandum, subscription agreement, and other relevant documentation of any such fund which should be read in their entirety.) Past performance is not indicative of future results.

Charts and graphs provided within are for informational purposes solely and should not be relied upon when making any investment decision. Content in this newsletter speaks only as of the date indicated. Any projections, estimates, forecasts, targets, prospects and/or opinions expressed in these materials are subject to change without notice and may differ or be contrary to opinions expressed by others. Please see a16z.com/disclosures for additional important details.