Ubisoft and MiHoYo Among Publishers Signing Up for Nvidia's AI-Generated Video Character Tool

Ubisoft and MiHoYo, the developers behind Assassin’s Creed and Genshin Impact, are among the first studios to sign up for Nvidia Avatar Cloud Engine (ACE), a new AI-driven tech platform said to create more lifelike video game characters.

The technology works by passing players’ voices through a speech recognition model to create text, according to Nvidia, which is then added to a large language model (LLM). The LLM’s response is moved through text-to-speech and animation models to create the final product: in-game characters that can “speak” dynamically with players.

In a video demonstration of the technology shown at CES 2024, Nvidia showcased dialogue between two NPCs, who unsettlingly muse on the possibility of being AI-generated constructs, before the characters begin interacting directly with the human demonstrator’s speech. The demo was created by Nvidia in collaboration with Convai, a company working on conversational AI.

Other notable developers exploring the utilization of ACE include China’s two biggest gaming companies, Tencent and NetEase.

The technology, first revealed at Computex 2023, is trained on “safe and secure data,” according to Nvidia. It can be used “across any engine,” including Unreal Engine 5, and can run in real-time through the cloud or offline on local hardware.

Game developers have been ambivalent about AI since the technology became mainstream with the launch of ChatGPT in late 2022.

Speaking to IGN in April 2023, CD Projekt Red’s Pawel Sasko said AI is “just going to be another tool that we’ll use for productivity and game development.” Epic Games CEO Tim Sweeney said, “I think we’re going to see some really incredible advances and actual progress mixed in with the hype cycle where a lot of crazy stuff is promised. Nobody’s going to be able to deliver.”

Other developers, specifically in the creative fields of writing, acting, and animating, have voiced concern over having their work plagiarized or made redundant.

Abbey Veffer, who did voice work for The Elder Scrolls Online’s Necrom expansion, told IGN, “Due to common misconceptions around actors’ wages, many uneducated observers may think we’re all equipped with expensive lawyers who can save us at the drop of a hat. They also assume that these AI impersonations won’t jeopardize our jobs — but that’s entirely untrue. Most voice actors are not rich, and job stability doesn’t really exist in this industry. We’re all in the same boat, and we all deserve protection from AI abuse.”

AI was a major sticking point in actors’ and writers’ 2023 negotiations with Hollywood’s studios. Both The Writers Guild of America and The Screen Actors Guild ultimately earned protections from AI.

it’s set to be a big year for AI as the technology becomes more efficient and pervasive. Speaking at the APEC CEO Summit in November, OpenAI CEO Sam Altman said the tech is set to take “a leap forward that no one expected” in 2024.

For more on the topic, check out all of the commentary and analysis from IGN’s AI Week, an exploration of “all the ways that the rise of everything from ChatGPT to AI art will change games and entertainment, both for good and for ill.”

Jordan covers games, shows, and movies as a freelance writer for IGN.