The ILM Evolutions series continues in celebration of Industrial Light & Magic’s 50 years of innovative storytelling, with Amy Richau looking back at ten years of Immersive Entertainment.
Industrial Light & Magic (ILM) has always been at the forefront of innovation, drawing audiences into new worlds by pushing technological boundaries. As part of our special series, “ILM Evolutions,” ILM.com talked with Vicki Dobbs Beck (vice president, immersive content innovation), Julie Peng (director of production), Tim Alexander (visual design director), Ben Snow (senior visual effects supervisor), and Shereif Fattouh (executive producer) about the past, present, and future of ILM’s immersive storytelling.
Photo: ILM’s LiveCGX team, including visual effects supervisor Mohen Leo (bottom).
A New Way to Tell Stories
While ILMxLAB was formally established in 2015 to explore the possibilities of immersive storytelling, the seeds of this endeavor actually began much earlier for Vicki Dobbs Beck. In the 1990s, Beck worked at Lucasfilm Learning, where a certain prototype caught her attention. “It was called Paul Parkranger and the Mystery of the Disappearing Ducks,” Beck tells ILM.com. “And what was really cool about all of the projects we were doing at that time is they really did sit at the intersection of storytelling, interactivity, and high-fidelity media – such that it was back then – all through an educational lens.”
Fast-forward to Beck’s time at ILM as head of strategic planning, she started bringing together talent from ILM and LucasArts (now Lucasfilm Games). “It was kind of this little rebel unit that was doing some pioneering R&D [research and development] in high-fidelity, real-time graphics,” says Beck. “Their success gave us confidence that the foundation was in place to build an immersive storytelling studio, expanding on the R&D work done by teams like the Lucasfilm Advanced Development Group (ADG).”
In 2015, ILM and Lucasfilm announced the formation of ILM’s Experience Lab (ILMxLAB) – a new division that would combine the talents of Lucasfilm, ILM, and Skywalker Sound. Lynwen Brennan, then Lucasfilm executive vice president and ILM president, announced that “The combination of ILM, Skywalker Sound, and Lucasfilm’s story group is unique and that creative collaboration will lead to captivating immersive experiences in the Star Wars universe and beyond. ILMxLAB brings together an incredible group of creatives and technologists together to push the boundaries and explore new ways to tell stories. We have a long history of collaborating with the most visionary filmmakers and storytellers, and we look forward to continuing these partnerships in this exciting space.”
Photo: The Holocinema team.
From its inception, a studio telling immersive stories with emerging technology platforms made ILMxLAB an appealing destination. Julie Peng, who worked in Lucasfilm Animation as a production manager for projects like Star Wars: The Clone Wars (2008-13) and Strange Magic (2015), was looking to break into the emerging interactive storytelling space when she received a call about a new ILM division that would focus on technology like augmented and virtual reality. “When we started, we were five people,” remembers Peng. “I did my best to take care of any need that arose, big and small, from developing the production infrastructure to writing job descriptions, to ordering pizza and running to the store for batteries. It was about doing whatever was needed to build a team and start exploring what we could bring to the immersive entertainment space.”
In 2016, the studio debuted its first VR experience, Star Wars: Trials on Tatooine, where the Millennium Falcon lands in front of players, and they help R2-D2 and Han Solo with repairs. This was an important step in the studio’s goal of creating a living world. “Everybody was just so blown away by the scale,” says Beck, “because that’s something that VR is so good at – delivering scope and scale.”
Looking to the future was also always a part of the plan. “Because we were so early in the whole immersive storytelling space, we really wanted to help drive the industry,” says Beck. “So we actually very consciously shared our prototypes in public. We spoke about them. We made them available to people because we wanted to actively inspire others to create in this space alongside us.”
Over time, the team evolved into a mix of creatives from the film industry and people with backgrounds in games and interactive development. While bringing in developers with both backgrounds was essential, it also brought challenges for Peng in her role as production manager. “Early on, I realized that they spoke two different dialects,” says Peng. “They used similar terminology, but their approaches to making a creative product were quite different in terms of process and priorities. I found myself becoming a bridge, translating concepts and driving the development of a common language so we could all communicate effectively.”
The team also had to be comfortable with fluidity as the technology they were working with was constantly evolving. Peng noted staying abreast of what was going on in the industry was key, as well as the leadership team being willing to take some risks. “I always call it ‘holding hands and jumping off the cliff together’.”
The First Big Leaps
Visual effects supervisor Tim Alexander became involved with ILMxLAB after a history in traditional visual effects, including the 2015 blockbuster Jurassic World. He was also a lifelong gamer intrigued by the work ADG was pioneering at the time: bringing real-time, game engine-type techniques into visual effects. When director Alejandro G. Iñárritu approached ILM about a collaboration on a virtual reality project, Alexander came aboard as visual effects supervisor. The result was CARNE y ARENA, which debuted in 2017.
Still early in ILMxLAB’s history, CARNE was an ambitious project involving a short VR piece bookended by physical experiential rooms that put the audience into a story of immigrants being detained while crossing the border from Mexico to the United States. At the beginning of the experience, participants are brought into a physical holding cell (where the temperature inside is cold) where they have to remove their shoes and items like backpacks. “There are ambient noises and real artifacts like abandoned shoes that have been found in the desert, from people crossing,” notes Alexander.
Photo: The key art for CARNE y ARENA.
Participants are fitted with a VR headset and led barefoot into a 50-foot-by-50-foot room full of sand. In the VR portion of the experience, they assume the role of a group of immigrants attempting to cross the U.S.-Mexico border at night when they are stopped by U.S. Border Patrol agents. After the VR story, participants exit and are led down a hallway where video monitors play interviews of the real people CARNE is based on. “He [Iñárritu] cast people that had crossed the border as the people within this experience and wove a story around that, so you actually see the real people and hear their experiences,” says Alexander.
CARNE was a challenge from an artistic and engineering standpoint. What Iñárritu and Alexander wanted to do was sometimes hindered by the current technology. Wanting the images to appear as photoreal as possible, the team realized the immersive film’s computing requirements outweighed what was possible in headsets at the time, so Lutz Latta, ADG graphics engineer, designed a supercomputer with four high-end GPUs (graphics processing unit) to handle work such as calculating shadows in the film.
Other challenges included allowing participants to traverse and turn around in a 50-foot-by-50-foot room. “At the time, there was no way to really run a VR headset over more than 100 feet. You were lucky if you could get five feet away because of the HDMI cables and all kinds of things,” remembers Alexander. VR tracking abilities at the time were also far below where ILM’s engineers wanted them to be. “So then we started mixing in stuff that we know from visual effects of how to track cameras in large spaces. A motion capture stage was built to track the headset instead of what we would usually track the camera in. So it started becoming a mixture of different things that we knew how to do for different reasons, and kind of applying it to this situation.”
Read the story in full and learn what the future holds for ILM Immersive at ILM.com.
The post ILM Evolutions: Pushing the Boundaries of Interactive Experiences appeared first on Jedi News.
