Where's my Holodeck? The latest interactive movie news

IT IS time for cinema to take its next step. 3D technology now fills our screens with beautifully rendered characters and virtual environments, but we could have so much more.
So says Dennis Del Favero, director of what he calls the world's first 3D interactive film, Scenario. Rather than having audience members sit back and enjoy the action, the interactive narrative has them drive the story.
Undoubtedly, the ultimate synthetic interactive environment must be the virtual worlds generated by Star Trek's "Holodeck". To date, steps in this direction have been restricted because computer-generated characters cannot yet understand and speak in natural language. One solution is to sidestep the need for language and interact with audience members using physical markers, like movement.

In Scenario, which is loosely based on the life of Elisabeth Fritzl, the Austrian woman imprisoned by her father in a cellar for 24 years, audience members' movements around the cinema are tracked using 16 near-infrared cameras. Treading the line between a movie and a game, audience members are introduced to the plot, and assigned avatars and a mission: to collect virtual body parts and return them to an oversized baby. But they must work against artificially intelligent sentinel avatars, which use information from the cameras and the position of objects in the film's virtual world to plan their actions. For example, the sentinels can only move the baby's head if they are next to it, but might better achieve their objective by pushing an audience member's avatar.

Makers of interactive films can also hook into physiological reactions. Earlier this year, Unsound debuted in Austin, Texas. In this horror film, the visuals, music score and sound effects change depending on the heart rate and skin response of its collective audience members. For example, something horrific happens to the lead character, but in one version of the scene the audience can see it happening, while in another they can only hear it. "If the audience is highly reactive we will not show the graphic scene, and if the audience is bored to tears we would," says Ben Knapp of BioControl Systems, a technology firm based in San Francisco that collaborated on the film. According to Knapp, amalgamating the average emotional response of an audience overcomes any "noise" in the data - such as an audience member thinking "I need to pee."

Approaching the hurdle of language recognition, Marc Cavazza at Teesside University, UK, has created a computer system that detects the emotional content of speech. While it takes no notice of the user's words per se, it categorises speech according to a range of attributes in their voice including pitch, duration and pauses. Using this technology, you can enjoy a conversation of sorts with a virtual character from Gustave Flaubert's Madame Bovary. The character's responses depend on predispositions based on their personality in the novel.

The Holodeck is still some way off, then. According to Cavazza, for complex plots to be driven by the audience, virtual characters must understand words.
Even once this is achieved, another issue appears: confining the narrative. Computer-memory constraints mean that an interactive movie cannot allow for unlimited plot choices. One work-around is to build a movie like a choose-your-own-adventure book, where audiences only influence the plot at specific points. But Michael Mateas at the University of California, Santa Cruz, reckons it would be more interesting if interactive films are developed as a virtual world, where the plot is directed by characters and their relationships.
Along these lines, Mateas has created Prom Week, which follows a fictional group of high school students in the week before their final dance. To be released on Facebook in August, Prom Week will immerse the player in hallway politics as they dictate the future of the virtual students through a range of options. Each interaction between characters is recorded into a database, the sum of these interactions driving the plot's direction by evolving the characters' sentiments towards each other. "We want it to feel like these characters are alive," says Mateas.

Source New Scientist