Behind every great movie, are hundreds of men and women toiling away to make sure you don’t actually know there are hundreds of men and women putting that movie together. And because of that, we get to focus on the actors and the director, while increasingly tuning out the effects... Or, arguably since audiences started to accept CGI characters as just part of the cast, in movies like Stuart Little or Lord of The Rings, we don’t really know how much of a movie is affected by Digital Effects, until something goes horribly wrong.
One of the main guys who makes sure those things never, ever go wrong is Jerome Chen, an Oscar nominee known for breaking new ground on fur effects on the aforementioned Stuart Little, and feather effects on Stuart Little 2. Now, the CGI master, who works at Sony Pictures Imageworks, is taking on the Amazing Spider-Man in his new feature length movie, and you’d be surprised just how much of the movie exists in real life. The answer? Not a whole lot.Adding to that challenge was, of course, shooting the movie in 3-D. “When I first started talking to [Director] Marc Webb about it,” said Chen, “We wanted to be able to capitalize on the stereo experience, really give the audience a much more physical sense of Spider-Man, and what it would be like to be swinging through the canyons of New York.”
The first problem Chen had to deal with was that Amazing Spider-Man was shot in native 3-D, meaning that the production used two cameras, instead of one and converting the footage to three dimensions in post. That meant designing stunts and sequences, all in 3-D, thinking of two sets of images at the same time. “For any given frame in the movie, there’s two images - which is automatically twice as much data,” said Chen with a laugh.
This adds a hefty amount of time to the prep process, because before an effects artists goes in to add, say, The Lizard, or the streets of New York, he first has to make sure the left eye frame, and the right eye frame actually match up. Not only that, but you have to make sure the color is the same: “You’re shooting on two different cameras, with two different lenses, and in fact, one of the images is going through a mirror because of way the cameras are set up relative to each other,” continued Chen.
Only when these images are matched up can you start tackling adding in the characters - and that, too, presents its own challenges. In 3-D, you need to think about depth, elements that come forward, and does the shot you’re working on work with the shot before and after it. “It comes down to not only does the shot work artistically, but does it work spatially,” said Chen.
Chen calls this an editorial concern, saying that it really comes down to making sure not only does the pacing work in a movie, but that the depth of an object stays spatially consistent from shot to shot; and if the editor’s job isn’t done properly, it will be extremely jarring for the audience.
And despite advances in technology, Chen says that making sure this all works still comes down to humans, rather than computer programs. “There’s a person called a stereographer, and his job is to make sure the technical side and the creative side work together,” said Chen. “It becomes a collaborative effort between everybody.”
Okay, so you’ve figured out how to shoot a scene, you’ve got your cameras set up the right way, and the 3-D coded properly. How about the characters? In Amazing Spider-Man, Chen was tasked with not only creating the entirely digital Lizard, but also digital Spider-Man, and even digital doubles for stars Andrew Garfield and Emma Stone! Interestingly Chen found it easier to deal with the Lizard, than with a digital version of of the stars - because for an audience member, you don’t know what the Lizard is supposed to look like, until the effects artists show you; for Garfield, you’ve been watching him all film, so if he looks different in an effects shot, he looks wrong.
To get around this, Chen and team took a digital scan of Garfield’s body for the movie, as well as a specific scan of his face. They used this to create a virtual version of Garfield, with specific emphasis on getting the proportions of his body, and facial features consistent throughout the film. But even with these great powers, come great responsibility: Chen believes that just because you CAN create a digital Garfield, doesn’t mean the actor can do whatever a spider can:
“You can have a perfect CG person, but if you design the shot in a way where the actual person could never do it, it doesn’t matter how good it looks,” said Chen. “You have to make sure that in the context of your design that the audience says, oh, they shot it, but they staged it in an elaborate way to get the shot.”
Beyond the people, though, Chen feels the biggest challenge the team faced was creating a nearly all digital New York City for the film. “You have Spider-Man swinging, and the camera choreography prevented us from shooting plates in New York City,” said Chen. “Where you do a plate of the move, and then Spider-Man placed in the CG later. So we had a whole couple of sequences where he’s swinging through the canyons of seventh avenue - that whole environment had to be CG.”
Chen continued, saying, “You had to create hundreds of thousands of thousands of room interiors, because the city is represented by windows at night, right? We studied footage of buildings in New York at night, and discovered that some windows are green, others are orange, there’s flickering televisions inside, and when you see down to the street you get a mix of all kinds of colors coming from store fronts, cabs, and reflections. That was extremely daunting, the amount of details you need to add to this city - not to mention people running around down there.”
Beyond the attention to detail, there’s the time: to render one frame of city footage for the film could take anywhere from forty to one hundred hours! Still, Chen thinks it paid off - and matches up with the spectacular footage of Spider-Man swinging (all digital too, of course).
So who are all the little digital people running around in digital New York City? Not Chen in a surprise cameo, though we did ask. Some of them are artists who worked on the film, but if you’re looking for where the effects guys inserted themselves in the movie, pay careful attention to Peter Parker’s High School. In a scene where Spider-Man fights the Lizard in school, the entire hallway needed to be CG generated. The class photos on this CG generated hallway? Pretty much all the movie’s digital artists.
Wrapping it up, Chen talked about how proud he was that most audience members probably won’t notice that the hallway scene changes from practical sets, to entirely CGI, or that the final battle with the Lizard is also an entirely CGI fight. “It’s not to tout, oh we can do this,” said Chen, “It’s creating this continuity of illusion so that people believe that it’s really Andrew Garfield in a suit swinging up there, and we were able to somehow photograph it, and maybe enhance it slightly - but not take you out of the movie so you think it’s all CG now. Making sure the feeling of the motion you’re seeing makes you believe... That’s Spider-Man.”
Amazing Spider-Man hits theaters on July 3rd!