Tag Archives: The Man in the High Castle

VFX in Series: The Man in the High Castle, Westworld

By Karen Moltenbrey

The look of television changed forever starting in the 1990s as computer graphics technology began to mature to the point where it could be incorporated within television productions. Indeed, the applications initially were minor, but soon audiences were witnessing very complicated work on the small screen. Today, we see a wide range of visual effects being used in television series, from minor wire and sign removal to all-CG characters and complete CG environments — pretty much anything and everything to augment the action and story, or to turn a soundstage or location into a specific locale that could be miles away or even non-existent.

Here, we examine two prime examples where a wide range of visual effects are used to set the stage and propel the action for a pair of series with very unique settings. For instance, The Man in the High Castle uses effects to turn back the clock to the 1960s, but also to create an alternate reality for the period, turning the familiar on its head. In  Westworld, effects create a unique Wild West of the future. In both series, VFX also help turn up the volume on these series’ very creative storylines.

The Man in the High Castle

What would life in the US be like if the Axis powers had defeated the Allied forces during World War II? The Amazon TV series The Man in the High Castle explores that alternate history scenario. Created by Frank Spotnitz and produced by Amazon Studios, Scott Free Productions, Headline Pictures, Electric Shepherd Productions and Big Light Productions, the series is scheduled to start its fourth and final season in mid-November. The story is based on the book by Philip K. Dick.

High Castle begins in the early 1960s in a dystopian America. Nazi Germany and the Empire of Japan have divvied up the US as their spoils of war. Germany rules the East, known as the Greater Nazi Reich (with New York City as the regional capital), while Japan controls the West, known as the Japanese Pacific States (whose capital is now San Francisco). The Rocky Mountains serve as the Neutral Zone. The American Resistance works to thwart the occupiers, spurred on after the discovery of materials displaying an alternate reality where the Allies were victorious, making them ponder this scenario.

With this unique storyline, visual effects artists were tasked with turning back the clock on present-day locations to the ’60s and then turning them into German- and Japanese-dominated and inspired environments. Starting with Season 2, the main studio filling this role has been Barnstorm Visual Effects (Los Angeles, Vancouver). Barnstorm operated as one of the vendors for Season 1, but has since ramped up its crew from a dozen to around 70 to take on the additional work. (Barnstorm also works on CBS All Access shows such as The Good Fight and Strange Angel, in addition to Get Shorty, Outlander and the HBO series Room 104 and Silicon Valley.)

According to Barnstorm co-owner and VFX supervisor Lawson Deming, the studio is responsible for all types of effects for the series — ranging from simple cleanup and fixes such as removing modern objects from shots to more extensive period work through the addition of period set pieces and set extensions. In addition, there are some flashback scenes that call for the artists to digitally de-age the actors and lots of military vehicles to add, as well as science-fiction objects. The majority of the overall work entails CG set extensions and world creation, Deming explains, “That involves matte paintings and CG vehicles and buildings.”

The number of visual effects shots per episode also varies greatly, depending on the story line; there are an average of 60 VFX shots an episode, with each season encompassing 10 episodes. Currently the team is working on Season 4. A core group of eight to 10 CG artists and 12 to 18 compositors work on the show at any given time.

For Season 3, released last October, there are a number of scenes that take place in the Reich-occupied New York City. Although it was possible to go to NYC and photograph buildings for reference, the city has changed significantly since the 1960s, “even notwithstanding the fact that this is an alternate history 1960s,” says Deming. “There would have been a lot of work required to remove modern-day elements from shots, particularly at the street level of buildings where modern-day shops are located, even if it was a building from the 1940s, ’50s or ’60s. The whole main floor would have needed replaced.”

So, in many cases, the team found it more prudent to create set extensions for NYC from scratch. The artists created sections of Fifth and Sixth avenues, both for the area where American-born Reichmarshall and Resistance investigator John Smith has his apartment and also for a parade sequence that occurs in the middle of Season 3. They also constructed a digital version of Central Park for that sequence, which involved crafting a lot of modular buildings with mix-and-match pieces and stories to make what looked like a wide variety of different period-accurate buildings, with matte paintings for the backgrounds. Elements such as fire escapes and various types of windows (some with curtains open, some closed) helped randomize the structures. Shaders for brick, stucco, wood and so forth further enabled the artists to get a lot of usage from relatively few assets.

“That was a large undertaking, particularly because in a lot of those scenes, we also had crowd duplication, crowd systems, tiling and so on to create everything that was there,” Deming explains. “So even though it’s just a city and there’s nothing necessarily fantastical about it, it was almost fully created digitally.”

The styles of NYC and San Francisco are very different in the series narrative. The Nazis are rebuilding NYC in their own image, so there is a lot of influence from brutalist architecture, and cranes often dot the skyline to emphasize all the construction taking place. Meanwhile, San Francisco has more of a 1940s look, as the Japanese are less interested in influencing architectural changes as they are in occupation.

“We weren’t trying to create a science-fiction world because we wanted to be sure that what was there would be believable and sell the realistic feel of the story. So, we didn’t want to go too far in what we created. We wanted it to feel familiar enough, though, that you could believe this was really happening,” says Deming.

One of the standout episodes for visual effects is “Jahr Null” (Season 3, Episode 10), which has been nominated for a 2019 Emmy in the Outstanding Special Visual Effects category. It entails the destruction of the Statue of Liberty, which crashes into the water, requiring just about every tool available at Barnstorm. “Prior to [the upcoming] Season 4, our biggest technical challenge was the Statue of Liberty destruction. There were just so many moving parts, literally and figuratively,” says Deming. “So many things had to occur in the narrative – the Nazis had this sense of showmanship, so they filmed their events and there was this constant stream of propaganda and publicity they had created.”

There are ferries with people on them to watch the event, spotlights are on the statue and an air show with music prior to the destruction as planes with trails of colored smoke fly toward the statue. When the planes fire their missiles at the base of the statue, it’s for show, as there are a number of explosives planted in the base of the statue that go off in a ring formation to force the collapse. Deming explains the logistics challenge: “We wanted the statue’s torch arm to break off and sink in the water, but the statue sits too far back. We had to manufacture a way for the statue to not just tip over, but to sort of slide down the rubble of the base so it would be close enough to the edge and the arm would snap off against the side of the island.”

The destruction simulation, including the explosions, fire, water and so forth, was handled primarily in Side Effects Houdini. Because there was so much sim work, a good deal of the effects work for the entire sequence was done in Houdini as well. Lighting and rendering for the scene was done within Autodesk’s Arnold.

Barnstorm also used Blender, an open-source 3D program for modeling and asset creation, for a small portion of the assets in this sequence. In addition, the artists used Houdini Mantra for the water rendering, while textures and shaders were built in Adobe’s Substance Painter; later the team used Foundry’s Nuke to composite the imagery. “There was a lot of deep compositing involved in that scene because we had to have the lighting interact in three dimensions with things like the smoke simulation,” says Deming. “We had a bunch of simulations stacked on top of one another that created a lot of data to work with.”

The artists referenced historical photographs as they designed and built the statue with a period-accurate torch. In the wide aerial shots, the team used some stock footage of the statue with New York City in the background, but had to replace pretty much everything in the shot, shortening the city buildings and replacing Liberty Island, the water surrounding it and the vessels in the water. “So yeah, it ended up being a fully digital model throughout the sequence,” says Deming.

Deming cannot discuss the effects work coming up in Season 4, but he does note that Season 3 contained a lot of digital NYC. This included a sequence wherein John Smith was installed as the Reichmarshall near Central Park, a scene that comprised a digital NYC and digital crowd duplication. On the other side of the country, the team built digital versions of all the ships in San Francisco harbor, including CG builds of period Japanese battleships retrofitted with more modern equipment. Water simulations rounded out the scene.

In another sequence, the Japanese performed nuclear testing in Monument Valley, blowing the caps off the mesas. For that, the artists used reference photos to build the landscape and then created a digital simulation of a nuclear blast.

In addition, there were a multitude of banners on the various buildings. Because of the provocative nature of some of the Nazi flags and Fascist propaganda, solid-color banners were often hung on location, with artists adding the offensive VFX image in post as to not upset locals where the series was filmed. Other times, the VFX artists added all-digital signage to the scenes.

As Deming points out, there is only so much that can be created through production design and costumes. Some of the big things have to be done with visual effects. “There are large world events in the show that happen and large settings that we’re not able to re-create any other way. So, the visual effects are integral to the process of creating the aesthetic world of the show,” he adds. “We’re creating things that while they are visually impressive, also feel authentic, like a world that could really exist. That’s where the power and the horror of the world here comes from.”

High Castle is up for a total of three Emmy awards later this month. It was nominated for three Emmys in 2017 for Season 2 and four in 2016 for Season 1, taking home two Emmys that year: one for Outstanding Cinematography for a Single-Camera Series and another for Outstanding Title Design.

Westworld

What happens when high tech meets the Wild West, and wealthy patrons can indulge their fantasies with no limits? That is the premise of the Emmy-winning HBO series Westworld from creators Jonathan Nolan and Lisa Joy, who executive produce along with J.J. Abrams, Athena Wickham, Richard J. Lewis, Ben Stephenson and Denise Thé.

Westworld is set in the fictitious western theme park called Westworld, one of multiple parks where advanced technology enables the use of lifelike android hosts to cater to the whims of guests who are able to pay for such services — all without repercussions, as the hosts are programmed not to retaliate or harm the guests. After each role-play cycle, the host’s memory is erased, and then the cycle begins anew until eventually the host is either decommissioned or used in a different narrative. Staffers are situated out of sight while overseeing park operations and performing repairs on the hosts as necessary. As you can imagine, guests often play out the darkest of desires. So, what happens if some of the hosts retain their memories and begin to develop emotions? What if some escape from the park? What occurs in the other themed parks?

The series debuted in October 2016, with Season 2 running from April through June of 2018. The production for Season 3 began this past spring and it is planned for release in 2020.

The first two seasons were shot in various locations in California, as well as in Castle Valley near Moab, Utah. Multiple vendors provide the visual effects, including the team at CoSA VFX (North Hollywood, Vancouver and Atlanta), which has been with the show since the pilot, working closely with Westworld VFX supervisor Jay Worth. CoSA worked with Worth in the past on other series, including Fringe, Undercovers and Person of Interest.

The number of VFX shots per episode varies, depending on the storyline, and that means the number of shots CoSA is responsible for varies widely as well. For instance, the facility did approximately 360 shots for Season 1 and more than 200 for Season 2. The studio is unable to discuss its work at this time on the upcoming Season 3.

The type of effects work CoSA has done on Westworld varies as well, ranging from concept art through the concept department and extension work through the studio’s environments department. “Our CG team is quite large, so we handle every task from modeling and texturing to rigging, animation and effects,” says Laura Barbera, head of 3D at CoSA. “We’ve created some seamless digital doubles for the show that even I forget are CG! We’ve done crowd duplication, for which we did a fun shoot where we dressed up in period costumes. Our 2D department is also sizable, and they do everything from roto, to comp and creative 2D solutions, to difficult greenscreen elements. We even have a graphics department that did some wonderful shots for Season 2, including holograms and custom interfaces.”

On the 3D side, the studio’s pipeline js mainly comprised of Autodesk’s Maya and Side Effects Houdini, along with Adobe’s Substance, Foundry’s Mari and Pixologic’s ZBrush. Maxon’s Cinema 4D and Interactive Data Visualization’s SpeedTree vegetation modeler are also used. On the 2D side, the artists employ Foundry’s Nuke and the Adobe suite, including After Effects and Photoshop; rendering is done in Chaos Group’s V-Ray and Redshift’s renderer.

Of course, there have been some recurring effects each season, such as the host “twitches and glitches.” And while some of the same locations have been revisited, the CoSA artists have had to modify the environments to fit with the changing timeline of the story.

“Every season sees us getting more and more into the characters and their stories, so it’s been important for us to develop along with it. We’ve had to make our worlds more immersive so that we are feeling out the new and changing surroundings just like the characters are,” Barbera explains. “So the set work gets more complex and the realism gets even more heightened, ensuring that our VFX become even more seamless.”

At center stage have been the park locations, which are rooted in existing terrain, as there is a good deal of location shooting for the series. The challenge for CoSA then becomes how to enhance it and make nature seem even more full and impressive, while still subtly hinting toward the changes in the story, says Barbera. For instance, the studio did a significant amount of work to the Skirball Cultural Center locale in LA for the outdoor environment of Delos, which owns and operates the parks. “It’s now sitting atop a tall mesa instead of overlooking the 405!” she notes. The team also added elements to the abandoned Hawthorne Plaza mall to depict the sublevels of the Delos complex. They’re constantly creating and extending the environments in locations inside and out of the park, including the town of Pariah, a particularly lawless area.

“We’ve created beautiful additions to the outdoor sets. I feel sometimes like we’re looking at a John Ford film, where you don’t realize how important the world around you is to the feel of the story,” Barbera says.

CoSA has done significant interior work too, creating spaces that did not exist on set “but that you’d never know weren’t there unless you’d see the before and afters,” Barbera says. “It’s really very visually impressive — from futuristic set extensions, cars and [Westworld park co-creator] Arnold’s house in Season 2, it’s amazing how much we’ve done to extend the environments to make the world seem even bigger than it is on location.”

One of the larger challenges in the first seasons came in Season 2: creating the Delos complex and the final episodes where the studio had to build a world inside of a world – the Sublime –as well as the gateway to get there. “Creating the Sublime was a challenge because we had to reuse and yet completely change existing footage to design a new environment,” explains Barbera. “We had to find out what kind of trees and foliage would live in that environment, and then figure out how to populate it with hosts that were never in the original footage. This was another sequence where we had to get particularly creative about how to put all the elements together to make it believable.”

In the final episode of the second season, the group created environment work on the hills, pinnacles and quarry where the door to the Sublime appears. They also did an extensive rebuild of the Sublime environment, where the hosts emerge after crossing over. “In the first season, we did a great deal of work on the plateau side of Delos, as well as adding mesas into the background of other shots — where [hosts] Dolores and Teddy are — to make the multiple environments feel connected,” adds Barbera.

Aside from the environments, CoSA also did some subtle work on the robots, especially in Season 2, to make them appear as if they were becoming unhinged, hinting at a malfunction. The comp department also added eye twitches, subtle facial tics and even rapid blinks to provide a sense of uneasiness.

While Westworld’s blending of the Old West’s past and the robotic future initially may seem at thematic odds, the balance of that duality is cleverly accomplished in the filming of the series and the way it is performed, Barbera points out. “Jay Worth has a great vision for the integrated feel of the show. He established the looks for everything,” she adds.

The balance of the visual effects is equally important because it enhances the viewer experience. “There are things happening that can be so subtle but have so much impact. Much of our work on the second season was making sure that the world stayed grounded, so that the strangeness that happened with the characters and story line read as realistic,” Barbera explains. “Our job as visual effects artists is to help our professional storytelling partners tell their tales by adding details and elements that are too difficult or fantastic to accomplish live on set in the midst of production. If we’re doing our job right, you shouldn’t feel suddenly taken out of the moment because of a splashy effect. The visuals are there to supplement the story.”


Karen Moltenbrey is a veteran writer/editor covering VFX and post production.

DP Chat: The Man in the High Castle’s Gonzalo Amat

By Randi Altman

Amazon’s The Man in the High Castle is based on the 1962 Phillip K. Dick novel, which asks the question: “What would it look like if the Germans and Japanese won World War II?” It takes a look at the Nazi and Japanese occupation of portions of the United States and the world. But it’s a Philip K. Dick story, so you know there is more to it than that… like an alternate reality.

The series will premiere its fourth and final season this fall on the streaming service. We recently reached out to cinematographer Gonzalo Amat, who was kind enough to talk to us about workflow and more.

How did you become interested in cinematography?
Since I was very young, I had a strong interest in photography and was shooting stills as long as I can remember. Then, when I was maybe 10 or 12 years old, I discovered that movies also had a photographic aspect. I didn’t think about doing it until I was already in college studying communications, and that is when I decided to make it my career.

What inspires you artistically? And how do you simultaneously stay on top of advancing technology?
Artistically, I get inspiration from a lot of sources, such as photography, film, literature, painting or any visual medium. I try to curate what I consume, though. I believe that everything we feed our brain somehow shows up in the work we do, so I am very careful about consuming films, books and photography that feed the story that I will be working on. I think any creation is inspiration. It can be all the way from a film masterpiece to a picture drawn by a kid, music, performance art, historical photographs or testimonies, too.

About staying on top: I read trade magazines and stay educated through seminars and courses, but at some point, it’s also about using those tools. So I try to test the tools instead of reading about them. Almost any rental place or equipment company will let you try newer tools. If I’m shooting, we try to schedule a test for a particular piece of equipment we want to use, during a light day.

What new technology has changed the way you work?
The main new technology would be the migration of most projects to digital. That has changed the way we work on set and collaborate with the directors, since everyone can now see, on monitors, something closely resembling the final look of the project.

A lot of people think this is a bad thing that has happened, but for me, it actually allows more clear communication about the concrete aspects of a sometimes very personal vision. Terms like dark, bright, or colorful are very subjective, so having a reference is a good point to continue the conversation.

Also, digital technology has helped use more available light on interiors and use less light on exterior nights. Still, it hasn’t reached the latitude of film, where you could just let the windows burn. It’s trickier for exterior day shots, where I think you end up needing more control. I would also say that the evolution of visual effects as a more invisible tool has helped us achieve a lot more from a storytelling perspective and has affected the way we shoot scenes in general.

What are some of your best practices, or rules you try to follow on each job?
Each project is different, so I try to learn how that particular project will be. But there are some time-tested rules that I try to implement. The main line is to always go for the story; every answer is always in the script. Another main rule is communication. So being open about questions, even if they seem silly. It’s always good to ask.

Another rule is listening to ideas. People that end up being part of my team are very experienced and sometimes have solutions to problems that come up. If you are open to ideas, more ideas will come, and people will do their jobs with more intention and commitment. Gratitude, respect, collaboration, communication and being conscious about safety is important and part of my process.

Gonzalo Amat on set

Explain your ideal collaboration with the director when setting the look of a project.
Every director is different, so I look at each new project as an opportunity to learn. As a DP, you have to learn and adapt, since through your career you will be asked for different levels of involvement. Because of my interest in storytelling, I personally prefer a bit more of a hands-off approach from directors; talking more about story and concepts, where we collaborate setting up the shoots for covering a scene, and same with lighting: talking moods and concepts that get polished as we are on set. Some directors will be very specific, and that is a challenge because you have to deliver what is inside their heads and hopefully make it better. I still enjoy this challenge, because it also makes you work for someone’s vision.

Ideally, developing the look of a project comes from reading the script together and watching movies and references together. This is when you can say “dark like this” or “moody like this” because visual concepts are very subjective, and so is color. From then on, it’s all about breaking up the script and the visual tone and arc of the story, and subsequently all the equipment and tools for executing the ideas. Lots of meetings as well as walking the locations with just the director and DP are very useful.

How would you describe the overarching look of the show?
Basically, the main visual concept of this project is based in film noir, and our main references were The Conformist and Blade Runner. As we went along, we added some more character-based visual ideas inspired by projects like In the Mood for Love and The Insider for framing.

The main idea is to visually portray the worlds of the characters through framing and lighting. Sometimes, we play it the way the script tells us; sometimes we counterpoint visually what it says, so we can make the audience respond in an emotional way. I see cinematography as the visual music that makes people respond emotionally to different moods. Sometimes it’s more subtle and sometimes more obvious. We prefer to not be very intrusive, even though it’s not a “realist” project.

How early did you get involved in the production?
I start four or five weeks before the season. Even if I’m not doing the first episode, I will still be there to prepare new sets and do some tests for new equipment or characters. Preparation is key in a project like this, because once we start with the production the time is very limited.

Did you start out on the pilot? Did the look change from season to season at all?
James Hawkinson did the pilot, and I came in when the series got picked up. He set up the main visual concepts, and when it came to series I adapted some of the requirements from the studio and the notes from Ridley Scott into the style we see now.

The look has been evolving from season to season, as we feel we can be bolder with the visual language of the show. If you look at the pilot all the way to the end of Season 3, or Season 4, which is filming, you can definitely see a change, even though it still feels like the same project — the language has been polished and distilled. I think we have reached the sweet spot.

Does the look change at all when the timelines shift?
Yes, all of the timelines require a different look and approach with lighting and camera use. Also, the art design and wardrobe changes, so we combine all those subtle changes to give each world, place and timeline a different feel. We have lots of conceptual meetings, and we develop the look and feel of each timeline and place. Once these concepts are established, the team gets to work constructing the sets and needed visual elements, and then we go from there.

This is a period piece. How did that affect the look, if at all?
We have tried to give it a specific and unique look that still feels tied to the time period so, yes, the fact that this happens in our own version of the ‘60s has determined the look, feeling and language of the series. We base our aesthetics in what the real world was in 1945, which our story diverges from to form this alternate world.

The 1960s of the story are not the real 1960s because there is no USA and no free Europe, so that means most of the music and wardrobe doesn’t look like the 1960s we know. There are many Nazi and Japanese visual elements on the visuals that distinguish us from a regular 1960s look, but it still feels period.

How did you go about choosing the right camera and lenses for this project?
Because we had a studio mandate to finish in 4K, the Red One with Zeiss Master Prime lenses was chosen in the pilot, so when I came on we inherited that tech. We stuck with all this for the first season, but after a few months of shooting we adapted the list and filters and lighting. On Season 2, we pushed to change to an ARRI Alexa camera, so we ended up adjusting all the equipment around this new camera and it’s characteristics — such as needing less light, so we ended up with less lighting equipment.

We also added classic Mitchell Diffusion Filters and some zooms. Lighting and grip equipment have been evolving toward less and less equipment since we light less and less. It’s a constant evolution. We also looked at some different lens options in the season breaks, but we haven’t added them because we don’t want to change our budget too much from season to season, and we use them as required.

Any challenging scenes that you are particularly proud of in Season 3?
I think the most challenging scene was the one in the Nebenwelt tunnel set. We had to have numerous meetings about what this tunnel was as a concept and then, based on the concept, find a way to execute it in a visual way. We wanted to make sure that the look of the scene matched the concepts of quantum physics within the story.

I wanted to achieve lighting that felt almost like plasma. We decided to put a mirror at the end of the tunnel with circle lighting right above it. We then created the effect of the space travel by using a blast of light — using lighting strikes with an elaborate setup that collectively used more than a million watts. It was a complex setup, but fortunately we had a lot of very talented people come together to execute it.

What’s your go-to gear (camera, lens, mount/accessories) — things you can’t live without?
On this project, I’d say it’s the 40mm lens. I don’t think this project would have the same vibe without this lens. Then, of course, I love the Technocrane, but we don’t use it every day, for budgetary and logistical reasons.

For other projects, I would say the ARRI Alexa camera and the 40mm and handheld accessories. You can do a whole movie with just those two; I have done it, and it’s liberating. But if I had an unlimited budget, I would love to use a Technocrane every day with a stabilized remote head.


Randi Altman is the founder and editor-in-chief of postPerspective. She has been covering production and post production for more than 20 years. 

Creating new worlds for Amazon’s The Man in the High Castle

Zoic Studios used visual effects to recreate occupied New York and San Francisco.

By Randi Altman

What if Germany and Japan had won World War II? What would the world look like? That is the premise of Philip K. Dick’s 1963 novel and Amazon’s series, The Man in the High Castle, which is currently gearing up for its second season premiere in December.

The Man in the High Castle features familiar landmarks with unfamiliar touches. For example, New York’s Time Square has its typical billboards, but sprinkled in are giant swastika banners, images of Hitler and a bizarro American flag, whose blue stripes have been replaced with yet another swastika. San Francisco, and the entire West Coast, is now under Japanese rule, complete with Japanese architecture and cultural influences. It’s actually quite chilling.

Jeff

Jeff Baksinski

Helping to create these “new” worlds was Zoic Studios, whose team received one of the show’s four Emmy nods for its visual effects work. That team was led by visual effects supervisor Jeff Baksinski.

Zoic’s path to getting the VFX gig was a bit different than most. Instead of waiting to be called for a bid, they got aggressive… in the best possible way. “Both myself and another supervisor here, Todd Shifflett, had read Dick’s book, and we really wanted this project.”

They began with some concept stills and bouncing ideas off each other of what a German-occupied New York would look like. One of Zoic’s producers saw what they were up to and helped secure some money for a real test. “Todd found a bunch of late ‘50s/early 60’s airline commercials about traveling to New York, and strung it together as one piece. Then we added various Nazi banners, swastikas and statues. Our commercial features a pullback from a 1960s-era TV. Then we pull back to reveal a New York penthouse with a Nazi solider standing at the window. The commercial’s very static-y and beat up, but as we pull back out the window, we have a very high-resolution version of Nazi New York.”

And that, my friends, is how they got the show. Let’s find out more from Baksinski…

The Man in the High Castle is an Amazon show. Does the workflow differ from traditional broadcast shows?
Yes. For example, on our network TV shows, typically you’ll get a script each week, you’ll break it down and maybe have 10 days worth of post to execute the visual effects. Amazon and Netflix shows are different. They have a good idea of where their season is going, so you can start building assets well in advance.

High Castle’s version of the Brooklyn Bridge features a Nazi/American flag.

When we did the pilot, we were already building assets while I was going out to set. We were building San Francisco’s Hirohito Airport, the airplane that featured heavily in a few episodes and the cities of New York and San Francisco — a lot of that started before we ever shot a single frame.

It’s a whole new world with the streaming channels.
Everybody does it a little bit differently. Right now when we work on Netflix shows, we are working in shooting order, episode 105, 106, 107, etc., but we have the flexibility to say, “Okay, that one’s going to push longer because it’s more effects-heavy. We’re going to need four weeks on that episode and only two on this other one.” It’s very different than normal episodic TV.

Do you have a preference?
At the moment, my preference is for the online shows. I come from a features background where we had much longer schedules. Even worse, I worked in the days where movies had a year-and-a-half worth of schedule to do their visual effects. That was a different era. When I came into television, I had never seen anything this fast in my life. TV has a super quick turnaround, and obviously audiences have gotten smarter and smarter and want better and better work; television is definitely pushing closer to a features-type look.

Assuming you get more time with the pilots?
I love pilots. You get a big chunk of the story going, and a longer post schedule — six to eight weeks. We had about six weeks on Man in the High Castle, which is a good amount of time to ask, “What does this world look like, and what do they expect? In the case of High Castle, it was really about building a world. We were never going to create a giant robot. It was about how do we do make the world interesting and support our actors and story? You need time to do that.

You were creating a world that doesn’t exist, but also a period piece that takes place in the early ‘60s. Can you talk about that?
We started with what the normal versions of New York and San Francisco looked like in the ‘60s. We did a lot of sketch work, some simple modeling and planning. The next step was what would New York look like if Germany had taken over, and how would San Francisco be different under the influence of Japan?

Zoic added a Japanese feel to San Francisco streets and buildings.

In the case of San Francisco, we found areas in other countries that have heavy Japanese populations and how they influence the architecture —so buildings that were initially built by somebody else and then altered for a Japanese feel. We used a lot of that for what you see in the San Francisco shots.

What about New York?
That was a little bit tougher, because if you’re going to reference back to Germany during the war, you have propaganda signs, but our story takes place in 1962, so you’ve got some 17 years there where the world has gotten used to this German and Nazi influence. So while those signs do exist, we scaled back and added normal signs with German names.

In terms of the architecture, we took some buildings down and put new ones in place. You’ll notice that in our Times Square, traffic doesn’t move as it does in real life. We altered the roads to show how traffic would move if somebody eliminated some buildings and put cross-traffic in.

You also added a little bit of German efficiency to some scenes?
Absolutely. It’s funny… in the show’s New York there are long lines of people waiting to get into various restaurants and stores, and it’s all very regimented and controlled. Compare that to San Francisco where we have people milling about everywhere and it’s overcrowded with a lot of randomness.

How much of what you guys did were matte paintings, and could those be reused?
We use a few different types of matte paintings. We have the Rocky Mountains, for example, in the Neutral Zone. Those are a series of matte paintings we did from different angles that show mountains, trees and rivers. That is reusable for the most part.

Other matte paintings are very specific. For example, in the alley outside of Frank’s apartment, you see clothes hanging out to dry, and buildings all the way down the alleyway that lead to this very crowded-looking city. Those matte paintings are shot-specific.

Then we use matte paintings to show things far off in the distance to cut off the CG. Our New York is maybe four square city blocks around in every direction. When we get down to that fourth block, we started using old film tricks — what they used to do on studio lots, where you start curving the roads, dead-ending, or pinching the roads together. There is no way we could build 30 blocks of CG in every direction. I just can’t get there, so we started curving the CG and doing little tricks so the viewer can’t tell the difference.

What was the most challenging type of effects you created for the show? Which shots are you most proud of?
We are most proud of the Hirohito Airport and the V9 rocket plane. What most people don’t realize is that there’s actually nothing there — we weren’t at a real airport and there’s no plane for the actors to interact with. The actors are literally standing on a giant set of grip pipe and crates and walking down a set of stairs. That plane looks very realistic, even super close-up. You see every bolt and hinge and everything as the actors walk out. The monorail and embassy are also cool.

What do you call on in terms of tools?
We use Maya for modeling and lighting environments and for any animation work, such as a plane flying or the cars driving. There is a plug-in for Maya called Miarmy that we used to create CG people walking around in backgrounds. Some of those shots have hundreds of extras, but it still felt a little bit thin, so we were used CG people to fill in the gaps.

What about compositing?
It’s all Nuke. A lot of our environments are combinations of Photoshop and Nuke or projections onto geometry. Nuke will actually let you use geometry and projections in 3D inside of the compositing package, so some of our compositors are doing environment work as well.

Did you do a lot of greenscreen work?
We didn’t do any on the pilot, but did on the following episodes. We decided to go all roto on the pilot because the show has such a unique lighting set-up — the way the DP wanted to light that show — that green would have completely ruined it. This is abnormal for visual effects, where everyone’s always greenscreening.

street-before-nyRoto is such a painstaking process.
Absolutely. Our DP Jim Hawkinson was coming off of Hannibal at the time. DPs are always super wary of visual effects supervisors because when you come on the set you’re immediately the enemy; you’re about to tell them how to screw up all their lighting (smiles).

He said very clearly, “This is how I like to use light, and these are the paintings and the artwork.” This is the stuff I really enjoy. Between talking to him and director David Semel, and knowing that it was an RSA project, your brain immediately starts going to things like Blade Runner. You’re just listening to the conversations. It’s like, “Oh, this is not straightforward. They’re going to have a very contrast-y, smoky look to this show.”

We did use greenscreen on the rest of the episodes because we had less time. So out of necessity we used green.

What about rendering?
We use V-Ray, which is a global illumination renderer. We’d go out and take HDR images of the entire area for lighting and capture all of the DPs lights — that’s what’s most important to me. The DP set up his lights for a reason. I want to capture as much of his lighting as humanly possible so when I need to add a building or car into the shot, I’m using his lighting to light it.

It’s a starting point because you usually build a little bit on top of that, but that’s typically what we do. We get our HDRs, we bring them into Maya, we light the scene inside of Maya, then we render through V-Ray, and it all gets composited together.