Tag Archives: Mike McCarthy

Nvidia, AMD and Intel news from Computex

By Mike McCarthy

A number of new technologies and products were just announced at this year’s Computex event in Taipei, Taiwan. Let’s take a look at ones that seem relevant to media creation pros.

Nvidia released a line of mobile workstation GPUs based on its newest Turing architecture. Like the GeForce lineup, the Turing line has versions without the RTX designation. The Quadro RTX 5000, 4000 and 3000 have raytracing and Tensor cores, while the Quadro T2000 and T1000 do not, similar to the GeForce 16 products. The RTX 5000 matches the desktop version, with slightly more CUDA cores than the GeForce RTX 2080, although at lower clock speeds for reduced power consumption.

Nvidia’s new RTX 5000

The new Quadro RTX 3000 has similar core configuration to the desktop Quadro RTX 4000 and GeForce RTX 2070. This leaves the new RTX 4000 somewhere in between, with more cores than the desktop variant, aiming to provide similar overall performance at lower clock speeds and power consumption. While I can respect the attempt to offer similar performance at given tiers, doing so makes it more complicated than just leaving consistent naming for particular core configurations.

Nvidia also announced a new “RTX Studio” certification program for laptops targeted at content creators. These laptops are designed to support content creation applications with “desktop-like” performance. RTX Studio laptops will include an RTX GPU (either GeForce or Quadro), an H-Series or better Intel CPU, at least 16GB RAM and 512GB SSD, and at least a 1080p screen. Nvidia also announced a new line of studio drivers that are supposed to work with both Quadro and GeForce hardware. They are optimized for content creators and tested for stability with applications from Adobe, Autodesk, Avid, and others. Hopefully these drivers will simplify certain external GPU configurations that mix Quadro and GeForce hardware. It is unclear whether or not these new “Studio” drivers will replace the previously announced “Creator Ready” series of drivers.

Intel announced a new variant of its top end 9900K CPU. The i9-9900KS has a similar configuration, but runs at higher clock speeds on more cores, with a 4GHz base frequency and allowing 5GHz boost speeds on all eight cores. Intel also offered more details on its upcoming 10nm Ice Lake products with Gen 11 integrated graphics, which offers numerous performance improvements and VNNI support to accelerate AI processing. Intel is also integrating support for Thunderbolt 3 and Wi-Fi 6 into the new chipsets, which should lead to wider support for those interfaces. The first 10nm products to be released will be the lower-power chip for tablets and ultra portable laptops with higher power variants coming further in the future.

AMD took the opportunity to release new generations of both CPUs and GPUs. On the CPU front, AMD has a number of new third-generation 7nm Ryzen processors, with six to 12 cores in the 4GHz range and supporting 20 lanes of fourth-gen PCIe. Priced between $200 and $500, they are targeted at consumers and gamers and are slated to be available July 7th. These CPUs compete with Intel’s 9900K and similar CPUs, which have been offering top performance for Premiere and After Effects users due to their high clock speed. It will be interesting to see if AMD’s new products offer competitive performance at that price point.

AMD also finally publicly released its Navi generation GPU architecture, in the form of the new Radeon 5700. The 5000 series has an entirely new core design, which they call Radeon DNA (RDNA) to replace the GCN architecture first released seven years ago. RDNA is supposed to offer 25% more performance per clock cycle and 50% more performance per watt. This is important, because power consumption was AMD’s weak point compared to competing products from Nvidia.

AMD president and CEO Dr. Lisa Su giving her keynote.

While GPU power consumption isn’t as big of a deal for gamers using it a couple hours a day, commercial compute tasks that run 24/7 see significant increases in operating costs for electricity and cooling when power consumption is higher. AMD’s newest Radeon 5700 is advertised to compete performance-wise with the GeForce RTX 2070, meaning that Nvidia still holds the overall performance crown for the foreseeable future. But the new competition should drive down prices in the mid-range performance segment, which are the cards most video editors need.


Mike McCarthy is an online editor/workflow consultant with over 10 years of experience on feature films and commercials. He has been involved in pioneering new solutions for tapeless workflows, DSLR filmmaking and multi-screen and surround video experiences. Check out his site.

Review: Razer Blade 15-inch mobile workstation

By Mike McCarthy

I am always looking for the most powerful tools in the smallest packages, so I decided to check out the Razer Blade 15-inch laptop with an Nvidia GeForce RTX 2080 Max-Q graphics card. The Max-Q variants are optimized for better thermals and power usage — at the potential expense of performance — in order to allow more powerful GPUs to be used in smaller laptops. The RTX 2080 is Nvidia’s top-end mobile GPU, with 2,944 CUDA cores and 8GB of DDR6 memory, running at 384GB/s with 13.6 billion transistors on the chip.

The new Razer Blade has a six-core Intel i7-8750H processor with 16GB RAM and a 512GB SSD. It has mDP 1.4, HDMI 2.0b, Thunderbolt 3 and three USB 3.1 ports. Its 15.6-inch screen can run at 144Hz refresh rate but only supports full HD 1920×1080, which is optimized for gaming, not content creation. The past four laptops I have used have all been UHD resolution at various sizes, which gives far more screen real estate for creative applications and better resolution to review your imagery.

I also prefer to have an Ethernet port, but I am beginning to accept that a dongle might be acceptable for that, especially since it opens up the possibility of using 10 Gigabit Ethernet. We aren’t going to see 10GigE on laptops anytime soon due to the excessive power consumption, but you only need 10GigE when at certain locations that support it, so a dongle or docking station is reasonable for those use cases.

Certain functionality on the system required a free account to be registered with Razer, which is annoying, but I’ve found this requirement is becoming the norm these days. That gives access to the Razer Synapse utility for customizing the system settings, setting fan speed and even remapping keyboard functionality. Any other Razer peripherals would be controlled here as well. As part of a top-end modern gaming system, the keyboard has fully controllable color back lighting. While I find most of the default “effects” to be distracting, the option to color code your shortcut keys is interesting. And if you really want to go to the next level, you can customize it further.

For example, when you press the FN key, by default the keys that have function behaviors connected with them light up white, which impressed me. The colors and dimming are generated by blinking the LEDs, but I was able to perceive the flicker when moving my eyes, so I stuck with colors that didn’t involve dimming channels. But that still gave me six options (RGB, CYM) plus white.

This is the color config I was running in the photos, but the camera does not reflect how it actually looks. In pictures, the keys look washed out, but in person they are almost too bright and vibrant. But we are here for more than looks, so it was time to put it through its paces and see what can happen under the hood.

Testing
I ran a number of benchmarks, starting with Adobe Premiere Pro. I now have a consistent set of tests to run on workstations in order to compare each system. The tests involve Red, Sony Venice and ARRI Alexa source files, with various GPU effects applied and exported to compressed formats. It handled the 4K and 8K renders quite well — pretty comparable to full desktop systems — showcasing the power of the RTX GPU. Under the sustained load of rendering for 30 minutes, it did get quite warm, so you will want adequate ventilation … and you won’t want it sitting on your lap.

My next test was RedCine-X Pro, with its new CUDA playback acceleration of files up to 8K. But what is the point of decoding 8K if you can’t see all the pixels you are processing? So for this test, I also connected my Dell UP3218K screen to the Razer Blade’s Mini DisplayPort 1.4 output. Outputting to the monitor does affect performance a bit, but that is a reasonable expectation. It doesn’t matter if you can decode 8K in real time if you can’t display it. Nvidia provides reviewers with links to some test footage, but I have 40TB to choose from, in addition to test clips from all different settings on the various cameras from my Large Format Camera test last year.

The 4K Red files worked great at full res to the external monitor — full screen or pixel for pixel — while the system barely kept up with the 6K and 8K anamorphic files. 8K full frame required half-res playback to view smoothly on the 8K display. Full-frame 8K was barely realtime with the external monitor disabled, but that is still very impressive for a laptop (I have yet to accomplish that on my desktop). The rest of the files played back solidly to the local display. Disabling the CUDA GPU acceleration requires playing back below 1/8th res to do anything on a laptop, so this is where having a powerful GPU makes a big difference.

Blackmagic Resolve is the other major video editing program to consider, and while I do not find it intuitive to use myself, I usually recommend it to others who are looking for a high level of functionality but aren’t ready to pay for Premiere. I downloaded and rendered a test project from Nvidia, which plays Blackmagic Raw files in real time with a variety of effects and renders to H.264 in 40 seconds, but it takes 10 times longer with CUDA disabled in Resolve.

Here, as with the other tests, the real-world significance isn’t how much faster it is with a GPU than without, but how much faster is it with this RTX GPU compared to with other options. Nvidia clams this render takes 2.5 times as long on a Radeon-based MacBook Pro, and 10% longer on a previous-generation GTX 1080 laptop, which seems consistent with my previous experience and tests.

The primary differentiation of Nvidia’s RTX line of GPUs is the inclusion of RT cores to accelerate raytracing and Tensor cores to accelerate AI inferencing, so I wanted to try tasks that used those accelerations. I started by testing Adobe’s AI-based image enhancement in Lightroom Classic CC. Nvidia claims that the AI image enhancement uses the RTX’s Tensor cores, and it is four times faster with the RTX card. The visual results of the process didn’t appear to be much better than I could have achieved with manual development in Photoshop, but it was a lot faster to let the computer figure out what to do to improve the images. I also ran into an issue where certain blocks of the image got corrupted in the process, but I am not sure if Adobe or Nvidia is at fault here.

Raytracing
While I could have used this review as an excuse to go play Battlefield V to experience raytracing in video games, I stuck with the content-creation focus. In looking for a way to test raytracing, Nvidia pointed me to OctaneRender. Otoy has created a utility called OctaneBench for measuring the performance of various hardware configurations with its render engine. It reported that the RTX’s raytracing acceleration was giving me a 3x increase in render performance.

I also tested ProRender in Maxon Cinema 4D, which is not a raytracing renderer but does use GPU acceleration through OpenCL. Apparently, there is a way to use the Arnold ray-tracing engine in Cinema 4D, but I was reaching the limits of my 3D animation expertise and resources, so I didn’t pursue that path, and I didn’t test Maya for the same reason.

With ProRender, I was able to render views of various demo scenes 10 to 20 times faster than I could with a CPU only. I will probably include this as a regular test in future reviews, allowing me to gauge render performance far better than I can with Cinebench (which returned a CPU score of 836). And compiling a list of comparison render times will add more context to raw data. But, for now, I was able to render the demo “Bamboo” scene in 39 seconds and the more complex “Coffee Bean” scene in 188 seconds, beating even the Nvidia marketing team’s expected results.

VR
No test of a top-end GPU would be complete without trying out its VR performance. I connected my Windows-based Lenovo Explorer Mixed Reality headset, installed SteamVR and tested both 360 video editing in Premiere Pro and the true 3D experiences available in Steam. As would be expected, the experience was smooth, making this one of the most portable solutions for full-performance VR.

The RTX 2080 is a great GPU, and I had no issues with it. Outside of true 3D work, the upgrade from the Pascal-based GTX 1080 is minor, but for anyone upgrading from systems older than that, or doing true raytracing or AI processing, you will see a noticeable improvement in performance.

The new Razer Blade is a powerful laptop for its size, and while I did like it, that doesn’t mean I didn’t run into a few issues along the way. Some of those, like the screen resolution, are due to its focus on gaming instead of content creation, but I also had an issue with the touch pad. Touch pad issues are common when switching between devices constantly, but in this case, right-clicking instead of left-clicking and not registering movement when the mouse button was pressed were major headaches. The problems were only alleviated by connecting a mouse and sticking with that, which I frequently do anyway. The power supply has a rather large connector on a cumbersome thick and stiff cord, but it isn’t going to be falling out once you get it inserted. Battery life will vary greatly depending on how much processing power you are using.

These RTX chips are the first mobile GPUs with dedicated RT cores and with Tensor cores, since Volta-based chips never came to laptops. So for anyone with processing needs that are accelerated by those developments, the new RTX chip is obviously worth the upgrade. If you want the fastest thing out there, this is it. (Or at least it was, until Razer added options for 9th Generation Intel processors this week and a 4K OLED screen (an upgrade I would highly recommend for content creators). The model I reviewed goes for $3,000. The new 9th Gen version with a 240Hz screen is the same price, while the 4K OLED Touch version costs an extra $300.

Summing Up
If you are looking for a more balanced solution or are on a more limited budget, you should definitely compare the new Razer Blade to the new Nvidia GTX 16 line of mobile products that was just announced. Then decide which option is a better fit for your particular needs and budget.

The development of eGPUs has definitely shifted this ideal target for my usage. While this system has a Thunderbolt 3 port, it is fast enough that you won’t see significant gains from an eGPU, but that advantage comes at the expense of battery life and price. I am drawn to eGPUs because I only need maximum performance at my desk, but if you need top-end graphics performance totally untethered, RTX Max-Q chips are the solution for you.


Mike McCarthy is an online editor/workflow consultant with over 10 years of experience on feature films and commercials. He has been involved in pioneering new solutions for tapeless workflows, DSLR filmmaking and multi-screen and surround video experiences. Check out his site.

NAB 2019: First impressions

By Mike McCarthy

There are always a slew of new product announcements during the week of NAB, and this year was no different. As a Premiere editor, the developments from Adobe are usually the ones most relevant to my work and life. Similar to last year, Adobe was able to get their software updates released a week before NAB, instead of for eventual release months later.

The biggest new feature in the Adobe Creative Cloud apps is After Effects’ new “Content Aware Fill” for video. This will use AI to generate image data to automatically replace a masked area of video, based on surrounding pixels and surrounding frames. This functionality has been available in Photoshop for a while, but the challenge of bringing that to video is not just processing lots of frames but keeping the replaced area looking consistent across the changing frames so it doesn’t stand out over time.

The other key part to this process is mask tracking, since masking the desired area is the first step in that process. Certain advances have been made here, but based on tech demos I saw at Adobe Max, more is still to come, and that is what will truly unlock the power of AI that they are trying to tap here. To be honest, I have been a bit skeptical of how much AI will impact film production workflows, since AI-powered editing has been terrible, but AI-powered VFX work seems much more promising.

Adobe’s other apps got new features as well, with Premiere Pro adding Free-Form bins for visually sorting through assets in the project panel. This affects me less, as I do more polishing than initial assembly when I’m using Premiere. They also improved playback performance for Red files, acceleration with multiple GPUs and certain 10-bit codecs. Character Animator got a better puppet rigging system, and Audition got AI-powered auto-ducking tools for automated track mixing.

Blackmagic
Elsewhere, Blackmagic announced a new version of Resolve, as expected. Blackmagic RAW is supported on a number of new products, but I am not holding my breath to use it in Adobe apps anytime soon, similar to ProRes RAW. (I am just happy to have regular ProRes output available on my PC now.) They also announced a new 8K Hyperdeck product that records quad 12G SDI to HEVC files. While I don’t think that 8K will replace 4K television or cinema delivery anytime soon, there are legitimate markets that need 8K resolution assets. Surround video and VR would be one, as would live background screening instead of greenscreening for composite shots. No image replacement in post, as it is capturing in-camera, and your foreground objects are accurately “lit” by the screens. I expect my next major feature will be produced with that method, but the resolution wasn’t there for the director to use that technology for the one I am working on now (enter 8K…).

AJA
AJA was showing off the new Ki Pro Go, which records up to four separate HD inputs to H.264 on USB drives. I assume this is intended for dedicated ISO recording of every channel of a live-switched event or any other multicam shoot. Each channel can record up to 1080p60 at 10-bit color to H264 files in MP4 or MOV and up to 25Mb.

HP
HP had one of their existing Z8 workstations on display, demonstrating the possibilities that will be available once Intel releases their upcoming DIMM-based Optane persistent memory technology to the market. I have loosely followed the Optane story for quite a while, but had not envisioned this impacting my workflow at all in the near future due to software limitations. But HP claims that there will be options to treat Optane just like system memory (increasing capacity at the expense of speed) or as SSD drive space (with DIMM slots having much lower latency to the CPU than any other option). So I will be looking forward to testing it out once it becomes available.

Dell
Dell was showing off their relatively new 49-inch double-wide curved display. The 4919DW has a resolution of 5120×1440, making it equivalent to two 27-inch QHD displays side by side. I find that 32:9 aspect ratio to be a bit much for my tastes, with 21:9 being my preference, but I am sure there are many users who will want the extra width.

Digital Anarchy
I also had a chat with the people at Digital Anarchy about their Premiere Pro-integrated Transcriptive audio transcription engine. Having spent the last three months editing a movie that is split between English and Mandarin dialogue, needing to be fully subtitled in both directions, I can see the value in their tool-set. It harnesses the power of AI-powered transcription engines online and integrates the results back into your Premiere sequence, creating an accurate script as you edit the processed clips. In my case, I would still have to handle the translations separately once I had the Mandarin text, but this would allow our non-Mandarin speaking team members to edit the Mandarin assets in the movie. And it will be even more useful when it comes to creating explicit closed captioning and subtitles, which we have been doing manually on our current project. I may post further info on that product once I have had a chance to test it out myself.

Summing Up
There were three halls of other products to look through and check out, but overall, I was a bit underwhelmed at the lack of true innovation I found at the show this year.

Full disclosure, I was only able to attend for the first two days of the exhibition, so I may have overlooked something significant. But based on what I did see, there isn’t much else that I am excited to try out or that I expect to have much of a serious impact on how I do my various jobs.

It feels like most of the new things we are seeing are merely commoditized versions of products that may originally have been truly innovative when they were initially released, but now are just slightly more fleshed out versions over time.

There seems to be much less pioneering of truly new technology and more repackaging of existing technologies into other products. I used to come to NAB to see all the flashy new technologies and products, but now it feels like the main thing I am doing there is a series of annual face-to-face meetings, and that’s not necessarily a bad thing.

Until next year…


Mike McCarthy is an online editor/workflow consultant with over 10 years of experience on feature films and commercials. He has been involved in pioneering new solutions for tapeless workflows, DSLR filmmaking and multi-screen and surround video experiences. Check out his site.

Review: HP’s double-hinged ZBook Studio x360 mobile workstation

By Mike McCarthy

I recently had the opportunity to test HP’s ZBook Studio x360 mobile workstation over the course of a few weeks. HP’s ZBook mobile workstation division has really been thinking outside the box lately, with the release of the ZBook X2 tablet, the HP Z-VR backpack-mounted system and now the ZBook Studio x360.

The ZBook Studio x360 is similar in design functionality to HP’s other x360 models — the Pavilion, Spectre, Envy, ProBook and Elitebook x360 — in that the display is double-hinged. The keyboard can be folded all the way behind the screen, allowing it to be used similarly to a tablet or placed in “tent” or “presentation” mode with the keyboard partially folded behind it. But the ZBook is clearly the top-end option of the systems available in that form factor. And it inherits all of the engineering from the rest of HP’s extensive product portfolio, in regards to security, serviceability, and interface.

Performance-wise, this Studio x360 model sits somewhere in the middle of HP’s extensive ZBook mobile workstation lineup. It is above the lightweight ZBook 14U and 15U and X2 tablet with their low-voltage U-Series CPUs and the value-oriented 15v. It is similar to the more traditional clamshell ultrabook ZBook Studio, and has less graphics power and RAM than the top-end ZBook 15 and 17.

It is distinguished from the ZBook Studio by its double-hinged 360 folding chassis, and its touch and pen inking capability. It is larger than the ZBook X2 with more powerful internal hardware. This model is packed with processing power in the form of a 6-core 8th generation Xeon processor, 32GB RAM and an Nvidia Quadro P1000 GPU. The 15-inch UHD screen boosts up to 400 nits at full brightness and, of course, supports touch and pen input.

Configuration Options
The unit has a number of interesting configuration options with two M.2 slots and a 2.5-inch bay allowing up to 6TB of internal storage, but most users will forgo the 2.5-inch SATA bay for an extended 96whr battery. There is the option of choosing between a 4G WWAN card or DreamColor display, giving users a wide selection of possible capabilities.

Because of the work I do, I am mostly interested in answering the question: “How small and light can I go, and still get my work done effectively?” In order to answer that question, I am reviewing a system with most of the top-end options. I started at a 17-inch Lenovo P71 last year, then tried a large 15-inch PNY PrevailPro and now am trying out this much lighter 15-inch book. There is no compromise with the 6-core CPU, as that is the same as in a 17-inch beast. So the biggest difference is in the GPU, with the mobile Quadro P1000 only having the 512 CUDA core, one third the power of the Quadro P4000 I last tested. So VR is not going to work, but besides heavy color grading, most video editing tasks should be supported. And 32GB of RAM should be enough for most users, but I installed a second NVMe drive, giving me a total of 2TB of storage.

Display
The 15.6-inch display is available in a number of different options, all supporting touch and digital pen input. The base-level full-HD screen can be upgraded to a Sure View screen, allowing the user to selectively narrow the viewing angle at the press of a key in order to increase their privacy. Next up is the beautiful 400-nit UHD screen that my unit came with. And the top option is a 600-nit DreamColor calibrated UHD panel. All of the options fully support touch and pen input.

Connectivity
The unit has dual-Thunderbolt 3 ports, supporting DisplayPort 1.3, as well as HDMI, dual-USB3.1 Type-A ports, an SDXC card slot and an audio jack. The main feature I am missing is an RJ-45 jack for Gigabit Ethernet. I get that there are trade-offs to be made in any configuration, but that is the item I am missing from this unit. On the flip side, with the release of affordable Thunderbolt-based 10GbE adapters, that is probably what I would pair with this unit if I was going to be using it to edit assets I have stored on my network. So that is a solvable problem.

Serviceability
Unlike the heavier ZBook 15 and 17 models, it does not have a tool-less chassis, but that is an understandable a compromise to reduce size and weight, and totally reasonable. I was able to remove the bottom cover with a single torx screwdriver, giving me access to the RAM, wireless cards, and M.2 slots I was populating with a second NVMe drive to test. The battery can also be replaced that way should the need arise, but the 96whr long-life battery is fully covered by the system warranty, be that three or five years depending on your service level.

Security
There are a number of unique features that this model shares with many others in HP’s lineup. The UEFI-based HP Sure Start BIOS and pre-boot environment provide a host of options for enterprise-level IT management, and make it less likely that the boot process will get corrupted. HP Sure Click is a security mechanism that isolates each Chromium browser tab in its own virtual machine, protecting the rest of your system from any malware that it might otherwise be exposed to. Sure Run and Sure Recover are designed to prevent and recover from security failures that render the system unusable.

The HP Client Security Manager brings the controls for all of this functionality into one place and uses the system’s integrated fingerprint reader. HP Workwise is a utility for integrating the laptop with one’s cell phone, allowing automatic system lock and unlock when the cell phone leaves or enters Bluetooth range and phone notifications from the other “Sure” security applications.

Thunderbolt Dock
HP also supplied me with their new Thunderbolt dock. The single most important feature on that unit from my perspective is the Gigabit Ethernet port, since there isn’t one built into the laptop. It also adds two DisplayPorts and one VGA output and includes five more USB ports. I was able to connect my 8K display to the DisplayPort output and it ran fine at 30Hz, as is to be expected from a single Thunderbolt connection. The dock should run anything smaller than that at 60Hz, including two 4K displays.

The dock also supports an optional audio module to facilitate better conference calls, with a built-in speaker, microphone and call buttons. It is a nice idea but a bit redundant since the laptop has a “world-facing” microphone for noise cancellation or group calling and even has “Collaboration Keys” for controlling calls built into the top of the keyboard. Apparently, HP sees this functionality totally replacing office phones.

I initially struggled to get the dock to work — besides the DisplayPorts — but this was because I connected it before boot-up. Unlike docking stations from back in the day, Thunderbolt is fully hot-swappable and actually needs to be powered on the first time it is connected in order to trigger the dialog box, which gives it low-level access to your computer for security reasons. Once I did that, it has worked seamlessly.

The two-part cable integrates a dedicated power port and Thunderbolt 3 connection, magnetically connected for simple usage while maintaining flexibility for future system compatibility. The system can receive power from the Thunderbolt port, but for maximum power and performance uses a 130W dedicated power plug as well, which appears to be standardized across much of HP’s line of business products.

Touchscreens and Pens
I had never seriously considered tablets or touchscreen solutions for my own work until one of HP’s reps showed me an early prototype of the ZBook X2 a few years ago. I initially dismissed it until he explained how much processing power they had packed into it. Only then did I recognize that HP had finally fulfilled two of my very different and long-standing requests in a way that I hadn’t envisioned. I had been asking the display team for a lightweight battery-powered DreamColor display, and I had been asking the mobile workstation team for a 12- or 14-inch Nvidia-powered model — this new device was both.

I didn’t end up reviewing the X2 during its initial release last year, although I plan to soon. But once the X2 shifted my thinking about tablet and touch-based tools, I saw this ZBook Studio x360 as an even more powerful implementation of that idea, in a slightly larger form factor. While I have used pens on other people’s systems in the past, usually when doing tech support for other editors, this is my first attempt to do real work with a pen instead of a mouse and keyboard.

One of the first obstacles I encountered was getting the pen to work at all. Unlike the EMR-based pens from Wacom tablets and the ZBook X2, the x360 uses an AES-based pen, which requires power and a Bluetooth connection to communicate with the system. I am not the only user to be confused by this solution, but I have been assured by HP that the lack of documentation and USB-C charging cable have been remedied in currently shipping systems.

It took me a while (and some online research) to figure out that there was a USB-C port hidden in the pen and that it needed to be charged and paired with the system. Once I did that, it has functioned fine for me. The pen itself works great, with high precision and 4K levels of pressure sensitivity and tilt support. I am not much of a sketcher or painter, but I do a lot of work in Photoshop, either cleaning images up or creating facial expressions for my Character Animator puppets. The pen is a huge step up from the mouse for creating smooth curves and natural lines. And the various buttons worked well for me once I got used to them. But I don’t do a lot of work that benefits from having the pen support, and trying to adapt other tasks to the pen-based input was more challenging than I anticipated.

The other challenge I encountered was with the pen holder, which fits into the SD card slot. The design is good and works better than I would have expected, but removing the original SD plug that protects the slot was far more difficult than it should be. I assume the plug is necessary for the system to pass the 13 MilSpec type tests that HP runs all of its ZBooks through, but I probably won’t be wedging it back in that slot as long as I have the system.

Inking
I am not much of a tablet user as of yet since this was my first foray into that form factor, but the system is a bit large and bulky when folded back into tablet mode. I have hit the power button by accident on multiple occasions, hibernating the system while I was trying to use it. This has primarily been an issue when I am using it in tablet mode and holding it with my left hand in that area by default. But the biggest limitation I encountered in tablet mode was recognizing just how frequently I use the keyboard during the course of my work. While Windows Inking does allow for an onscreen keyboard to be brought up for text entry, functions like holding Alt for anchor-based resizing are especially challenging. I am curious to see if some of these issues are alleviated on the X2 by the buttons they built into the edge of the display. As long as I have easy access to Shift, Ctrl, Alt, C, V and a couple others, I think I would be good to go, but it is one of those things that you can’t know for sure until you try it yourself. And different people with varying habits and preferences might prefer different solutions to the same tasks. In my case, I have not found the optimal touch and inking experience yet.

Performance
I was curious to see what level of performance I would get from the Quadro P1000, as I usually use systems with far more GPU power. But I was impressed with how well it was able to handle the animating and editing of the 5K assets for my Grounds of Freedom animated series. I was even able to dynamically link between the various Adobe apps with a reasonable degree of interactive feedback. That is where you start to see a difference between this mobile system and a massive desktop workstation.

eGPU
Always looking for more power, I hooked up Sonnet’s Breakaway Box 550 with a variety of different Nvidia GPUs to accelerate the graphics performance of the system. The Quadro P6000 was the best option, as it used the same Quadro driver and Pascal architecture as the integrated P1000 GPU but greatly increased performance.

It allowed me to use my Lenovo Explorer WMR headset to edit 360 video in VR with Premiere Pro, and I was able to playback 8K DNxHR files at full resolution in Premiere to my Dell 8K LCD display. I was also able to watch 8K HEVC files in Windows movie player smoothly. Pretty impressive for a 15-inch convertible laptop, but the 6-Core Xeon processor pairs well with the desktop GPU, making this an ideal system to harness the workflow possibilities offered by eGPU solutions.

Media Export Benchmarks
I did extensive benchmark testing, measuring the export times of various media at different settings with different internal and external GPU options. The basic conclusion was that currently simple transcodes and conversions are not much different with an eGPU, but that once color correction and other effects are brought into the equation, increasing GPU power makes processing two to five times faster.

I also tested DCP exports with Quvis’ Wraptor plugin for AME and found the laptop took less than twice as long as my top-end desktop to make DCPs, which I consider to be a good thing. You can kick out a 4K movie trailer in under 10 minutes. And if you want to export a full feature film, I would recommend a desktop, but this will do it in a couple of hours.

Final Observations
The ZBook Studio x360 is a powerful machine and an optimal host for eGPU workflows. While it exceeded my performance expectations, I did not find the touch and ink solution to be optimal for my needs as I am a heavy keyboard user, even when doing artistic tasks. (To be clear, I haven’t found a better solution. This just doesn’t suitably replace my traditional mouse and keyboard approach to work.) So if buying one for myself, I would personally opt for the non-touch ZBook Studio model. But for anyone to whom inking is a critical part of their artistic workflow, who needs a powerful system on the go, this is a very capable model that doesn’t appear to have too many similar alternatives. It blends the power of the ZBook Studio with the inking experience of HP’s other x360 products.


Mike McCarthy is an online editor/workflow consultant with over 10 years of experience on feature films and commercials. He has been involved in pioneering new solutions for tapeless workflows, DSLR filmmaking and multi-screen and surround video experiences. Check out his site.

Review: eGPUs and the Sonnet Breakaway Box

By Mike McCarthy

As a laptop user and fan of graphics performance, I have always had to weigh the balance between performance and portability when selecting a system. And this usually bounces back and forth, as neither option is totally satisfactory. Systems are always too heavy or not powerful enough.

My first laptop when I graduated high school was the 16-inch Sony Vaio GRX570, with the largest screen available at the time, running 1600×1200 pixels. After four years carrying that around, I was eager to move to the Dell XPS M1210, the smallest laptop with a discrete GPU. That was followed by a Quadro-based Dell Precision M4400 workstation, which was on the larger side. I then bounced to the lightweight Carbon Fiber 13-inch Sony Vaio Z1 in 2010, which my wife still uses. This was followed by my current Aorus X3 Plus, which has both power (GF870M) and a small form factor (13 inch), but at the expense of everything else.

Some More History
The Vaio Z1 was one of the first hybrid graphics solutions to allow users to switch between different GPUs. Its GeForce 330M was powerful enough to run Adobe’s Mercury CUDA Playback engine in CS5, but was at the limit of its performance. It didn’t support my 30-inch display, and while the SSD storage solution had the throughput for 2K DPX playback, the GPU processing couldn’t keep up.

Other users were upgrading the GPU with an ExpressCard-based ViDock external PCIe enclosure, but a single-lane of PCIe 1.0 bandwidth (2Gb/s) wasn’t enough to make is worth the effort for video editing. (3D gaming requires less source bandwidth than video processing.) Sony’s follow-on Z2 model offered the first commercial eGPU, connected via LightPeak, the forerunner to Thunderbolt. It allowed the ultra-light Z series laptop to use an AMD Radeon 6650M GPU and Blu-ray drive in the proprietary Media Dock, presumably over a PCIe x4 1.0 (8Gb/s) connection.

Thunderbolt 3
Alienware also has a propriety eGPU solution for their laptops, but Thunderbolt is really what makes eGPUs a marketable possibility, giving direct access to the PCIe bus at x4 speed, in a standardized connection. The first generation offered a dedicated 10Gb connection, while Thunderbolt 2 increased that to a 20Gb shared connection. The biggest thing holding back eGPUs at that point was lack of PC adoption of the Apple technology licensed from Intel, and OS X limitations on eGPUs.

Thunderbolt 3 changed all of that, increasing the total connection bandwidth to 40Gb, the same as first-generation PCIe x16 cards. And far more systems support Thunderbolt 3 than the previous iterations. Integrated OS support for GPU switching in Windows 10 and OS X (built on laptop GPU power saving technology) further paved the path to eGPU adoption.

Why eGPUs Now?
Even with all of this in my favor, I didn’t take the step into eGPU solutions until very recently. I bought my personal system in 2014. This was just before Thunderbolt 3 hit the mainstream. The last two systems I reviewed had Thunderbolt 3, but didn’t need eGPUs with their mobile Quadro P4000 and P5000 internal GPUs. So I hadn’t had the opportunity to give it a go until I received an HP Zbook Studio x360 to review. Now, its integrated Quadro P1000 is nothing to scoff at, but there was significantly more room for performance gains from an external GPU.

Sonnet Breakaway Box
I have had the opportunity to review the 550W version of Sonnet’s Breakaway Box PCIe enclosure over the course of a few weeks, allowing me to test out a number of different cards, including four different GPUs, as well as my Red-Rocket-X and 10GbE cards. Sonnet has three different eGPU enclosure options, depending on the power requirements of your GPU.

They sent me the mid-level 550 model, which should support every card on the market, aside from AMD’s power-guzzling Vega 64-based GPUs. The base 350 model should support GF1080 or 2080 cards, but not overclocked Titanium or Titan versions. The 550 model includes two PCIe power cables that can be used in 6- or 8-pin connectors. This should cover any existing GPU on the market, and I have cards requiring nearly every possible combo — 6-pin, 8-pin, both, and dual 8-pin. Sonnet has a very thorough compatibility list available, for more specific details.

Installation
I installed my Quadro P6000 into the enclosure, because it used the same drivers as my internal Quadro P1000 GPU and would give me the most significant performance boost. I plugged the Thunderbolt connector into the laptop while it was booted. It immediately recognized the device, but only saw it as a “Microsoft Basic Display Adapter” until I re-installed my existing 411.63 Quadro drivers and rebooted. After that, it worked great, I was able to run my benchmarks and renders without issue, and I could see which GPU was carrying the processing load just by looking in the task manager performance tab.

Once I had finished my initial tests, safely removed the hardware in the OS and disconnected the enclosure, I swapped the installed card with my Quadro P4000 and plugged it back into the system without rebooting. It immediately detected it, and after a few seconds the new P4000 was recognized and accelerating my next set of renders. When I attempted to do the same procedure with my GeForce 2080TI, it did make me install the GeForce driver (416.16) and reboot before it would function at full capacity (subsequent transitions between Nvidia cards were seamless).

The next step was to try an AMD GPU, since I have a new RadeonPro WX8200 to test, which is a Pro version of the Vega 56 architecture. I was a bit more apprehensive about this configuration due to the integrated Nvidia card, and having experienced those drivers not co-existing well in the distant past. But I figured: “What’s the worst that could happen?”

Initially, plugging it in gave me the same Microsoft Basic Display Adapter device until I installed the RadeonPro drivers. Installing those drivers caused the system to crash and refuse to boot. Startup repair, system restore and OS revert all failed to run, let alone fix the issue. I was about to wipe the entire OS and let it reinstall from the recovery partition when I came across one more idea online. I was able to get to a command line in the pre-boot environment and run a Deployment Image Servicing and Management (DISM) command to see which drivers were installed — DISM /image:D:\ /Get-Drivers|more.

This allowed me to see that the last three drivers — oem172.inf through oem174.inf —were the only AMD-related ones on the system. I was able to remove them via the same tool — DISM /Image:D:\ /Remove-Driver /Driver:oem172.inf”) — and when I restarted, the system booted up just fine.

I then pulled the card from the eGPU box, wiped all the AMD files from the system, and vowed never to do something like that again. Lesson of the day: Don’t mix AMD and Nvidia cards and drivers. To AMDs credit, the WX8200 does not officially support eGPU installations, but extraneous drivers shouldn’t cause that much problem.

Performance Results
I tested Adobe Media Encoder export times with a variety of different sources and settings. Certain tests were not dramatically accelerated by the eGPU, while other renders definitely were. The main place we see differences between the integrated P1000 and a more-powerful external GPU is when effects are applied to high-res footage. That is when the GPU is really put to work, so those are the tests that improve with more GPU power. I had a one-minute sequence of Red clips with lots of effects (Lumetri, selective blur and mosaic: all GPU FX) that took 14 minutes to render internally, but finished in under four minutes with the eGPU attached. Exporting the same sequence with the effects disabled took four minutes internally and three minutes with the GPU. So the effects cost 10 minutes of render time internally, but under one minute of render time (35 seconds to be precise) when a powerful GPU is attached.

So if you are trying to do basic cuts-only editorial, an eGPU may not improve your performance much, but if you are doing VFX or color work, it can make a noticeable difference.

VR Headset Support
The external cards, of course, do increase performance in a measurable way, especially since I am using such powerful cards. It’s not just a matter of increasing render speeds, but about enabling functionality that was previously unavailable on the system. I connected my Lenovo Explorer WMR headset to the RTX2080TI in the Breakaway Box and gave it a shot. I was able to edit 360 video in VR in Premiere Pro, which is not supported on the included Quadro P1000 card. I did experience some interesting ghosting on occasion, where if I didn’t move my head everything looked perfect, but movement caused a double image — as if one eye was a frame behind the other — but the double image was appearing in each eye, as if there was an excessive motion blur applied to the rendered frames.

I thought this might be a delay based on extra latency in the Thunderbolt bus, but other times the picture looked crisp regardless of how quickly I moved my head. So it can work great, but there may need to be a few adjustments made to smooth things out. Lots of other users online report it working just fine, so there is probably a solution available out there.

Full-Resolution 8K Tests
I was able to connect my 8K display to the card as well, and while the x360 happens to support that display already (DP1.3 over Thunderbolt), most notebooks do not — and it increased the refresh rate from 30Hz to the full 60Hz. I was able to watch HEVC videos smoothly at 8K in Windows, and was able to playback 8K DNxHR files in Premiere at full res, as long as there were no edits or effects.

Just playing back footage at full 8K taxed the 2080TI at 80% compute utilization. But this is 8K we are talking about, playing back on a laptop, at full resolution. 4K anamorphic and 6K Venice X-OCN footage played back smoothly at half res in Premiere, and 8K Red footage played back at quarter. This is not the optimal solution for editing 8K footage, but it should have no problem doing serious work at UHD and 4K.

Other Cards and Functionality
GPUs aren’t the only PCIe cards that can be installed in the Breakaway Box, so I can add a variety of other functionality to my laptop if desired. Thunderbolt array controllers minimize the need for SATA or SAS cards in enclosures, but that is a possibility. I installed an Intel X520-DA2 10GbE card into the box and was copying files from my network at 700MB/s within a minute, without even having to install any new drivers. But unless you need to have SFP ports, most people looking for 10GbE functionality would be better served to look into Sonnet’s Solo 10G for smaller form factor, lower power use, and cheaper price. There are a variety of other options for Thunderbolt 3 to 10GbE hitting the market as well.

The Red-Rocket-X card has been a popular option for external PCIe enclosures over the last few years, primarily for on-set media transcoding. I installed mine in the Breakaway Box to give that functionality a shot as well.

I ran into two issues, both of which I was able to overcome, but are worth noting. First, the 6-pin power connector is challenging to fit into the poorly designed Rocket power port, due to the retention mechanism being offset for 8-pin compatibility. But it can fit if you work at it a bit, although I prefer to keep a 6-pin extension cable plugged into my Rocket since I move it around so much. Once I had all of the hardware hooked up, it was recognized in the OS, but installing the drivers from Red resulted in a Code-52 error that is usually associated with USB devices. The recommended solution online was to disable Windows 10 driver signing, in the pre-boot environment, and that did the trick. (My theory is that my HP’s SureStart security functionality was hesitating to give direct memory access to an external device, as that is the level of access Thunderbolt devices get to your system, and the Red Rocket-X driver wasn’t signed for that level of security.)

Anyhow, the card worked fine after that, and I verified that it accelerated my renders in Premiere Pro and AME. I am looking forward to a day when CUDA acceleration allows me to get that functionality out of my underused GPU power instead of requiring a dedicated card.

I did experience an issue with the Quadro P4000, where the fans spun up to 100% when the laptop went to shut off, hibernated, or went to sleep. None of the other cards had that issue, instead they shut off when the host system did and turned back on automatically when I booted up the system. I have no idea why the P4000 acted differently than the architecturally very similar P6000. Manually turning off the Breakaway Box or disconnecting the Thunderbolt cable solves the problem with the P4000, but then you have to remember to reconnect again when you are booting up.

In the process of troubleshooting the fan issue, I did a few other driver installs and learned a few tricks. First off, I already knew Quadro drivers can’t run GeForce cards (otherwise why pay for a Quadro), but GeForce drivers can run on Quadro cards. So it makes sense you would want to install GeForce drivers when mixing both types of GOUs. But I didn’t realize that apparently GeForce drivers take preference when they are installed. So when I had an issue with the internal Quadro card, reinstalling the Quadro drivers had no effect, since the GeForce drivers were running the hardware. Removing them (with DDU just to be thorough) solved the issue, and got everything operating seamlessly again. Sonnet’s support people were able to send me the solution to the problem on the first try. That was a bit of a hiccup, but once it was solved I could again swap between different GPUs without even rebooting. And most users will always have the same card installed when they connect their eGPU, further simplifying the issue.

Do you need an eGPU?
I really like this unit, and I think that eGPU functionality in general will totally change the high-end laptop market for the better. For people who only need high performance at their desk, there will be a class of top-end laptop with high-end CPU, RAM and storage, but no GPU to save on space and weight (CPU can’t be improved by external box, and needs to keep up with GPU).

There will be another similar class with mid-level GPUs to support basic 3D work on the road, but massive increases at home. I fall in the second category, as I can’t forego all GPU acceleration when I am traveling or even walking around the office. But I don’t need to be carrying around an 8K rendering beast all the time either. I can limit my gaming, VR work and heavy renders to my desk. That is the configuration I have been able to use with this ZBook x360.: enough power to edit un-tethered, but combining the internal 6-core CPU with a top -end external GPU gives great performance when attached to the Breakaway Box. As always, I still want to go smaller, and plan to test with an even lighter weight laptop as soon as the opportunity arises.

Summing Up
The Breakaway Box is a simple solution to a significant issue. No bells and whistles, which I initially appreciated. But the eGPU box is inherently a docking station, so there is an argument to be made for adding other functionality. In my case, once I am setup at my next project, using a 10GbE adapter in the second TB3 port on my laptop will be a better solution for top performance and bandwidth anyway.

So I am excited about the possibilities that eGPUs bring to the table, now that they are fully supported by the OS and applications I use, and I don’t imagine buying a laptop setup without one anytime in the foreseeable future. The Sonnet Breakaway Box meets my needs and has performed very well for me over the last few weeks.


Mike McCarthy is an online editor/workflow consultant with over 10 years of experience on feature films and commercials. He has been involved in pioneering new solutions for tapeless workflows, DSLR filmmaking and multi-screen and surround video experiences. Check out his site.

New codec, workflow options via Red, Nvidia and Adobe

By Mike McCarthy

There were two announcements last week that will impact post production workflows. The first was the launch of Red’s new SDK, which leverages Nvidia’s GPU-accelerated CUDA framework to deliver realtime playback of 8K Red footage. I’ll get to the other news shortly. Nvidia was demonstrating an early version of this technology at Adobe Max in October, and I have been looking forward to this development since I am about to start post on a feature film shot on the Red Monstro camera. This should effectively render the RedRocket accelerator cards obsolete, replacing them with cheaper, multipurpose hardware that can also accelerate other computational tasks.

While accelerating playback of 8K content at full resolution requires a top-end RTX series card from Nvidia (Quadro RTX 6000, Titan RTX or GeForce RTX 2080Ti), the technology is not dependent on RTX’s new architecture (RT and Tensor cores), allowing earlier generation hardware to accelerate smooth playback at smaller frame sizes. Lots of existing Red footage is shot at 4K and 6K, and playback of these files will be accelerated on widely deployed legacy products from previous generations of Nvidia GPU architecture. It will still be a while before this functionality is in the hands of end users, because now Adobe, Apple, Blackmagic and other software vendors have to integrate the new SDK functionality into their individual applications. But hopefully we will see those updates hitting the market soon (targeting late Q1 of 2019).

Encoding ProRes on Windows via Adobe apps
The other significant update, which is already available to users as of this week, is Adobe’s addition of ProRes encoding support on its video apps in Windows. Developed by Apple, ProRes encoding has been available on Mac for a long time, and ProRes decoding and playback has been available on Windows for over 10 years. But creating ProRes files on Windows has always been a challenge. Fixing this was less a technical challenge than a political one, as Apple owns the codec and it is not technically a standard. So while there were some hacks available at various points during that time, Apple has severely restricted the official encoding options available on Windows… until now.

With the 13.0.2 release of Premiere Pro and Media Encoder, as well as the newest update to After Effects, Adobe users on Windows systems can now create ProRes files in whatever flavor they happen to need. This is especially useful since many places require delivery of final products in the ProRes format. In this case, the new export support is obviously a win all the way around.

Adobe Premiere

Now users have yet another codec option for all of their intermediate files, prompting another look at the question: Which codec is best for your workflow? With this release, Adobe users have at least three major options for high-quality intermediate codecs: Cineform, DNxHR and now ProRes. I am limiting the scope to integrated cross-platform codecs supporting 10-bit color depth, variable levels of image compression and customizable frame sizes. Here is a quick overview of the strengths and weaknesses of each option:

ProRes
ProRes was created by Apple over 10 years ago and has become the de-facto standard throughout the industry, regardless of the fact that it is entirely owned by Apple. ProRes is now fully cross-platform compatible, has options for both YUV and RGB color and has six variations, all of which support at least 10-bit color depth. The variable bit rate compression scheme scales well with content complexity, so encoding black or static images doesn’t require as much space as full-motion video. It also supports alpha channels with compression, but only in the 444 variants of the codec.

Recent tests on my Windows 10 workstation resulted in ProRes taking 3x to 5x as much CPU power to playback as similar DNxHR of Cineform files, especially as frame sizes get larger. The codec supports 8K frame sizes but playback will require much more processing power. I can’t even playback UHD files in ProRes 444 at full resolution, while the Cineform and DNxHR files have no problem, even at 444. This is less of concern if you are only working at 1080p.

Multiply those file sizes by four for UHD content (and by 16 for 8K content).

Cineform
Cineform, which has been available since 2004, was acquired by GoPro in 2011. They have licensed the codec to Adobe, (among other vendors) and it is available as “GoPro Cineform” in the AVI or QuickTime sections of the Adobe export window. Cineform is a wavelet compression codec, with 10-bit YUV and 12-bit RGB variants, which like ProRes support compressed alpha channels in the RGB variant. The five levels of encoding quality are selected separately from the format, so higher levels of compression are available for 4444 content compared to the limited options available in the other codecs.

It usually plays back extremely efficiently on Windows, but my recent tests show that encoding to the format is much slower than it used to be. And while it has some level of support outside of Adobe applications, it is not as universally recognized as ProRes or DNxHD.

DNxHD
DNxHD was created by Avid for compressed HD playback and has now been extended to DNxHR (high resolution). It is a fixed bit rate codec, with each variant having a locked multiplier based on resolution and frame rate. This makes it easy to calculate storage needs but wastes space for files that are black or contain a lot of static content. It is available in MXF and Mov wrappers and has five levels of quality. The top option is 444 RGB, and all variants support alpha channels in Mov but uncompressed, which takes a lot of space. For whatever reason, Adobe has greatly optimized DNxHR playback in Premiere Pro, of all variants, in both MXF and Mov wrappers. On my project 6Below, I was able to get 6K 444 files to playback, with lots of effects, without dropping frames. The encodes to and from DNxHR are faster in Adobe apps as well.

So for most PC Adobe users, DNxHR-LB (low bandwidth) is probably the best codec to use for intermediate work. We are using it to offline my current project, with 2.2K DNxHR-LB, Mov files. People with a heavy Mac interchange may lean toward ProRes, but up your CPU specs for the same level of application performance.


Mike McCarthy is an online editor/workflow consultant with 10 years of experience on feature films and commercials. He has been involved in pioneering new solutions for tapeless workflows, DSLR filmmaking and multi-screen and surround video experiences. Check out his site.

Making an animated series with Adobe Character Animator

By Mike McCarthy

In a departure from my normal film production technology focus, I have also been working on an animated web series called Grounds of Freedom. Over the past year I have been directing the effort and working with a team of people across the country who are helping in various ways. After a year of meetings, experimentation and work we finally started releasing finished episodes on YouTube.

The show takes place in Grounds of Freedom, a coffee shop where a variety of animated mini-figures gather to discuss freedom and its application to present-day cultural issues and events. The show is created with a workflow that weaves through a variety of Adobe Creative Cloud apps. Back in October I presented our workflow during Adobe Max in LA, and I wanted to share it with postPerspective’s readers as well.

When we first started planning for the series, we considered using live action. Ultimately, after being inspired by the preview releases of Adobe Character Animator, I decided to pursue a new digital approach to brick filming (a film made using Legos), which is traditionally accomplished through stop-motion animation. Once everyone else realized the simpler workflow possibilities and increased level of creative control offered by that new animation process, they were excited to pioneer this new approach. Animation gives us more control and flexibility over the message and dialog, lowers production costs and eases collaboration over long distances, as there is no “source footage” to share.

Creating the Characters
The biggest challenge to using Character Animator is creating digital puppets, which are deeply layered Photoshop PSDs with very precise layer naming and stacking. There are ways to generate the underlying source imagery in 3D animation programs, but I wanted the realism and authenticity of sourcing from actual photographs of the models and figures. So we took lots of 5K macro shots of our sets and characters in various positions with our Canon 60D and 70D DSLRs and cut out hundreds of layers of content in Photoshop to create our characters and all of their various possible body positions. The only thing that was synthetically generated was the various facial expressions digitally painted onto their clean yellow heads, usually to match an existing physical reference character face.

Mike McCarthy shooting stills.

Once we had our source imagery organized into huge PSDs, we rigged those puppets in Character Animator with various triggers, behaviors and controls. The walking was accomplished by cycling through various layers, instead of the default bending of the leg elements. We created arm movement by mapping each arm position to a MIDI key. We controlled facial expressions and head movement via webcam, and the mouth positions were calculated by the program based on the accompanying audio dialog.

Animating Digital Puppets
The puppets had to be finished and fully functional before we could start animating on the digital stages we had created. We had been writing the scripts during that time, parallel to generating the puppet art, so we were ready to record the dialog by the time the puppets were finished. We initially attempted to record live in Character Animator while capturing the animation motions as well, but we didn’t have the level of audio editing functionality we needed available to us in Character Animator. So during that first session, we switched over to Adobe Audition, and planned to animate as a separate process, once the audio was edited.

That whole idea of live capturing audio and facial animation data is laughable now, looking back, since we usually spend a week editing the dialog before we do any animating. We edited each character audio on a separate track and exported those separate tracks to Character Animator. We computed lipsync for each puppet based on their dedicated dialog track and usually exported immediately. This provided a draft visual that allowed us to continue editing the dialog within Premiere Pro. Having a visual reference makes a big difference when trying to determine how a conversation will feel, so that was an important step — even though we had to throw away our previous work in Character Animator once we made significant edit changes that altered sync.

We repeated the process once we had a more final edit. We carried on from there in Character Animator, recording arm and leg motions with the MIDI keyboard in realtime for each character. Once those trigger layers had been cleaned up and refined, we recorded the facial expressions, head positions and eye gaze with a single pass on the webcam. Every re-record to alter a particular section adds a layer to the already complicated timeline, so we limited that as much as possible, usually re-recording instead of making quick fixes unless we were nearly finished.

Compositing the Characters Together
Once we had fully animated scenes in Character Animator, we would turn off the background elements, and isolate each character layer to be exported in Media Encoder via dynamic link. I did a lot of testing before settling on JPEG2000 MXF as the format of choice. I wanted a highly compressed file, but need alpha channel support, and that was the best option available. Each of those renders became a character layer, which was composited into our stage layers in After Effects. We could have dynamically linked the characters directly into AE, but with that many layers that would decrease performance for the interactive part of the compositing work. We added shadows and reflections in AE, as well as various other effects.

Walking was one of the most challenging effects to properly recreate digitally. Our layer cycling in Character Animator resulted in a static figure swinging its legs, but people (and mini figures) have a bounce to their step, and move forward at an uneven rate as they take steps. With some pixel measurement and analysis, I was able to use anchor point keyframes in After Effects to get a repeating movement cycle that made the character appear to be walking on a treadmill.

I then used carefully calculated position keyframes to add the appropriate amount of travel per frame for the feet to stick to the ground, which varies based on the scale as the character moves toward the camera. (In my case the velocity was half the scale value in pixels per seconds.) We then duplicated that layer to create the reflection and shadow of the character as well. That result can then be composited onto various digital stages. In our case, the first two shots of the intro were designed to use the same walk animation with different background images.

All of the character layers were pre-comped, so we only needed to update a single location when a new version of a character was rendered out of Media Encoder, or when we brought in a dynamically linked layer. It would propagate all the necessary comp layers to generate updated reflections and shadows. Once the main compositing work was finished, we usually only needed to make slight changes in each scene between episodes. These scenes were composited at 5K, based on the resolution off the DSLR photos of the sets we had built. These 5K plates could be dynamically linked directly into Premiere Pro, and occasionally used later in the process to ripple slight changes through the workflow. For the interactive work, we got far better editing performance by rendering out flattened files. We started with DNxHR 5K assets, but eventually switched to HEVC files since they were 30x smaller and imperceptibly different in quality with our relatively static animated content.

Editing the Animated Scenes
In Premiere Pro, we had the original audio edit, and usually a draft render of the characters with just their mouths moving. Once we had the plate renders, we placed them each in their own 5K scene sub-sequence and used those sequences as source on our master timeline. This allowed us to easily update the content when new renders were available, or source from dynamically linked layers instead if needed. Our master timeline was 1080p, so with 5K source content we could push in two and a half times the frame size without losing resolution. This allowed us to digitally frame every shot, usually based on one of two rendered angles, and gave us lots of flexibility all the way to the end of the editing process.

Collaborative Benefits of Dynamic Link
While Dynamic Link doesn’t offer the best playback performance without making temp renders, it does have two major benefits in this workflow. It ripples change to the source PSD all the way to the final edit in Premiere just by bringing each app into focus once. (I added a name tag to one character’s PSD during my presentation, and 10 seconds later, it was visible throughout my final edit.) Even more importantly, it allows us to collaborate online without having to share any exported video assets. As long as each member of the team has the source PSD artwork and audio files, all we have to exchange online are the Character Animator project (which is small once the temp files are removed), the .AEP file and the .PrProj file.

This gives any of us the option to render full-quality visual assets anytime we need them, but the work we do on those assets is all contained within the project files that we sync to each other. The coffee shop was built and shot in Idaho, our voice artist was in Florida, our puppets faces were created in LA. I animate and edit in Northern California, the AE compositing was done in LA, and the audio is mixed in New Jersey. We did all of that with nothing but a Dropbox account, using the workflow I have just outlined.

Past that point, it was a fairly traditional finish, in that we edited in music and sound effects, and sent an OMF to Steve, our sound guy at DAWPro Studios http://dawpro.com/photo_gallery.html for the final mix. During that time we added other b-roll visuals or other effects, and once we had the final audio back we rendered the final result to H.264 at 1080p and uploaded to YouTube.


Mike McCarthy is an online editor/workflow consultant with over 10 years of experience on feature films and commercials. He has been involved in pioneering new solutions for tapeless workflows, DSLR filmmaking and multi-screen and surround video experiences. Check out his site.

A Technologist’s Data Storage Primer

By Mike McCarthy

Storage is the concept of keeping all of the files for a particular project or workflow, but they may not all be stored in the same place — different types of data have different requirements and different storage solutions have different strengths and features.

At a fundamental level, most digital data is stored on HDDs or SSDs. HDDs, or hard disk drives, are mechanical devices that store the data on a spinning magnetic surface and move read/write heads over that surface to access the data. They currently max out around 200MB/s and 5ms latency.

SSDs, or solid-state drives, involve no moving parts. SSDs can be built with a number of different architectures and interfaces, but most are based on the same basic Flash memory technology as the CF or SD card in your camera. Some SSDs are SATA drives that use the same interface and form factor as a spinning disk for easy replacement in existing HDD-compatible devices. These devices are limited to SATA’s bandwidth of 600MB/s. Other SSDs use the PCIe interface, either in full-sized PCIe cards or the smaller M.2 form factor. These have much higher potential bandwidths, up to 3000MB/s.

Currently, HDDs are much cheaper for storing large quantities of data but require some level of redundancy for security. SSDs are also capable of failure, but it is a much more rare occurrence. Data recovery for either is very expensive. SSDs are usually cheaper for achieving high bandwidth, unless large capacities are also needed.

RAIDs
Traditionally, hard drives used in professional contexts are grouped together for higher speeds and better data security. These are called RAIDs, which stands for redundant array of independent disks. There are a variety of different approaches to RAID design that are very different from one another.

RAID-0 or striping is technically not redundant, but every file is split across each disk, so each disk only has to retrieve its portion of a requested file. Since these happen in parallel, the result is usually faster than if a single disk had read the entire file, especially for larger files. But if one disk fails, every one of your files will be missing a part of its data, making the remaining partial information pretty useless. The more disks in the array, the higher the chances of one failing, so I rarely see striped arrays composed of more than four disks. It used to be popular to create striped arrays for high-speed access to restorable data, like backed-up source footage, or temp files, but now a single PCIe SSD is far faster, cheaper, smaller and more efficient in most cases.

Sugar

RAID-1 or mirroring is when all of the data is written to more than one drive. This limits the array’s capacity to the size of the smallest source volume, but the data is very secure. There is no speed benefit to writes since each drive must write all of the data, but reads can be distributed across the identical drives with similar performance as RAID-0.

RAID-3, -5 and -6 try to achieve a balance between those benefits for larger arrays with more disks (minimum three). They all require more complicated controllers, so they are more expensive for the same levels of performance. RAID-3 stripes data across all but one drive and then calculates and stores parity (odd/even) data across the data drives and stores it on the last drive. This allows the data from any single failed drive to be restored, based on the parity data. RAID-5 is similar, but the parity volume is alternated depending on the block, allowing the reads to be shared across all disks, not just the “data drives.”

So the capacity of a RAID-3 or RAID-5 array will be the minimum individual disk capacity times the number of disks minus one. RAID-6 is similar but stores two drives worth of parity data, which via some more advanced math than odd/even, allows it to restore the data even if two drives fail at the same time. RAID-6 capacity will be the minimum individual disk capacity times the number of disks minus two, and is usually only used on arrays with many disks. RAID-5 is the most popular option for most media storage arrays, although RAID-6 becomes more popular as the value of the data stored increases and the price of extra drives decreases over time.

Storage Bandwidth
Digital data is stored as a series of ones and zeros, each of which is a bit. One byte is 8 bits, which frequently represents one letter of text, or one pixel of an image (8-bit single channel). Bits are frequently referenced in large quantities to measure data rates, while bytes are usually referenced when measuring stored data. I prefer to use bytes for both purposes, but it is important to know the difference. A Megabit (Mb) is one million bits, while a Megabyte (MB) is one million bytes, or 8 million bits. Similar to metric, Kilo is thousand, Mega is million, Giga is billion, and Tera is trillion. Anything beyond that you can learn as you go.

Networking speeds are measured in bits (Gigabits), but with headers and everything else, it is safer to divide by 10 when converting speed into bytes per second. Estimate 100MB/s for Gigabit, up to 1000MB/s on 10GB, and around 500MB/s for the new N-BaseT standard. Similarly, when transferring files over a 30Mb Internet connection, expect around 3MB/s, then multiple by 60 or 3,600 to get to minutes or hours (180MB/min or 9600MB/hr in this case). So if you have to download a 10GB file on that connection, come back to check on it in an hour.

Magnopus

Because networking standards are measured in bits, and because networking is so important for sharing video files, many video file types are measured in bits as well. An 8Mb H.264 stream is 1MB per second. DNxHD36 is 36Mb/s (or 4.5MB/s when divided by eight), DV and HDV are 25Mb, DVCProHD is 100Mb, etc. Other compression types have variable bit rates depending on the content, but there are still average rates we can make calculations from. Any file’s size divided by its duration will reveal its average data rate. It is important to make sure that your storage has the bandwidth to handle as many streams of video as you need, which will be that average data rate times the number of streams. So 10 streams of DNxHD36 will be 360Mb or 45MB/s.

The other issue to account for is IO requests and drive latency. Lots of small requests require not just high total transfer rates, but high IO performance as well. Hard drives can only fulfill around 100 individual requests per second, regardless of how big those requests are. So while a single drive can easily sustain a 45MB/s stream, satisfying 10 different sets of requests may keep it so busy bouncing between the demands that it can’t keep up. You may need a larger array, with a higher number of (potentially) smaller disks to keep up with the IO demands of multiple streams of data. Audio is worse in this regard in that you are dealing with lots of smaller individual files as your track count increases, even though the data rate is relatively low. SSDs are much better at handling larger numbers of individual requests, usually measured in the thousands or tens of thousands per second per drive.

Storage Capacity
Capacity on the other hand is simpler. Megabytes are usually the smallest increments of data that we have to worry about calculating. A media type’s data rate (in MB/sec) times its duration (in seconds) will give you its expected file size. If you are planning to edit a feature film with 100 hours of offline content in DNxHD36, that is 3600×100 seconds, times 4.5MB/s, equaling 1620000MB, 1620GB, or simply about 1.6TB. But you should add some headroom for unexpected needs, and then a 2TB disk is about 1.8TB when formatted, so it will just barely fit. It is probably worth sizing up to at least 3TB if you are planning to store your renders and exports on there as well.

Once you have a storage solution of the required capacity there is still the issue of connecting it to your system. The most expensive options connect through the network to make them easier to share (although more is required for true shared storage), but that isn’t actually the fastest option or the cheapest. A large array can be connected over USB3 or Thunderbolt, or via the SATA or SAS protocol directly to an internal controller.

There are also options for Fibre Channel, which can allow sharing over a SAN, but this is becoming less popular as 10GbE becomes more affordable. Gigabit Ethernet and USB3 won’t be fast enough for high-bandwidth files to playback, but 10GbE, multichannel SAS, Fibre Channel and Thunderbolt can all handle almost anything up to uncompressed 4K.

Direct attached storage will always have the highest bandwidth and lowest latency, as it has the fewest steps between the stored files and the user. Using Thunderbolt or USB adds another controller and hop, Ethernet even more so.

Different Types of Project Data
Now that we know the options for storage, let’s look at the data we anticipate needing to store. First off we will have lots of video footage of source media (either camera original files, transcoded editing dailies, or both). This is usually in the Terabytes, but the data rates vary dramatically — from 1Mb H.264 files to 200Mb ProRes files to 2400Mb Red files. The data rate for the files you are playing back, combined with the number of playback streams you expect to use, will determine the bandwidth you need from your storage system. These files are usually static in that they don’t get edited or written to in any way after creation.

The exceptions would be sidecar files like RMD and XML files, which will require write access to the media volume. If a certain set of files is static, as long as a backup of the source data exists, they don’t need to be backed up on a regular basis and don’t even necessarily need redundancy. Although if the cost of restoring that data would be high, in regards to lost time during that process, some level of redundancy is still recommended.

Another important set of files we will have is our project files, which actually record the “work” we do in our application. They contain instructions for manipulating our media files during playback or export. The files are usually relatively small, and are constantly changing as we use them. That means they need to be backed up on a regular basis. The more frequent the backups, the less work you lose when something goes wrong.

We will also have a variety of exports and intermediate renders over the course of the project. Whether they are flattened exports for upload and review, VFX files or other renders, these are a more dynamic set of files than our original source footage. And they are generated on our systems instead of being imported from somewhere else. These can usually be regenerated from their source projects, if necessary, but the time and effort required usually makes it worth it to invest in protecting or backing them up. In most workflows, these files don’t change once they are created, which makes it easier to back them up if desired.

There will also be a variety of temp files generated by most editing or VFX programs. Some of these files need high-speed access for best application performance, but they rarely need to be protected or backed up because they can be automatically regenerated by the source applications on the fly if needed.

Choosing the Right Storage for Your Needs
Ok, so we have source footage, project files, exports and temp files that we need to find a place for. If you have a system or laptop with a single data volume, the answer is simple: It all goes on the C drive. But we can achieve far better performance if we have the storage infrastructure to break those files up onto different devices. Newer laptops frequently have both a small SSD and a larger hard disk. In that case we would want our source footage on the (larger) HDD, while the project files should go on the (safer) SSD.

Usually your temp file directories should be located on the SSD as well since it is faster, and your exports can go either place, preferably the SSD if they fit. If you have an external drive of source footage connected, you can back all files up there, but you should probably work from projects stored on the local system, playing back media from the external drive.

A professional workstation can have a variety of different storage options available. I have a system with two SSDs and two RAIDs, so I store my OS and software on one SSD, my projects and temp files on the other SSD, my source footage on one RAID and my exports on the other. I also back up my project folder to the exports RAID on a daily basis, since the SSDs have no redundancy.

Individual Store Solution Case Study Examples
If you are natively editing a short film project shot on Red, then R3Ds can be 300MB/s. That is 1080GB/hour, so five hours of footage will be just over 5TB. It could be stored on a single 6TB external drive, but that won’t give you the bandwidth to play back in real-time (hard drives usually top out around 200MB/s).

Striping your data across two drives in one of those larger external drives would probably provide the needed performance, but with that much data you are unlikely to have a backup elsewhere. So data security becomes more of a concern, leading us toward a RAID-5-based solution. A four-disk array of 2TB drives provides 6TB of usable storage at RAID-5 (4x2TB = 8TB raw capacity, minus 2TB of parity data equals 6TB of usable storage capacity). Using an array of 8 1TB drives would provide higher performance, and 7TB of space before formatting (8x1TB = 8TB raw capacity, minus 1TB of parity, because a single drive failure would only lose 1TB of data in this configuration) but will cost more. (eight-port RAID controller, eight-bay enclosure, and two 1TB drives are usually more expensive than one 2TB drive.)

Larger projects deal with much higher numbers. Another project has 200TB of Red footage that needs to be accessible on a single volume. A 24-bay enclosure with 12TB drives provides 288TB of space, minus two drives worth of data for RAID-6 redundancy (288TB raw-[2x12TB for parity]=264TB usable capacity), which will be more like 240TB of space available in Windows once it is formatted.

Sharing Storage and Files With Others
As Ethernet networking technology has improved, the benefits of expensive SAN (storage area network) solutions over NAS (network attached storage) solutions has diminished. 10Gigabit Ethernet (10GbE) transfers over 1GB of data a second and is relatively cheap to implement. NAS has the benefit of a single host system controlling the writes, usually with software included in the OS. This prevents data corruption and also isolates the client devices from the file system allowing PC, Mac and Linux devices to all access the same files. This comes at the cost of slightly increased latency and occasionally lower total bandwidth, but the prices and complexity of installation are far lower.

So now all but the largest facilities and most demanding workflows are being deployed with NAS-based shared storage solutions. This can be as simple as a main editing system with a large direct attached array sharing its media with an assistant station, over a direct 10GbE link, for about $50. This can be scaled up by adding a switch and connecting more users to it, but the more users sharing the data, the greater the impact on the host system, and the lower the overall performance. Over 3-4 users, it becomes prudent to have a dedicated host system for the storage, for both performance and stability. Once you are buying a dedicated system, there are a variety of other functionalities offered by different vendors to improve performance and collaboration.

Bin Locking and Simultaneous Access
The main step to improve collaboration is to implement what is usually referred to as a “bin locking system.” Even with a top-end SAN solution and strict permissions controls there is still the possibility of users overwriting each other’s work, or at the very least branching the project into two versions that can’t easily be reconciled.

If two people are working on the same sequence at the same time, only one of their sets of changes is going to make it to the master copy of the file without some way of combining the changes (and solutions are being developed). But usually the way to avoid that is to break projects down into smaller pieces and make sure that no two people are ever working on the exact same part. This is accomplished by locking the part (or bin) of the project that a user is editing so that no one else may edit it at the same time. This usually requires some level of server functionality because it involves changes that are not always happening at the local machine.

Avid requires specific support for that from the storage host in order for it to enable that feature. Adobe on the other hand has implemented a simpler storage-based solution, which is effective but not infallible, that works on any shared storage device that offers users write access.

A Note on iSCSI
iSCSI arrays offer some interesting possibilities for read-only data, like source footage, as iSCSI gives block-level access for maximum performance and runs on any network without expensive software. The only limit is that only one system can copy new media to the volume, and there must be a secure way to ensure the remaining systems have read-only access. Projects and exports must be stored elsewhere, but those files require much less capacity and bandwidth than source media. I have not had the opportunity to test out this hybrid SAN theory since I don’t have iSCSI appliances to test with.

A Note on Top-End Ethernet Options
40Gb Ethernet products have been available for a while and we are now seeing 25GB and 100Gb Ethernet products as well. 40Gb cards can be gotten quite cheaply, and I was tempted to use them for direct connect, hoping to see 4GB/s to share fast SSDs between systems. But 40Gb Ethernet is actually a trunk of four parallel 10Gb links and each individual connection is limited to 10Gb. It is easy to share the 40Gb of aggregate bandwidth across 10 systems accessing a 40Gb storage host, but very challenging to get more than 10Gb to a single client system. Having extra lanes on the highway doesn’t get you to work any faster if there are no other cars on the road, it only helps when there is lots of competing traffic.

25Gb Ethernet on the other hand will give you access to nearly 3GB/s for single connections, but as that is newer technology, the prices haven’t come down yet ($500 instead of $50 for a 10GbE direct link). 100Gb Ethernet is four 25Gb links trunked together, and subject to the same aggregate limitations as 40Gb.

Main Image: Courtesy of Sugar Studios LA


Mike McCarthy is an online editor/workflow consultant with 10 years of experience on feature films and commercials. He has been involved in pioneering new solutions for tapeless workflows, DSLR filmmaking and multi-screen and surround video experiences. Check out his site.

Adobe Max 2018: Creative Cloud updates and more

By Mike McCarthy

I attended my first Adobe Max 2018 last week in Los Angeles. This huge conference takes over the LA convention center and overflows into the surrounding venues. It began on Monday morning with a two-and-a-half-hour keynote outlining the developments and features being released in the newest updates to Adobe’s Creative Cloud. This was followed by all sorts of smaller sessions and training labs for attendees to dig deeper into the new capabilities of the various tools and applications.

The South Hall was filled with booths from various hardware and software partners, with more available than any one person could possibly take in. Tuesday started off with some early morning hands-on labs, followed by a second keynote presentation about creative and career development. I got a front row seat to hear five different people, who are successful in their creative fields — including director Ron Howard — discuss their approach to work and life. The rest of the day was so packed with various briefings, meetings and interviews that I didn’t get to actually attend any of the classroom sessions.

By Wednesday, the event was beginning to wind down, but there was still a plethora of sessions and other options for attendees to split their time. I presented the workflow for my most recent project Grounds of Freedom at Nvidia’s booth in the community pavilion, and spent the rest of the time connecting with other hardware and software partners who had a presence there.

Adobe released updates for most of its creative applications concurrent with the event. Many of the most relevant updates to the video tools were previously announced at IBC in Amsterdam last month, so I won’t repeat those, but there are still a few new video ones, as well as many that are broader in scope in regards to media as a whole.

Adobe Premiere Rush
The biggest video-centric announcement is Adobe Premiere Rush, which offers simplified video editing workflows for mobile devices and PCs.  Currently releasing on iOS and Windows, with Android to follow in the future, it is a cloud-enabled application, with the option to offload much of the processing from the user device. Rush projects can be moved into Premiere Pro for finishing once you are back on the desktop.  It will also integrate with Team Projects for greater collaboration in larger organizations. It is free to start using, but most functionality will be limited to subscription users.

Let’s keep in mind that I am a finishing editor for feature films, so my first question (as a Razr-M user) was, “Who wants to edit video on their phone?” But what if the user shot the video on their phone? I don’t do that, but many people do, so I know this will be a valuable tool. This has me thinking about my own mentality toward video. I think if I was a sculptor I would be sculpting stone, while many people are sculpting with clay or silly putty. Because of that I would have trouble sculpting in clay and see little value in tools that are only able to sculpt clay. But there is probably benefit to being well versed in both.

I would have no trouble showing my son’s first-year video compilation to a prospective employer because it is just that good — I don’t make anything less than that. But there was no second-year video, even though I have the footage because that level of work takes way too much time. So I need to break free from that mentality, and get better at producing content that is “sufficient to tell a story” without being “technically and artistically flawless.” Learning to use Adobe Rush might be a good way for me to take a step in that direction. As a result, we may eventually see more videos in my articles as well. The current ones took me way too long to produce, but Adobe Rush should allow me to create content in a much shorter timeframe, if I am willing to compromise a bit on the precision and control offered by Premiere Pro and After Effects.

Rush allows up to four layers of video, with various effects and 32-bit Lumetri color controls, as well as AI-based audio filtering for noise reduction and de-reverb and lots of preset motion graphics templates for titling and such.  It should allow simple videos to be edited relatively easily, with good looking results, then shared directly to YouTube, Facebook and other platforms. While it doesn’t fit into my current workflow, I may need to create an entirely new “flow” for my personal videos. This seems like an interesting place to start, once they release an Android version and I get a new phone.

Photoshop Updates
There is a new version of Photoshop released nearly every year, and most of the time I can’t tell the difference between the new and the old. This year’s differences will probably be a lot more apparent to most users after a few minutes of use. The Undo command now works like other apps instead of being limited to toggling the last action. Transform operates very differently, in that they made proportional transform the default behavior instead of requiring users to hold Shift every time they scale. It allows the anchor point to be hidden to prevent people from moving the anchor instead of the image and the “commit changes” step at the end has been removed. All positive improvements, in my opinion, that might take a bit of getting used to for seasoned pros. There is also a new Framing Tool, which allows you to scale or crop any layer to a defined resolution. Maybe I am the only one, but I frequently find myself creating new documents in PS just so I can drag the new layer, that is preset to the resolution I need, back into my current document. For example, I need a 200x300px box in the middle of my HD frame — how else do you do that currently? This Framing tool should fill that hole in the features for more precise control over layer and object sizes and positions (As well as provide its easily adjustable non-destructive masking.).

They also showed off a very impressive AI-based auto selection of the subject or background.  It creates a standard selection that can be manually modified anywhere that the initial attempt didn’t give you what you were looking for.  Being someone who gives software demos, I don’t trust prepared demonstrations, so I wanted to try it for myself with a real-world asset. I opened up one of my source photos for my animation project and clicked the “Select Subject” button with no further input and got this result.  It needs some cleanup at the bottom, and refinement in the newly revamped “Select & Mask” tool, but this is a huge improvement over what I had to do on hundreds of layers earlier this year.  They also demonstrated a similar feature they are working on for video footage in Tuesday night’s Sneak previews.  Named “Project Fast Mask,” it automatically propagates masks of moving objects through video frames and, while not released yet, it looks promising.  Combined with the content-aware background fill for video that Jason Levine demonstrated in AE during the opening keynote, basic VFX work is going to get a lot easier.

There are also some smaller changes to the UI, allowing math expressions in the numerical value fields and making it easier to differentiate similarly named layers by showing the beginning and end of the name if it gets abbreviated.  They also added a function to distribute layers spatially based on the space between them, which accounts for their varying sizes, compared to the current solution which just evenly distributes based on their reference anchor point.

In other news, Photoshop is coming to iPad, and while that doesn’t affect me personally, I can see how this could be a big deal for some people. They have offered various trimmed down Photoshop editing applications for iOS in the past, but this new release is supposed to be based on the same underlying code as the desktop version and will eventually replicate all functionality, once they finish adapting the UI for touchscreens.

New Apps
Adobe also showed off Project Gemini, which is a sketch and painting tool for iPad that sits somewhere between Photoshop and Illustrator. (Hence the name, I assume) This doesn’t have much direct application to video workflows besides being able to record time-lapses of a sketch, which should make it easier to create those “white board illustration” videos that are becoming more popular.

Project Aero is a tool for creating AR experiences, and I can envision Premiere and After Effects being critical pieces in the puzzle for creating the visual assets that Aero will be placing into the augmented reality space.  This one is the hardest for me to fully conceptualize. I know Adobe is creating a lot of supporting infrastructure behind the scenes to enable the delivery of AR content in the future, but I haven’t yet been able to wrap my mind around a vision of what that future will be like.  VR I get, but AR is more complicated because of its interface with the real world and due to the variety of forms in which it can be experienced by users.  Similar to how web design is complicated by the need to support people on various browsers and cell phones, AR needs to support a variety of use cases and delivery platforms.  But Adobe is working on the tools to make that a reality, and Project Aero is the first public step in that larger process.

Community Pavilion
Adobe’s partner companies in the Community Pavilion were showing off a number of new products.  Dell has a new 49″ IPS monitor, the U4919DW, which is basically the resolution and desktop space of two 27-inch QHD displays without the seam (5120×1440 to be exact). HP was displaying their recently released ZBook Studio x360 convertible laptop workstation, (which I will be posting a review of soon), as well as their Zbook X2 tablet and the rest of their Z workstations.  NVidia was exhibiting their new Turing-based cards with 8K Red decoding acceleration, ray tracing in Adobe Dimension and other GPU accelerated tasks.  AMD was demoing 4K Red playback on a MacBookPro with an eGPU solution, and CPU based ray-tracing on their Ryzen systems.  The other booths spanned the gamut from GoPro cameras and server storage devices to paper stock products for designers.  I even won a Thunderbolt 3 docking station at Intel’s booth. (Although in the next drawing they gave away a brand new Dell Precision 5530 2-in-1 convertible laptop workstation.)   Microsoft also garnered quite a bit of attention when they gave away 30 MS Surface tablets near the end of the show.  There was lots to see and learn everywhere I looked.

The Significance of MAX
Adobe MAX is quite a significant event, especially now that I have been in the industry long enough to start to see the evolution of certain trends — things are not as static as we may expect.  I have attended NAB for the last 12 years, and the focus of that show has shifted significantly away from my primary professional focus. (No Red, Ncidia, or Apple booths, among many other changes)  This was the first year that I had the thought “I should have gone to Sundance,” and a number of other people I know had the same impression. Adobe Max is similar, although I have been a little slower to catch on to that change.  It has been happening for over ten years, but has grown dramatically in size and significance recently.  If I still lived in LA, I probably would have started attending sooner, but it was hardly on my radar until three weeks ago.  Now that I have seen it in person, I probably won’t miss it in the future.


Mike McCarthy is an online editor/workflow consultant with 10 years of experience on feature films and commercials. He has been involved in pioneering new solutions for tapeless workflows, DSLR filmmaking and multi-screen and surround video experiences. Check out his site.

The benefits of LTO

By Mike McCarthy

LTO stands for Linear Tape Open, and was initially developed nearly 20 years ago as an “open” format technology that allows manufacturing by any vendor that wishes to license the technology. It records any digital files onto half-inch magnetic tapes, stored in square single reel cartridges. The capacity started at 100GB and has increased by a factor of two nearly every generation; the most recent LTO-8 cartridges store 12TB of uncompressed data.

If you want to find out more about LTO, you should check out the LTO Consortium, which is made up of Hewlett Packard Enterprises, IBM and Quantum, although there are other companies that make LTO drives and tape cartridges. You might be familiar with their LTO Ultrium logo.

‘Tapeless’ Workflows
While initially targeting server markets, with the introduction of “tapeless workflows” in the media and entertainment industry, there became a need for long-term media storage. Since the first P2 cards and SxS sticks were too expensive for single write operations, they were designed to be reused repeatedly once their contents had been offloaded to hard drives. But hard drives are not ideal for long-term data storage, and insurance and bonding companies wanted their clients to have alternate data archiving solutions.

So, by the time the Red One and Canon 5D were flooding post facilities with CF cards, LTO had become the default archive solution for most high-budget productions. But this approach was not without limitations and pitfalls. The LTO archiving solutions being marketed at the time were designed around the Linux-based Tar system of storing files, while most media work is done on Windows and Mac OS X. Various approaches were taken by different storage vendors to provide LTO capabilities to M&E customers. Some were network appliances running Linux under the hood, while others wrote drivers and software to access the media from OS X or, in one case, Windows. Then there was the issue that Tar isn’t a self-describing file system, so you needed a separate application to keep track of what was on each tape in your library. All of these aspects cost lots of money, so the initial investment was steep, even though the margin cost of tape cartridges was the cheapest way to store data per GB.

LTFS
Linear Tape File System (LTFS) was first introduced with LTO-5 and was intended to make LTO tapes easier to use and interchange between systems. A separate partition on the tape stores the index of data in XML and other associated metadata. It was intended to be platform independent, although it took a while for reliable drivers and software to be developed for use in Windows and OS X.

At this point, LTFS-formatted tapes in LTO tape drives operate very similarly to old 3.5-inch floppy drives. You insert a cartridge, it makes some funny noises, and then after a minute it asks you to format a new tape, or it displays the current contents of the tape as a new drive letter. If you drag files into that drive, it will start copying the data to the tape, and you can hear it grinding away. The biggest difference is when you hit eject it will take the computer a minute or two to rewind the tape, write the updated index to the first partition and then eject the cartridge for you. Otherwise it is a seamless drag and drop, just like any other removable data storage device.

LTO Drives
All you need in order to use LTO in your media workflow — for archive or data transfer — is an LTO drive. I bought one last year on Amazon for $1,600, which was a bit of a risk considering that I didn’t know if I was going to be able to get it to work on my Windows 7 desktop. As far as I know, all tape drives are SAS devices, although you can buy ones that have adapted the SAS interface to Thunderbolt or Fibre Channel.

Most professional workstations have integrated SAS controllers, so internal LTO drives fit into a 5.25-inch bay and can just connect to those, or any SAS card. External LTO drives usually use Small Form Factor cables (SFF-8088) to connect to the host device. Internal SAS ports can be easily adapted to SFF-8088 ports, or a dedicated eSAS PCIe card can be installed in the system.

Capacity & Compression
How much data do LTO tapes hold? This depends on the generation… and the compression options. The higher capacity advertised on any LTO product assumes a significant level of data compression, which may be achievable with uncompressed media files (DPX, TIFF, ARRI, etc.) The lower value advertised is the uncompressed data capacity, which is the more accurate estimate of how much data it will store. This level of compression is achieved using two different approaches, eliminating redundant data segments and eliminating the space between files. LTO was originally designed for backing up lots of tiny files on data servers, like credit card transactions or text data, and those compression approaches don’t always apply well to large continuous blocks of unique data found in encoded video.

Using data compression on media files which are already stored in a compressed codec doesn’t save much space (there is little redundancy in the data, and few gaps between individual files).

Uncompressed frame sequences, on the other hand, can definitely benefit from LTO’s hardware data compression. Regardless of compression, I wouldn’t count on using the full capacity of each cartridge. Due to the way the drives are formatted, and the way storage vendors measure data, I have only been able to copy 2.2TB of data from Windows onto my 2.5TB LTO-6 cartridges. So keep that in mind when estimating real-world capacity, like with any other data storage medium.

Choosing the ‘Right’ Version to Use
So which generation of LTO is the best option? That depends on how much data you are trying to store. Since most media files that need to be archived these days are compressed, either as camera source footage or final deliverables, I will be calculating based on the uncompressed capacities. VFX houses using DPX frames, or vendors using DCDMs might benefit from calculating based on the compressed capacities.

Prices are always changing, especially for the drives, but these are the numbers as of summer 2018. On the lowest end, we have LTO-5 drives available online for $600-$800, which will probably store 1-1.2TB of data on a $15 tape. So if you have less than 10TB of data to backup at a time, that might be a cost-effective option. Any version lower than LTO-5 doesn’t support the partitioning required for LTFS, and is too small to be useful in modern workflows anyway.

As I mentioned earlier, I spent $1,600 on an LTO-6 drive last year, and while that price is still about the same, LTO-7 and LTO-8 drives have come down in cost since then. My LTO-6 drive stores about 2.2TB of data per $23 tape. That allowed me to backup 40TB of Red footage onto 20 tapes in 90 hours, or an entire week. Now I am looking at using the same drive to ingest 250TB of footage from a production in China, but that would take well over a month, so LTO-6 is not the right solution for that project. But the finished deliverables will probably be a similar 10TB set of DPX and TIFF files, so LTO-6 will still be relevant for that application.

I see prices as low as $2,200 for LTO-7 drives, so they aren’t much more expensive than LTO-6 drives at this point, but the 6TB tapes are. LTO-7 switched to a different tape material, which increased the price of the media. At $63 they are just over $10 per TB, but that is higher than the two previous generations.

LTO-8 drives are available for as low as $2,600, and store up to 12TB on a single $160 tape. LTO-8 drives can also write up to 9TB onto a properly formatted LTO-7 tape in a system called “LTO-7 Type M” This is probably the cheapest cost per TB approach at the moment, since 9TB on a $63 tape is $7/TB.

Compatibility Between Generations
One other consideration is backwards compatibility. What will it take to read your tapes back in the future? The standard for LTO has been that drives can write the previous generation tapes and read tapes from two generations back.

So if you invested in an LTO-2 drive and have tons of tapes, they will still work when you upgrade to an LTO-4 drive. You can then copy them to newer cartridges with the same hardware at a 4:1 ratio since the capacity will have doubled twice. The designers probably figured that after two generations (about five years) most data will have been restored at some point, or be irrelevant (the difference between backups and archives).

If you need your media archived longer than that, it would probably be wise to transfer it to fresh media of a newer generation to ensure it is readable in the future. The other issue is transfer if you are using LTO cartridges to move data from one place to another. You must use the same generation of tape and be within one generation to go both ways. If I want to send data to someone who has an LTO-5 drive, I have to use an LTO-5 tape, but I can copy the data to the tape with my LTO-6 drive (and be subject to the LTO-5 capacity and performance limits). If they then sent that LTO-5 tape to someone with an LTO-7 drive, they would be able to read the data, but wouldn’t be able to write to the tape. The only exception to this is that the LTO-8 drives won’t read LTO-6 tapes (of course, because I have a bunch of LTO-6 tapes now, right?).

So for my next 250TB project, I have to choose between a new LTO-7 drive with backwards compatibility to my existing gear or an LTO-8 drive that can fit 50% more data on a $63 cartridge, and use the more expensive 12TB ones as well. Owning both LTO-6 and LTO-8 drives would allow me to read or write to any LTFS cartridge (until LTO-9 is released), but the two drives couldn’t exchange tapes with each other.

Automated Backup Software & Media Management
I have just been using HPE’s free StoreOpen Utility to operate my internal LTO drive and track what files I copy to which tapes. There are obviously much more expensive LTO-based products, both in hardware with robotic tape libraries and in software with media and asset management programs and automated file backup solutions.

I am really just exploring the minimum investment that needs to be made to take advantage of the benefits of LTO tape, for manually archiving your media files and backing up your projects. The possibilities are endless, but the threshold to start using LTO is much lower than it used to be, especially with the release of LTFS support.


Mike McCarthy is an online editor/workflow consultant with 10 years of experience on feature films and commercials. He has been involved in pioneering new solutions for tapeless workflows, DSLR filmmaking and multi-screen and surround video experiences. Check out his site.

Review: HP DreamColor Z31x studio display for cinema 4K

By Mike McCarthy

Not long ago, HP sent me their newest high-end monitor to review, and I was eager to dig in. The DreamColor Z31x studio display is a 31-inch true 4K color-critical reference monitor. It has many new features that set it apart from its predecessors, which I have examined and will present here in as much depth as I can.

It is challenging to communicate the nuances of color quality through writing or any other form on the Internet, as some things can only be truly appreciated firsthand. But I will attempt to communicate the experience of using the new DreamColor as best I can.

First, we will start with a little context…

Some DreamColor History
HP revolutionized the world of color-critical displays with the release of the first DreamColor in June 2008. The LP2480zx was a 24-inch 1920×1200 display that had built-in color processing with profiles for standard color spaces and the ability to calibrate it to refine those profiles as the monitor aged. It was not the first display with any of these capabilities, but the first one that was affordable, by at least an order of magnitude.

It became very popular in the film industry, both sitting on desks in post facilities — as it was designed — and out in the field as a live camera monitor, which it was not designed for. It had a true 10-bit IPS pane and the ability to reproduce incredible detail in the darks. It could only display 10-bit sources from the brand-new DisplayPort input or the HDMI port, and the color gamut remapping only worked for non-interlaced RGB sources.

So many people using the DreamColor as a “video monitor” instead of a “computer monitor” weren’t even using the color engine — they were just taking advantage of the high-quality panel. It wasn’t just the color engine but the whole package, including the price, that led to its overwhelming success. This was helped by the lack of better options, even at much higher price points, since this was the period after CRT production ended but before OLED panels had reached the market. This was similar to (and in the same timeframe as) Canon’s 5D MarkII revolutionizing the world of independent filmmaking with its HDSLRs. The combination gave content creators amazing tools for moving into HD production at affordable price points.

It took six years for HP to release an update to the original model DreamColor in the form of the Z27x and Z24x. These had the same color engine but different panel technology. They never had the same impact on the industry as the original, because the panels didn’t “wow” people, and the competition was starting to catch up. Dell has PremierColor and Samsung and BenQ have models featuring color accuracy as well. The Z27x could display 4K sources by scaling them to its native 2560×1440 resolution, while the Z24x’s resolution was decreased to 1920×1080 with a panel that was even less impressive.

Fast forward a few more years, and the Z24x was updated to Gen2, and the Z32x was released with UHD resolution. This was four times the resolution of the original DreamColor and at half the price. But with lots of competition in the market, I don’t think it has had the reach of the original DreamColor, and the industry has matured to the point where people aren’t hooking them to 4K cameras because there are other options better suited to that environment, specifically battery powered OLED units.

DreamColor at 4K
Fast forward a bit and HP has released the Z31x DreamColor studio display. The big feature that this unit brings to the table is true cinema 4K resolution. The label 4K gets thrown around a lot these days, but most “4K” products are actually UHD resolution, at 3840×2160, instead of the full 4096×2160. This means that true 4K content is scaled to fit the UHD screen, or in the case of Sony TVs, cropped off the sides. When doing color critical work, you need to be able to see every pixel, with no scaling, which could hide issues. So the Z31x’s 4096×2160 native resolution will be an important feature for anyone working on modern feature films, from editing and VFX to grading and QC.

The 10-bit 4K Panel
The true 10-bit IPS panel is the cornerstone of what makes a DreamColor such a good monitor. IPS monitor prices have fallen dramatically since they were first introduced over a decade ago, and some of that is the natural progression of technology, but some of that has come at the expense of quality. Most displays offering 10-bit color are accomplishing that by flickering the pixels of an 8-bit panel in an attempt to fill in the remaining gradations with a technique called frame rate control (FRC). And cheaper panels are as low as 6-bit color with FRC to make them close to 8-bit. There are a variety of other ways to reduce cost with cheaper materials, and lower-quality backlights.

HP claims that the underlying architecture of this panel returns to the quality of the original IPS panel designs, but then adds the technological advances developed since then, without cutting any corners in the process. In order to fully take advantage of the 10-bit panel, you need to feed it 10-bit source content, which is easier than it used to be but not a forgone conclusion. Make sure you select 10-bit output color in your GPU settings.

In addition to a true 10-bit color display, it also natively refreshes at the rate of the source image, from 48Hz-60Hz, because displaying every frame at the right time is as important as displaying it in the right color. They say that the darker blacks are achieved by better crystal alignment in the LCD (Liquid Crystal Display) blocking out the backlight more fully. This also gives a wider viewing angle, since washing out the blacks is usually the main issue with off-axis viewing. I can move about 45 degrees off center, vertically or horizontally, without seeing any shift in the picture brightness or color. Past that I start to see the mid levels getting darker.

Speaking of brighter and darker, the backlight gives the display a native brightness of 250 nits. That is over twice the brightness needed to display SDR content, but this not an HDR display. It can be adjusted anywhere from 48 to 250 nits, depending on the usage requirements and environment. It is not designed to be the brightest display available, it is aiming to be the most accurate.

Much effort was put into the front surface, to get the proper balance of reducing glare and reflections as much as possible. I can’t independently verify some of their other claims without a microscope and more knowledge than I currently have, but I can easily see that the matte surface of the display is much better than other monitors in regards to fewer reflections and less glare for the surrounding environment, allowing you to better see the image on the screen. That is one of the most apparent strengths of the monitor, obviously visible at first glance.

Color Calibration
The other new headline feature is an integrated colorimeter for display calibration and verification, located in the top of the bezel. It can swing down and measure the color parameters of the true 10-bit IPS panel, to adjust the color space profiles, allowing the monitor to more accurately reproduce colors. This is a fully automatic feature, independent of any software or configuration on the host computer system. It can be controlled from the display’s menu interface, and the settings will persist between multiple systems. This can be used to create new color profiles, or optimize the included ones for DCI P3, BT.709, BT.2020, sRGB and Adobe RGB. It also includes some low-blue-light modes for use as an interface monitor, but this negates its color accurate functionality. It can also input and output color profiles and all other configuration settings through USB and its network connection.

The integrated color processor also supports using external colorimeters and spectroradiometers to calibrate the display, and even allows the integrated XYZ colorimeter itself to be calibrated by those external devices. And this is all accomplished internally in the display, independent of using any software on the workstation side. The supported external devices currently include:
– Klein Instruments: K10, K10-A (colorimeters)
– Photo Research: PR-655, PR-670, PR-680, PR-730, PR-740, PR-788 (spectroradiometers)
– Konica Minolta: CA-310 (colorimeter)
– X-Rite: i1Pro 2 (spectrophotometer), i1Display (colorimeter)
– Colorimetry Research: CR-250 (spectroradiometer)

Inputs and Ports
There are five main display inputs on the monitor: two DisplayPort 1.2, two HDMI 2.0 and one DisplayPort over USB-C. All support HDCP and full 4K resolution at up to 60 frames per second. It also has an 1/8-inch sound jack and a variety of USB options. There are four USB 3.0 ports that are shared via KVM switching technology between the USB-C host connection and a separate USB-B port to a host system. These are controlled by another dedicated USB keyboard port, giving the monitor direct access to the keystrokes. There are two more USB ports that connect to the integrated DreamColor hardware engine, for connecting external calibration instruments, and for loading settings from USB devices.

My only complaint is that while the many USB ports are well labeled, the video ports are not. I can tell which ones are HDMI without the existing labels, but what I really need is to know which one the display views as HDMI1 and which is HDMI2. The Video Input Menu doesn’t tell you which inputs are active, which is another oversight, given all of the other features they added to ease the process of sharing the display between multiple inputs. So I recommend labeling them yourself.

Full-Screen Monitoring Features
I expect the Z31x will most frequently be used as a dedicated full-resolution playback monitor, and HP has developed a bunch of new features that are very useful and applicable for that use case. The Z31x can overlay mattes (with variable opacity) for Flat and Scope cinema aspect ratios (1.85 and 2.39). It also can display onscreen markers for those sizes, as well as 16×9 or 3×4, including action and title safe, including further options for center and thirds markers with various colors available. The markers can be further customized with HP’s StudioCal.XML files. I created a preset that gives you 2.76:1 aspect ratio markers that you are welcome to download and use or modify. These customized XMLs are easy to create and are loaded automatically when you insert a USB stick containing them into the color engine port.

The display also gives users full control over the picture scaling, and has a unique 2:1 pixel scaling for reviewing 2K and HD images at pixel-for-pixel accuracy. It also offers compensation for video levels and overscan and controls for de-interlacing, cadence detection, panel overdrive and blue-channel-only output. You can even control the function of each bezel button, and their color and brightness. These image control features will definitely be significant to professional users in the film and video space. Combined with the accurate reproduction of color, resolution and frame rate, this makes for an ideal display for monitoring nearly any film or video content at the highest level of precision.

Interface Display Features
Most people won’t be using this as an interface monitor, due to the price and because the existing Z32x should suffice when not dealing with film content at full resolution. Even more than the original DreamColor, I expect it will primarily be used as a dedicated full-screen playback monitor and users will have other displays for their user interface and controls. That said, HP has included some amazing interface and sharing functionality in the monitor, integrating a KVM switch for controlling two systems on any of the five available inputs. They also have picture-in-picture and split screen modes that are both usable and useful. HD or 2K input can be displayed at full resolution over any corner of the 4K master shot.

The split view supports two full-resolution 2048×2160 inputs side by side and from separate sources. That resolution has been added as a default preset for the OS to use in that mode, but it is probably only worth configuring for extended use. (You won’t be flipping between full screen and split very easily in that mode.) The integrated KVM is even more useful in these configurations. It can also scale any other input sizes in either mode but at a decrease in visual fidelity.

HP has included every option that I could imagine needing for sharing a display between two systems. The only problem is that I need that functionality on my “other” monitor for the application UI, not on my color critical review monitor. When sharing a monitor like this, I would just want to be able to switch between inputs easily to always view them at full screen and full resolution. On a related note, I would recommend using DisplayPort over HDMI anytime you have a choice between the two, as HDMI 2.0 is pickier about 18Gb cables, occasionally preventing you from sending RGB input and other potential issues.

Other Functionality
The monitor has an RJ-45 port allowing it to be configured over the network. Normally, I would consider this to be overkill but with so many features to control and so many sub-menus to navigate through, this is actually more useful than it would be on any other display. I found myself wishing it came with a remote control as I was doing my various tests, until I realized the network configuration options would offer even better functionality than a remote control would have. I should have configured that feature first, as it would have made the rest of the tests much easier to execute. It offers simple HTTP access to the controls, with a variety of security options.

I also had some issues when using the monitor on a switched power outlet on my SmartUPS battery backup system, so I would recommend using an un-switched outlet whenever possible. The display will go to sleep automatically when the source feed is shut off, so power saving should be less of an issue that other peripherals.

Pricing and Options
The DreamColor Z31x is expected to retail for $4,000 in the US market. If that is a bit out of your price range, the other option is the new Z27x G2 for half of that price. While I have not tested it myself, I have been assured that the newly updated 27-inch model has all of the same processing functionality, just in a smaller form-factor, with a lower-resolution panel. The 2560×1440 panel is still 10-bit, with all of the same color and frame rate options, just at a lower resolution. They even plan to support scaling 4K inputs in the next firmware update, similar to the original Z27x.

The new DreamColor studio displays are top-quality monitors, and probably the most accurate SDR monitors in their price range. It is worth noting that with a native brightness of 250 nits, this is not an HDR display. While HDR is an important consideration when selecting a forward-looking display solution, there is still a need for accurate monitoring in SDR, regardless of whether your content is HDR compatible. And the Z31x would be my first choice for monitoring full 4K images in SDR, regardless of the color space you are working in.


Mike McCarthy is an online editor/workflow consultant with 10 years of experience on feature films and commercials. He has been involved in pioneering new solutions for tapeless workflows, DSLR filmmaking and multi-screen and surround video experiences. Check out his site.

Testing large format camera workflows

By Mike McCarthy

In the last few months, we have seen the release of the Red Monstro, Sony Venice, Arri Alexa LF and Canon C700 FF, all of which have larger or full-frame sensors. Full frame refers to the DSLR terminology, with full frame being equivalent to the entire 35mm film area — the way that it was used horizontally in still cameras. All SLRs used to be full frame with 35mm film, so there was no need for the term until manufacturers started saving money on digital image sensors by making them smaller than 35mm film exposures. Super35mm motion picture cameras on the other hand ran the film vertically, resulting in a smaller exposure area per frame, but this was still much larger than most video imagers until the last decade, with 2/3-inch chips being considered premium imagers. The options have grown a lot since then.

L-R: 1st AC Ben Brady, DP Michael Svitak and Mike McCarthy on the monitor.

Most of the top-end cinema cameras released over the last few years have advertised their Super35mm sensors as a huge selling point, as that allows use of any existing S35 lens on the camera. These S35 cameras include the Epic, Helium and Gemini from Red, Sony’s F5 and F55, Panasonic’s VaricamLT, Arri’s Alexa and Canon’s C100-500. On the top end, 65mm cameras like the Alexa65 have sensors twice as wide as Super35 cameras, but very limited lens options to cover a sensor that large. Full frame falls somewhere in between and allows, among other things, use of any 35mm still film lenses. In the world of film, this was referred to as Vista Vision, but the first widely used full-frame digital video camera was Canon’s 5D MkII, the first serious HDSLR. That format has suddenly surged in popularity recently, and thanks to this I recently had opportunity to be involved in a test shoot with a number of these new cameras.

Keslow Camera was generous enough to give DP Michael Svitak and myself access to pretty much all their full-frame cameras and lenses for the day in order to test the cameras, workflows and lens options for this new format. We also had the assistance of first AC Ben Brady to help us put all that gear to use, and Mike’s daughter Florendia as our model.

First off was the Red Monstro, which while technically not the full 24mm height of true full frame, uses the same size lenses due to the width of its 17×9 sensor. It offers the highest resolution of the group at 8K. It records compressed RAW to R3D files, as well as options for ProRes and DNxHR up to 4K, all saved to Red mags. Like the rest of the group, smaller portions of the sensor can be used at lower resolution to pair with smaller lenses. The Red Helium sensor has the same resolution but in a much smaller Super35 size, allowing a wider selection of lenses to be used. But larger pixels allow more light sensitivity, with individual pixels up to 5 microns wide on the Monstro and Dragon, compared to Helium’s 3.65-micron pixels.

Next up was Sony’s new Venice camera with a 6K full-frame sensor, allowing 4K S35 recording as well. It records XAVC to SxS cards or compressed RAW in the X-OCN format with the optional ASX-R7 external recorder, which we used. It is worth noting that both full-frame recording and integrated anamorphic support require additional special licenses from Sony, but Keslow provided us with a camera that had all of that functionality enabled. With a 36x24mm 6K sensor, the pixels are 5.9microns, and footage shot at 4K in the S35 mode should be similar to shooting with the F55.

We unexpectedly had the opportunity to shoot on Arri’s new AlexaLF (Large Format) camera. At 4.5K, this had the lowest resolution, but that also means the largest sensor pixels at 8.25microns, which can increase sensitivity. It records ArriRaw or ProRes to Codex XR capture drives with its integrated recorder.

Another other new option is the Canon C700 FF with a 5.9K full-frame sensor recording RAW, ProRes, or XAVC to CFast cards or Codex Drives. That gives it 6-micron pixels, similar to the Sony Venice. But we did not have the opportunity to test that camera this time around, maybe in the future.

One more factor in all of this is the rising popularity of anamorphic lenses. All of these cameras support modes that use the part of the sensor covered by anamorphic lenses and can desqueeze the image for live monitoring and preview. In the digital world, anamorphic essentially cuts your overall resolution in half, until the unlikely event that we start seeing anamorphic projectors or cameras with rectangular sensor pixels. But the prevailing attitude appears to be, “We have lots of extra resolution available so it doesn’t really matter if we lose some to anamorphic conversion.”

Post Production
So what does this mean for post? In theory, sensor size has no direct effect on the recorded files (besides the content of them) but resolution does. But we also have a number of new formats to deal with as well, and then we have to deal with anamorphic images during finishing.

Ever since I got my hands on one of Dell’s new UP3218K monitors with an 8K screen, I have been collecting 8K assets to display on there. When I first started discussing this shoot with DP Michael Svitak, I was primarily interested in getting some more 8K footage to use to test out new 8K monitors, editing systems and software as it got released. I was anticipating getting Red footage, which I knew I could playback and process using my existing software and hardware.

The other cameras and lens options were added as the plan expanded, and by the time we got to Keslow Camera, they had filled a room with lenses and gear for us to test with. I also had a Dell 8K display connected to my ingest system, and the new 4K DreamColor monitor as well. This allowed me to view the recorded footage in the highest resolution possible.

Most editing programs, including Premiere Pro and Resolve, can handle anamorphic footage without issue, but new camera formats can be a bigger challenge. Any RAW file requires info about the sensor pattern in order to debayer it properly, and new compression formats are even more work. Sony’s new compressed RAW format for Venice, called X-OCN, is supported in the newest 12.1 release of Premiere Pro, so I didn’t expect that to be a problem. Its other recording option is XAVC, which should work as well. The Alexa on the other hand uses ArriRaw files, which have been supported in Premiere for years, but each new camera shoots a slightly different “flavor” of the file based on the unique properties of that sensor. Shooting ProRes instead would virtually guarantee compatibility but at the expense of the RAW properties. (Maybe someday ProResRAW will offer the best of both worlds.) The Alexa also has the challenge of recording to Codex drives that can only be offloaded in OS X or Linux.

Once I had all of the files on my system, after using a MacBook Pro to offload the media cards, I tried to bring them into Premiere. The Red files came in just fine but didn’t play back smoothly over 1/4 resolution. They played smoothly in RedCineX with my Red Rocket-X enabled, and they export respectably fast in AME, (a five-minute 8K anamorphic sequence to UHD H.265 in 10 minutes), but for some reason Premiere Pro isn’t able to get smooth playback when using the Red Rocket-X. Next I tried the X-OCN files from the Venice camera, which imported without issue. They played smoothly on my machine but looked like they were locked to half or quarter res, regardless of what settings I used, even in the exports. I am currently working with Adobe to get to the bottom of that because they are able to play back my files at full quality, while all my systems have the same issue. Lastly, I tried to import the Arri files from the AlexaLF, but Adobe doesn’t support that new variation of ArriRaw yet. I would anticipate that will happen soon, since it shouldn’t be too difficult to add that new version to the existing support.

I ended up converting the files I needed to DNxHR in DaVinci Resolve so I could edit them in Premiere, and I put together a short video showing off the various lenses we tested with. Eventually, I need to learn how to use Resolve more efficiently, but the type of work I usually do lends itself to the way Premiere is designed — inter-cutting and nesting sequences with many different resolutions and aspect ratios. Here is a short clip demonstrating some of the lenses we tested with:

This is a web video, so even at UHD it is not meant to be an analysis of the RAW image quality, but instead a demonstration of the field of view and overall feel with various lenses and camera settings. The combination of the larger sensors and the anamorphic lenses leads to an extremely wide field of view. The table was only about 10 feet from the camera, and we can usually see all the way around it. We also discovered that when recording anamorphic on the Alexa LF, we were recording a wider image than was displaying on the monitor output. You can see in the frame grab below that the live display visible on the right side of the image isn’t displaying the full content that got recorded, which is why we didn’t notice that we were recording with the wrong settings with so much vignetting from the lens.

We only discovered this after the fact, from this shot, so we didn’t get the opportunity to track down the issue to see if it was the result of a setting in the camera or in the monitor. This is why we test things before a shoot, but we didn’t “test” before our camera test, so these things happen.

We learned a lot from the process, and hopefully some of those lessons are conveyed here. A big thanks to Brad Wilson and the rest of the guys at Keslow Camera for their gear and support of this adventure and, hopefully, it will help people better prepare to shoot and post with this new generation of cameras.

Main Image: DP Michael Svitak


Mike McCarthy is an online editor/workflow consultant with 10 years of experience on feature films and commercials. He has been involved in pioneering new solutions for tapeless workflows, DSLR filmmaking and multi-screen and surround video experiences. Check out his site.

postPerspective names NAB Impact Award MVPs and winners

NAB is a bear. Anyone who has attended this show can attest to that. But through all the clutter, postPerspective sought to seek out the best of the best for our Impact Awards. So we turned to a panel of esteemed industry pros (to whom we are very grateful!) to cast their votes on what they thought would be most impactful to their day-to-day workflows, and those of their colleagues.

In addition to our Impact Award winners, this year we are also celebrating two pieces of technology that not only caused a big buzz around the show, but are also bringing things a step further in terms of technology and workflow: Blackmagic’s DaVinci Resolve 15 and Apple’s ProRes RAW.

With ProRes RAW, Apple has introduced a new, high-quality video recording codec that has already been adopted by three competing camera vendors — Sony, Canon and Panasonic. According to Mike McCarthy, one of our NAB bloggers and regular contributors, “ProRes RAW has the potential to dramatically change future workflows if it becomes even more widely supported. The applications of RAW imaging in producing HDR content make the timing of this release optimal to encourage vendors to support it, as they know their customers are struggling to figure out simpler solutions to HDR production issues.”

Fairlight’s audio tools are now embedded in the new Resolve 15.

With Resolve 15, Blackmagic has launched the product further into a wide range of post workflows, and they haven’t raised the price. This standalone app — which comes in a free version — provides color grading, editing, compositing and even audio post, thanks to the DAW Fairlight, which is now built into the product.

These two technologies are Impact Award winners, but our judges felt they stood out enough to be called postPerspective Impact Award MVPs.

Our other Impact Award winners are:

• Adobe for Creative Cloud

• Arri for the Alexa LF

• Codex for Codex One Workflow and ColorSynth

• FilmLight for Baselight 5

• Flanders Scientific for the XM650U monitor

• Frame.io for the All New Frame.io

• Shift for their new Shift Platform

• Sony for their 8K CLED display

In a sea of awards surrounding NAB, the postPerspective Impact Awards stand out, and are worth waiting for, because they are voted on by working post professionals.

Flanders Scientific’s XM650U monitor.

“All of these technologies from NAB are very worthy recipients of our postPerspective Impact Awards,” says Randi Altman, postPerspective’s founder and editor-in-chief. “These awards celebrate companies that push the boundaries of technology to produce tools that actually have an impact on workflows as well as the ability to make users’ working lives easier and their projects better. This year we have honored 10 different products that span the production and post pipeline.

“We’re very proud of the fact that companies don’t ‘submit’ for our awards,” continues Altman. “We’ve tapped real-world users to vote for the Impact Awards, and they have determined what could be most impactful to their day-to-day work. We feel it makes our awards quite special.”

With our Impact Awards, postPerspective is also hoping to help those who weren’t at the show, or who were unable to see it all, with a starting point for their research into new gear that might be right for their workflows.

postPerspective Impact Awards are next scheduled to celebrate innovative product and technology launches at SIGGRAPH 2018.

NAB Day 2 thoughts: AJA, Sharp, QNAP

By Mike McCarthy

During my second day walking the show floor at NAB, I was able to follow up a bit more on a few technologies that I found intriguing the day before.

AJA released a few new products and updates at the show. Their Kumo SDI switchers now have options supporting 12G SDI, but their Kona cards still do not. The new Kona 1 is a single channel of 3G SDI in and out, presumably to replace the aging Kona LHe since analog is being phased out in many places.

There is also a new Kona HDMI, which just has four dedicated HDMI inputs for streaming and switching. This will probably be a hit with people capturing and streaming competitive video gaming. Besides a bunch of firmware updates to existing products, they are showing off the next step in their partnership with ColorFront in the form of a 1RU HDR image analyzer. This is not a product I need personally, but I know it will have an important role to fill as larger broadcast organizations move into HDR production and workflows.

Sharp had an entire booth dedicated to 8K video technologies and products. They were showing off 8Kp120 playback on what I assume is a prototype system and display. They also had 8K broadcast-style cameras on display in operation, outputting Quad 12G SDI that eventually fed an 8K TV with Quad HDMI. They also had a large curved video wall, composed of eight individual 2Kx5K panels. It obviously had large seams, but it had a more immersive feel that the LED based block walls I see elsewhere.

I was pleasantly surprised to discover that NAS vendor QNAP has released a pair of 10GbE switches, with both SFP+ and RJ45 ports. I was quoted a price under $600, but I am not sure if that was for the eight- or 12-port version. Either way, that is a good deal for users looking to move into 10GbE, with three to 10 clients — two clients can just direct connect. It also supports the new NBASE-T standard that connects at 2.5Gb or 5Gb instead of 10Gb, depending on the cables and NICs involved in the link. It is of course compatible with 1Gb and 100Mb connections as well.

On a related note, the release of 25GbE PCIe NICs allows direct connections between two systems to be much faster, for not much more cost than previous 10GbE options. This is significant for media production workflows, as uncompressed 4K requires slightly more bandwidth than 10GbE provides. I also learned all sorts of things about the relationship between 10GbE and its quad-channel variant 40GbE, which with the newest implementations is 25GbE, allowing 100GbE when four channels are combined.

I didn’t previously know that 40GbE ports and 100GB ports on switches could be broken into four independent connections with just a splitter cable, which offers some very interesting infrastructure design options — especially as facilities move towards IP video workflows, and SDI over IP implementations and products.


Mike McCarthy is an online editor/workflow consultant with 10 years of experience on feature films and commercials. He has been involved in pioneering new solutions for tapeless workflows, DSLR filmmaking and multi-screen and surround video experiences. Check out his site.

NAB First Thoughts: Fusion in Resolve, ProRes RAW, more

By Mike McCarthy

These are my notes from the first day I spent browsing the NAB Show floor this year in Las Vegas. When I walked into the South Lower Hall, Blackmagic was the first thing I saw. And, as usual, they had a number of new products this year. The headline item is the next version of DaVinci Resolve, which now integrates the functionality of their Fusion visual effects editor within the program. While I have never felt Resolve to be a very intuitive program for my own work, it is a solution I recommend to others who are on a tight budget, as it offers the most functionality for the price, especially in the free version.

Blackmagic Pocket Cinema Camera

The Blackmagic Pocket Cinema Camera 4K looks more like a “normal” MFT DSLR camera, although it is clearly designed for video instead of stills. Recording full 4K resolution in RAW or ProRes to SD or CFast cards, it has a mini-XLR input with phantom power and uses the same LP-E6 battery as my Canon DSLR. It uses the same camera software as the Ursa line of devices and includes a copy of Resolve Studio… for $1,300.  If I was going to be shooting more live-action video anytime soon, this might make a decent replacement for my 70D, moving up to 4K and HDR workflows. I am not as familiar with the Panasonic cameras that it is closely competes with in the Micro Four Thirds space.

AMD Radeon

Among other smaller items, Blackmagic’s new UpDownCross HD MiniConverter will be useful outside of broadcast for manipulating HDMI signals from computers or devices that have less control over their outputs. (I am looking at you, Mac users.) For $155, it will help interface with projectors and other video equipment. At $65, the bi-directional MicroConverter will be a cheaper and simpler option for basic SDI support.

AMD was showing off 8K editing in Premiere Pro, the result of an optimization by Adobe that uses the 2TB SSD storage in AMD’s Radeon Pro SSG graphics card to cache rendered frames at full resolution for smooth playback. This change is currently only applicable to one graphics card, so it will be interesting to see if Adobe did this because it expects to see more GPUs with integrated SSDs hit the market in the future.

Sony is showing crystal light emitting diode technology in the form of a massive ZRD video wall of incredible imagery. The clarity and brightness were truly breathtaking, but obviously my camera rendered to the web hardly captures the essence of what they were demonstrating.

Like nearly everyone else at the show, Sony is also pushing HDR in the form of Hybrid Log Gamma, which they are developing into many of their products. They also had an array for their tiny RX0 cameras on display with this backpack rig from Radiant Images.

ProRes RAW
At a higher level, one of the most interesting things I have seen at the show is the release of ProRes RAW. While currently limited to external recorders connected to cameras from Sony, Panasonic and Canon, and only supported in FCP-X, it has the potential to dramatically change future workflows if it becomes more widely supported. Many people confuse RAW image recording with the log gamma look, or other low-contrast visual interpretations, but at its core RAW imaging is a single-channel image format paired with a particular bayer color pattern specific to the sensor it was recorded with.

This decreases the amount of data to store (or compress) and gives access to the “source” before it has been processed to improve visual interpretation — in the form of debayering and adding a gamma curve to reverse engineer the response pattern of the human eye, compared to mechanical light sensors. This provides more flexibility and processing options during post, and reduces the amount of data to store, even before the RAW data is compressed, if at all. There are lots of other compressed RAW formats available; the only thing ProRes actually brings to the picture is widespread acceptance and trust in the compression quality. Existing compressed RAW formats include R3D, CinemaDNG, CineformRAW and Canon CRM files.

None of those caught on as a widespread multi-vendor format, but this ProRes RAW is already supported by systems from three competing camera vendors. And the applications of RAW imaging in producing HDR content make the timing of this release optimal to encourage vendors to support it, as they know their customers are struggling to figure out simpler solutions to HDR production issues.

There is no technical reason that ProRes RAW couldn’t be implemented on future Arri, Red or BMD cameras, which are all currently capable of recording ProRes and RAW data (but not the combination, yet). And since RAW is inherently a playback-only format, (you can’t alter a RAW image without debayering it), I anticipate we will see support in other applications, unless Apple wants to sacrifice the format in an attempt to increase NLE market share.

So it will be interesting to see what other companies and products support the format in the future, and hopefully it will make life easier for people shooting and producing HDR content.


Mike McCarthy is an online editor/workflow consultant with 10 years of experience on feature films and commercials. He has been involved in pioneering new solutions for tapeless workflows, DSLR filmmaking and multi-screen and surround video experiences. Check out his site.

A Closer Look: Why 8K?

By Mike McCarthy

As we enter 2018, we find a variety of products arriving to market that support 8K imagery. The 2020 Olympics are slated to be broadcast in 8K, and while clearly we have a way to go, innovations are constantly being released that get us closer to making that a reality.

The first question that comes up when examining 8K video gear is, “Why 8K?” Obviously, it provides more resolution, but that is more of an answer to the how question than the why question. Many people will be using 8K imagery to create projects that are finished at 4K, giving them the benefits of oversampling or re-framing options. Others will use the full 8K resolution on high DPI displays. There is also the separate application of using 8K images in 360 video for viewing in VR headsets.

Red Monstro 8K

Similar technology may allow reduced resolution extraction on-the-fly to track an object or person in a dedicated 1080p window from an 8K master shot, whether that is a race car or a basketball player. The benefit compared to tracking them with the camera is that these extractions can be generated for multiple objects simultaneously, allowing viewers to select their preferred perspective on the fly. So there are lots of uses for 8K imagery. Shooting 8K for finishing in 4K is not much different from a workflow perspective than shooting 5K or 6K, so we will focus on workflows and tools that actually result in an 8K finished product.

8K Production
The first thing you need for 8K video production is an 8K camera. There are a couple of options, the most popular ones being from Red. The Weapon 8K came out in 2016, followed by the smaller sensor Helium8K, and the recently announced Monstro8K. Panavision has the DXL, which by my understanding is really a derivation of the Red Dragon8K sensor. Canon has been demoing an 8K camera for two years now, with no released product that I am aware of. Sony announced the 8K 3-chip camera UHC-8300 at IBC 2017, but that is probably out of most people’s price range. Those are the only major options I am currently aware of, and the Helium8K is the only one I have been able to shoot with and edit footage from.

Sony UHC-8300 8K

Moving 8K content around in realtime is a challenge. DisplayPort 1.3 supports 8K at 30p, with dual cables being used for 60p. HDMI 2.1 will eventually allow devices to support 8K video on a single cable as well. (The HDMI 2.1 specification was just released at the end of November, so it will be a while before we see it implemented in products on the market. DisplayPort 1.4 exists today — GPUs, Dell monitor — while HDMI 2.1 only exists on paper and in CES technology demos.) Another approach is to use multiple parallel channels for 12G SDI, similar to how quad 3G SDI can be used to transmit 4K data. It is more likely that by the time most facilities are pushing around lots of realtime 8K content, they will have moved to video IP, and be using compression to move 8K streams on 10GbE networks, or moving uncompressed 8K content on 40Gb or 100Gb networks.

Software
The next step is the software part, which is in pretty good shape. Most high-end applications are already set for 8K, because high resolutions are already used as backplates and for other unique uses, and because software is the easiest part of allowing higher resolutions. I have edited 8K files in Adobe Premiere Pro in a variety of flavors without issue. Both Avid Media Composer and Blackmagic Resolve claim to support 8K content. Codec-wise, there are already lots of options for storing 8K, including DNxHR, Cineform, JPEG2000 and HEVC/H265, among many others.

Blackmagic DeckLink 8K Pro

The hardware to process those files in realtime is a much greater challenge, but we are just seeing the release of Intel’s next generation of high-end computing chips. The existing gear is just at the edge of functional at 8K, so I expect the new systems to make 8K editing and playback a reality at the upper end. Blackmagic has announced the DeckLink 8K Pro, a PCIe card with quad 12G SDI ports. I suspect that AJA’s new Io 4K Plus may support 8K at some point in the future, with quad bidirectional 12G SDI ports. Thunderbolt 3 is the main bandwidth limitation there, but it should do 4:2:2 at 24p or 30p. I am unaware of any display that can take that yet, but I am sure they are coming.

In regards to displays, the only one commercially available is Dell’s UP3218K monitor running on dual DisplayPort 1.4 cables. It looks amazing, but you won’t be able to hook it up to your 8K camera for live preview very easily. An adapter is a theoretical possibility, but I haven’t heard of any being developed. Most 8K assets are being recorded to be used in 4K projects, so the output and display at 8K aren’t as big of a deal. Most people will have their needs met with existing 4K options, with the 8K content giving them the option to reframe their shot without losing resolution.

Dell UP3218K

Displaying 8K content at 4K is a much simpler proposition with current technology. Many codecs allow for half-res decode, which makes the playback requirements similar to 4K at full resolution. While my dual-processor desktop workstation can playback most any intermediate codec at half resolution for 4K preview, my laptop seems like a better test-bed to evaluate the fractional resolution playback efficiency of various codecs at 8K, so that will be one of my next investigations.

Assuming you want to show your content at the full 8K, how do you deliver it to your viewers? H.264 files are hard-limited to 4K, but HEVC (or H.265) allows 8K files to be encoded and decoded at reasonable file sizes, and is hardware-accelerated on the newest GPU cards. So 8K HEVC playback should be possible on shipping mid- and high-end computers, provided that you have a display to see it on. 8K options will continue to grow as TV makers push to set apart their top-of-the-line models, and that will motivate development of the rest of the ecosystem to support them.


Mike McCarthy is an online editor/workflow consultant with 10 years of experience on feature films and commercials. He has been involved in pioneering new solutions for tapeless workflows, DSLR filmmaking and multi-screen and surround video experiences. Check out his site.

Review: Dell’s 8K LCD monitor

By Mike McCarthy

At CES 2017, Dell introduced its UP3218K LCD 32-inch monitor, which was the first commercially available 8K display. It runs 7680×4320 pixels at 60fps, driven by two DisplayPort 1.4 cables. That is over 33 million pixels per frame, and nearly 2 billion per second, which requires a lot of GPU power to generate. Available since March, not long ago I was offered one to review as part of a wider exploration of 8K video production workflows, and there will be more articles about that larger story in the near future.

For this review, I will be focusing on only this product and its uses.

The UP3218K showed up in a well-designed box that was easy to unpack — it was also easy getting the monitor onto the stand. I plugged it into my Nvidia Quadro P6000 card with the included DisplayPort cables, and it came up as soon as I turned it on… at full 60Hz and without any issues or settings to change. Certain devices with only one DisplayPort 1.4 connector will only power the display at 30Hz, as full 60Hz connections saturate the bandwidth of two DP 1.4 cables, but the display does require a Displayport 1.4 connection, and will not revert to lower resolution when connected to a 1.2 port. This limits the devices that can drive it to Pascal-based GPUs on the Nvidia side, or top-end Vega GPUs on the AMD side. I have a laptop with a P5000 in it, so I was disappointed to discover that the DisplayPort connector was still only 1.2, thereby making it incompatible with this 8K monitor.

Dell’s top Precision laptops (7720 and 7520) support DP1.4, while HP and Lenovo’s mobile workstations do not yet. This is a list of every device I am aware of that explicitly claims to support 8K output:
1. Quadro P6000, P5000, P4000, P2000 workstation GPU cards
2. TitanX and Geforce10 Series graphics cards
3. RadeonPro SSG, WX9100 & WX7100 workstation GPU cards
4. RX Vega 64 and 56 graphics cards
5. Dell Precision 7520 and 7720 mobile workstations
6. Comment if you know of other laptops with DP1.4

So once you have a system that can drive the monitor, what can you do with it? Most people reading this article will probably be using this display as a dedicated full-screen monitor for their 8K footage. But smooth 8K editing and playback is still a ways away for most people. The other option is to use it as your main UI monitor to control your computer and its applications. In either case, color can be as important as resolution when it comes to professional content creation, and Dell has brought everything it has to the table in this regard as well.

The display supports Dell’s PremierColor toolset, which is loosely similar to the functionality that HP offers under their DreamColor branding. PremierColor means a couple of things, including that the display has the internal processing power that allows it to correctly emulate different color spaces; it can also be calibrated with an X-Rite iDisplay Pro independent of the system driving it. It also interfaces with a few software tools that Dell has developed for its professional users. The mo

st significant functionality within that feature set is the factory-calibrated options for emulating AdobeRGB, sRGB, Rec.709 and DCI-P3. Dell tests each display individually after manufacturing to ensure that it is color accurate. These are great features, but they are not unique to this monitor, and many users have been using them on other display models for the last few years. While color accuracy is important, the main selling point of this particular model is resolution, and lots of it. And that is what I spent the majority of my time analyzing.

Resolution
The main issue here is the pixel density. Ten years ago, 24-inch displays were 1920×1200, and 30-inch displays had 2560×1600 pixels. This was around 100 pixels per inch, and most software was hard coded to look correct at that size. When UHD displays were released, the 32-inch version had a DPI of 140. That resulted in applications looking quite small and hard to read on the vast canvas of pixels, but this trend increased pressure on software companies to scale their interfaces better for high DPI displays. Windows 7 was able to scale things up an extra 50%, but a lot of applications ignored that setting or were not optimized for it. Windows 10 now allows scaling beyond 300%, which effectively triples the size of the text and icons. We have gotten to the point where even 15-inch laptops have UHD screens, resulting in 280 DPI, which is unreadable to most people without interface scaling.

Premiere Pro

With 8K resolution, this monitor has 280 DPI, twice that of a 4K display of similar size. This is on par with a 15-inch UHD laptop screen, but laptops are usually viewed from a much closer range. Since I am still using Windows 7 on my primary workstation, I was expecting 280 DPI to be unusable for effective work. And while everything is undoubtedly small, it is incredibly crisp, and once I enabled Windows scaling at 150%, it was totally usable (although I am used to small fonts and lots of screen real estate). The applications I use, especially Adobe CC, scale much smoother than they used to, so everything looks great, even with Windows 7, as long as I sit fairly close to the monitor.

I can edit 6K footage in Premiere Pro at full resolution for the first time, with space left over for my timeline and tool panels. In After Effects, I can work on 4K shots in full resolution and still have 70 layers of data visible in my composition. In Photoshop, setting the UI to 200% causes the panel to behave similar to a standard 4K 32-inch display, but with your image having four times the detail. I can edit my 5.6K DSLR files in full resolution, with nearly every palette open to work smoothly through my various tools.

This display replaces my 34-inch curved U3415W as my new favorite monitor for Adobe apps, although I would still prefer the extra-wide 34-inch display for gaming and other general usability. But for editing or VFX work, the 8K panel is a dream come true. Every tool is available at the same time, and all of your imagery is available at HiDPI quality.

Age of Empires II

When gaming, the resolution doesn’t typically affect the field of view of 3D applications, but for older 2D games, you can see the entire map at once. Age of Empires II HD offers an expansive view of really small units, but there is a texture issue with the background of the bottom quarter of the screen. I think I used to see this at 4K as well, and it got fixed in an update, so maybe the same thing will happen with this one, once 8K becomes more common.

I had a similar UI artifact issue in RedCine player when I full-screened the Window on the 8K display, which was disappointing since that was one of the few ways to smoothly play 8K footage on the monitor at full resolution. Using it as a dedicated output monitor works as well, but I did run into some limitations. I did eventually get it to work with RedCine-X Pro, after initially experiencing some aspect ratio issues. It would playback cached frames smoothly, but only for 15 seconds at a time before running out of decoded frames, even with a Rocket-X accelerator card.

When configured as a secondary display for dedicated full-screen output, it is accessible via Mercury Transmit in the Adobe apps. This is where it gets interesting, because the main feature that this monitor brings to the table is increased resolution. While that is easy to leverage in Photoshop, it is very difficult to drive that many pixels in real-time for video work, and decreasing the playback resolution negates the benefit of having an 8K display. At this point, effectively using the monitor becomes more an issue of workflow.

After Effects

I was going to use 8K Red footage for my test, but that wouldn’t play smoothly in Premiere, even on my 20-core workstation, so I converted it to a variety of other files to test with. I created 8K test assets that matched the monitor resolution in DNxHR, Cineform, JPEG2000, OpenEXR and HEVC. DNxHR was the only format that offered full-resolution playback at 8K, and even that resulted in dropped frames on a regular basis. But being able to view 8K video is pretty impressive, and probably forever shifts my view of “sharp” in the subjective sense, but we are at a place where we are still waiting for the hardware to catch up in regards to processing power — for 8K video editing to be an effective reality for users.

Summing Up
The UP3218K is the ultimate monitor for content creators and artists looking for a large digital canvas, regardless of whether that is measured in inches or pixels. All those pixels come at a price — it is currently available from Dell for $3,900. Is it worth it? That will depend on what your needs and your budget are. Is a Mercedes Benz worth the increased price over a Honda? Some people obviously think so.

There is no question that this display and the hardware to drive it effectively would be a luxury to the average user. But for people who deal with high resolution content on a regular basis, the increased functionality that it offers them can’t be measured in the same way, and reading an article and seeing pictures online can’t compare to actually using the physical item. The screenshots are all scaled to 25% to be a reasonable size for the web. I am just trying to communicate a sense of the scope of the desktop real estate available to users on an 8K screen. So yes, it is expensive, but at the moment, it is the highest resolution monitor that money can buy, and the closest alternative (5K screens) does not even come close.


Mike McCarthy is an online editor/workflow consultant with 10 years of experience on feature films and commercials. He has been involved in pioneering new solutions for tapeless workflows, DSLR filmmaking and multi-screen and surround video experiences. Check out his site.

 

Review: GoPro Fusion 360 camera

By Mike McCarthy

I finally got the opportunity to try out the GoPro Fusion camera I have had my eye on since the company first revealed it in April. The $700 camera uses two offset fish-eye lenses to shoot 360 video and stills, while recording ambisonic audio from four microphones in the waterproof unit. It can shoot a 5K video sphere at 30fps, or a 3K sphere at 60fps for higher motion content at reduced resolution. It records dual 190-degree fish-eye perspectives encoded in H.264 to separate MicroSD cards, with four tracks of audio. The rest of the magic comes in the form of GoPro’s newest application Fusion Studio.

Internally, the unit is recording dual 45Mb H.264 files to two separate MicroSD cards, with accompanying audio and metadata assets. This would be a logistical challenge to deal with manually, copying the cards into folders, sorting and syncing them, stitching them together and dealing with the audio. But with GoPro’s new Fusion Studio app, most of this is taken care of for you. Simply plug-in the camera and it will automatically access the footage, and let you preview and select what parts of which clips you want processed into stitched 360 footage or flattened video files.

It also processes the multi-channel audio into ambisonic B-Format tracks, or standard stereo if desired. The app is a bit limited in user-control functionality, but what it does do it does very well. My main complaint is that I can’t find a way to manually set the output filename, but I can rename the exports in Windows once they have been rendered. Trying to process the same source file into multiple outputs is challenging for the same reason.

Setting Recorded Resolution (Per Lens) Processed Resolution (Equirectangular)
5Kp30 2704×2624 4992×2496
3Kp60 1568×1504 2880×1440
Stills 3104×3000 5760×2880

With the Samsung Gear 360, I researched five different ways to stitch the footage, because I wasn’t satisfied with the included app. Most of those will also work with Fusion footage, and you can read about those options here, but they aren’t really necessary when you have Fusion Studio.

You can choose between H.264, Cineform or ProRes, your equirectangular output resolution and ambisonic or stereo audio. That gives you pretty much every option you should need to process your footage. There is also a “Beta” option to stabilize your footage, which once I got used to it, I really liked. It should be thought of more as a “remove rotation” option since it’s not for stabilizing out sharp motions — which still leave motion blur — but for maintaining the viewer’s perspective even if the camera rotates in unexpected ways. Processing was about 6x run-time on my Lenovo Thinkpad P71 laptop, so a 10-minute clip would take an hour to stitch to 360.

The footage itself looks good, higher quality than my Gear 360, and the 60p stuff is much smoother, which is to be expected. While good VR experiences require 90fps to be rendered to the display to avoid motion sickness that does not necessarily mean that 30fps content is a problem. When rendering the viewer’s perspective, the same frame can be sampled three times, shifting the image as they move their head, even from a single source frame. That said, 60p source content does give smoother results than the 30p footage I am used to watching in VR, but 60p did give me more issues during editorial. I had to disable CUDA acceleration in Adobe Premiere Pro to get Transmit to work with the WMR headset.

Once you have your footage processed in Fusion Studio, it can be edited in Premiere Pro — like any other 360 footage — but the audio can be handled a bit differently. Exporting as stereo will follow the usual workflow, but selecting ambisonic will give you a special spatially aware audio file. Premiere can use this in a 4-track multi-channel sequence to line up the spatial audio with the direction you are looking in VR, and if exported correctly, YouTube can do the same thing for your viewers.

In the Trees
Most GoPro products are intended for use capturing action moments and unusual situations in extreme environments (which is why they are waterproof and fairly resilient), so I wanted to study the camera in its “native habitat.” The most extreme thing I do these days is work on ropes courses, high up in trees or telephone poles. So I took the camera out to a ropes course that I help out with, curious to see how the recording at height would translate into the 360 video experience.

Ropes courses are usually challenging to photograph because of the scale involved. When you are zoomed out far enough to see the entire element, you can’t see any detail, or if you are so zoomed in close enough to see faces, you have no good concept of how high up they are — 360 photography is helpful in that it is designed to be panned through when viewed flat. This allows you to give the viewer a better sense of the scale, and they can still see the details of the individual elements or people climbing. And in VR, you should have a better feel for the height involved.

I had the Fusion camera and Fusion Grip extendable tripod handle, as well as my Hero6 kit, which included an adhesive helmet mount. Since I was going to be working at heights and didn’t want to drop the camera, the first thing I did was rig up a tether system. A short piece of 2mm cord fit through a slot in the bottom of the center post and a triple fisherman knot made a secure loop. The cord fit out the bottom of the tripod when it was closed, allowing me to connect it to a shock-absorbing lanyard, which was clipped to my harness. This also allowed me to dangle the camera from a cord for a free-floating perspective. I also stuck the quick release base to my climbing helmet, and was ready to go.

I shot segments in both 30p and 60p, depending on how I had the camera mounted, using higher frame rates for the more dynamic shots. I was worried that the helmet mount would be too close, since GoPro recommends keeping the Fusion at least 20cm away from what it is filming, but the helmet wasn’t too bad. Another inch or two would shrink it significantly from the camera’s perspective, similar to my tripod issue with the Gear 360.

I always climbed up with the camera mounted on my helmet and then switched it to the Fusion Grip to record the guy climbing up behind me and my rappel. Hanging the camera from a cord, even 30-feet below me, worked much better than I expected. It put GoPro’s stabilization feature to the test, but it worked fantastically. With the camera rotating freely, the perspective is static, although you can see the seam lines constantly rotating around you. When I am holding the Fusion Grip, the extended pole is completely invisible to the camera, giving you what GoPro has dubbed “Angel View.” It is as if the viewer is floating freely next to the subject, especially when viewed in VR.

Because I have ways to view 360 video in VR, and because I don’t mind panning around on a flat screen view, I am less excited personally in GoPro’s OverCapture functionality, but I recognize it is a useful feature that will greater extend the use cases for this 360 camera. It is designed for people using the Fusion as a more flexible camera to produce flat content, instead of to produce VR content. I edited together a couple OverCapture shots intercut with footage from my regular Hero6 to demonstrate how that would work.

Ambisonic Audio
The other new option that Fusion brings to the table is ambisonic audio. Editing ambisonics works in Premiere Pro using a 4-track multi-channel sequence. The main workflow kink here is that you have to manually override the audio settings every time you import a new clip with ambisonic audio in order to set the audio channels to Adaptive with a single timeline clip. Turn on Monitor Ambisonics by right clicking in the monitor panel and match the Pan, Tilt, and Roll in the Panner-Ambisonics effect to the values in your VR Rotate Sphere effect (note that they are listed in a different order) and your audio should match the video perspective.

When exporting an MP4 in the audio panel, set Channels to 4.0 and check the Audio is Ambisonics box. From what I can see, the Fusion Studio conversion process compensates for changes in perspective, including “stabilization” when processing the raw recorded audio for Ambisonic exports, so you only have to match changes you make in your Premiere sequence.

While I could have intercut the footage at both settings together into a 5Kp60 timeline, I ended up creating two separate 360 videos. This also makes it clear to the viewer which shots were 5K/p30 and which were recorded at 3K/p60. They are both available on YouTube, and I recommend watching them in VR for the full effect. But be warned that they are recorded at heights up to 80 feet up, so it may be uncomfortable for some people to watch.

Summing Up
GoPro’s Fusion camera is not the first 360 camera on the market, but it brings more pixels and higher frame rates than most of its direct competitors, and more importantly it has the software package to assist users in the transition to processing 360 video footage. It also supports ambisonic audio and offers the OverCapture functionality for generating more traditional flat GoPro content.

I found it to be easier to mount and shoot with than my earlier 360 camera experiences, and it is far easier to get the footage ready to edit and view using GoPro’s Fusion Studio program. The Stabilize feature totally changes how I shoot 360 videos, giving me much more flexibility in rotating the camera during movements. And most importantly, I am much happier with the resulting footage that I get when shooting with it.


Mike McCarthy is an online editor/workflow consultant with 10 years of experience on feature films and commercials. He has been involved in pioneering new solutions for tapeless workflows, DSLR filmmaking and multi-screen and surround video experiences. Check out his site.

Making 6 Below for Barco Escape

By Mike McCarthy

There is new movie coming out this week that is fairly unique. Telling the true story of Eric LeMarque surviving eight days lost in a blizzard, 6 Below: Miracle on the Mountain is the first film shot and edited in its entirety for the new Barco Escape theatrical format. If you don’t know what Barco Escape is, you are about to find out.

This article is meant to answer just about every question you might have about the format and how we made the film, on which I was post supervisor, production engineer and finishing editor.

What is Barco Escape?
Barco Escape is a wraparound visual experience — it consists of three projection screens filling the width of the viewer’s vision with a total aspect ratio of 7.16:1. The exact field of view will vary depending on where you are sitting in the auditorium, but usually 120-180 degrees. Similar to IMAX, it is not about filling the entire screen with your main object but leaving that in front of the audience and letting the rest of the image surround them and fill their peripheral vision in a more immersive experience. Three separate 2K scope theatrical images play at once resulting in 6144×858 pixels of imagery to fill the room.

Is this the first Barco Escape movie?
Technically, four other films have screened in Barco Escape theaters, the most popular one being last year’s release of Star Trek Beyond. But none of these films used the entire canvas offered by Escape throughout the movie. They had up to 20 minutes of content on the side screens, but the rest of the film was limited to the center screen that viewers are used to. Every shot in 6 Below was framed with the surround format in mind, and every pixel of the incredibly wide canvas is filled with imagery.

How are movies created for viewing in Escape?
There are two approaches that can be used to fill the screen with content. One is to place different shots on each screen in the process of telling the story. The other is to shoot a wide enough field of view and high enough resolution to stretch a single image across the screens. For 6 Below, director Scott Waugh wanted to shoot everything at 6K, with the intention of filling all the screens with main image. “I wanted to immerse the viewer in Eric’s predicament, alone on the mountain.”

Cinematographer Michael Svitak shot with the Red Epic Dragon. He says, “After testing both spherical and anamorphic lens options, I chose to shoot Panavision Primo 70 prime lenses because of their pristine quality of the entire imaging frame.” He recorded in 6K-WS (2.37:1 aspect ratio at 6144×2592), framing with both 7:1 Barco Escape and a 2.76:1 4K extraction in mind. Red does have an 8:1 option and a 4:1 option that could work if Escape was your only deliverable. But since there are very few Escape theaters at the moment, you would literally be painting yourself into a corner. Having more vertical resolution available in the source footage opens up all sorts of workflow possibilities.

This still left a few challenges in post: to adjust the framing for the most comfortable viewing and to create alternate framing options for other deliverables that couldn’t use the extreme 7:1 aspect ratio. Other projects have usually treated the three screens separately throughout the conform process, but we treated the entire canvas as a single unit until the very last step, breaking out three 2K streams for the DCP encode.

What extra challenges did Barco Escape delivery pose for 6 Below’s post workflow?
Vashi Nedomansky edited the original 6K R3D files in Adobe Premiere Pro, without making proxies, on some maxed-out Dell workstations. We did the initial edit with curved ultra-wide monitors and 4K TVs. “Once Mike McCarthy optimized the Dell systems, I was free to edit the source 6K Red RAW files and not worry about transcodes or proxies,” he explains. “With such a quick turnaround everyday, and so much footage coming in, it was critical that I could jump on the footage, cut my scenes, see if they were playing well and report back to the director that same day if we needed additional shots. This would not have been possible time-wise if we were transcoding and waiting for footage to cut. I kept pushing the hardware and software, but it never broke or let me down. My first cut was 2 hours and 49 minutes long, and we played it back on one Premiere Pro timeline in realtime. It was crazy!”

All of the visual effects were done at the full shooting resolution of 6144×2592, as was the color grade. Once Vashi had the basic cut in place, there was no real online conform, just some cleanup work to do before sending it to color as an 8TB stack of 6K frames. At that point, we started examining it from the three-screen perspective with three TVs to preview it in realtime, courtesy of the Mosaic functionality built into Nvidia’s Quadro GPU cards. Shots were realigned to avoid having important imagery in the seams, and some areas were stretched to compensate for the angle of the side screens from the audiences perspective.

DP Michael Svitak and director Scott Waugh

Once we had the final color grade completed (via Mike Sowa at Technicolor using Autodesk Lustre), we spent a day in an Escape theater analyzing the effect of reflections between the screens and its effect on the contrast. We made a lot of adjustments to keep the luminance of the side screens from washing out the darks on the center screen, which you can’t simulate on TVs in the edit bay. “It was great to be able to make the final adjustments to the film in realtime in that environment. We could see the results immediately on all three screens and how they impacted the room,” says Waugh.

Once we added the 7.1 mix, we were ready to export assets for our delivery in many different formats and aspect ratios. Making the three streams for Escape playback was a simple as using the crop tool in Adobe Media Encoder to trim the sides in 2K increments.

How can you see movies in the Barco Escape format?
Barco maintains a list of theaters that have Escape screens installed, which can be found at ready2escape.com. But for readers in the LA area, the only opportunity to see a film in Barco Escape in the foreseeable future is to attend one of the Thursday night screenings of 6Below at the Regal LA Live Stadium or the Cinemark XD at Howard Hughes Center. There are other locations available to see the film in standard theatrical format, but as a new technology, Barco Escape is only available in a limited number of locations. Hopefully, we will see more Escape films and locations to watch them in the future.


Mike McCarthy is an online editor/workflow consultant with 10 years of experience on feature films and commercials. He has been involved in pioneering new solutions for tapeless workflows, DSLR filmmaking and multi-screen and surround video experiences. Check out his site.

GoPro intros Hero6 and its first integrated 360 solution, Fusion

By Mike McCarthy

Last week, I traveled to San Francisco to attend GoPro’s launch event for its new Hero6 and Fusion cameras. The Hero6 is the next logical step in the company’s iteration of action cameras, increasing the supported frame rates to 4Kp60 and 1080p240, as well as adding integrated image stabilization. The Fusion on the other hand is a totally new product for them, an action-cam for 360-degree video. GoPro has developed a variety of other 360-degree video capture solutions in the past, based on rigs using many of their existing Hero cameras, but Fusion is their first integrated 360-video solution.

While the Hero6 is available immediately for $499, the Fusion is expected to ship in November for $699. While we got to see the Fusion and its footage, most of the hands-on aspects of the launch event revolved around the Hero6. Each of the attendees was provided a Hero6 kit to record the rest of the days events. My group was provided a ride on the RocketBoat through the San Francisco Bay. This adventure took advantage of a number of features of the camera, including the waterproofing, the slow motion and the image stabilization.

The Hero6

The big change within the Hero6 is the inclusion of GoPro’s new custom-designed GP1 image processing chip. This allows them to process and encode higher frame rates, and allows for image stabilization at many frame-rate settings. The camera itself is physically similar to the previous generations, so all of your existing mounts and rigs will still work with it. It is an easy swap out to upgrade the Karma drone with the new camera, which also got a few software improvements. It can now automatically track the controller with the camera to keep the user in the frame while the drone is following or stationary. It can also fly a circuit of 10 waypoints for repeatable shots, and overcoming a limitation I didn’t know existed, it can now look “up.”

There were fewer precise details about the Fusion. It is stated to be able to record a 5.2K video sphere at 30fps and a 3K sphere at 60fps. This is presumably the circumference of the sphere in pixels, and therefore the width of an equi-rectangular output. That would lead us to conclude that the individual fish-eye recording is about 2,600 pixels wide, plus a little overlap for the stitch. (In this article, GoPro’s David Newman details how the company arrives at 5.2K.)

GoPro Fusion for 360

The sensors are slightly laterally offset from one another, allowing the camera to be thinner and decreasing the parallax shift at the side seams, but adding a slight offset at the top and bottom seams. If the camera is oriented upright, those seams are the least important areas in most shots. They also appear to have a good solution for hiding the camera support pole within the stitch, based on the demo footage they were showing. It will be interesting to see what effect the Fusion camera has on the “culture” of 360 video. It is not the first affordable 360-degree camera, but it will definitely bring 360 capture to new places.

A big part of the equation for 360 video is the supporting software and the need to get the footage from the camera to the viewer in a usable way. GoPro already acquired Kolor’s Autopano Video Pro a few years ago to support image stitching for their larger 360 video camera rigs, so certain pieces of the underlying software ecosystem to support 360-video workflow are already in place. The desktop solution for processing the 360 footage will be called Fusion Studio, and is listed as coming soon on their website.

They have a pretty slick demonstration of flat image extraction from the video sphere, which they are marketing as “OverCapture.” This allows a cellphone to pan around the 360 sphere, which is pretty standard these days, but by recording that viewing in realtime they can output standard flat videos from the 360 sphere. This is a much simpler and more intuitive approach to virtual cinematography that trying to control the view with angles and keyframes in a desktop app.

This workflow should result in a very fish-eye flat video, similar to the more traditional GoPro shots, due to the similar lens characteristics. There are a variety of possible approaches to handling the fish-eye look. GoPro’s David Newman was explaining to me some of the solutions he has been working on to re-project GoPro footage into a sphere, to reframe or alter the field of view in a virtual environment. Based on their demo reel, it looks like they also have some interesting tools coming for using the unique functionality that 360 makes available to content creators, using various 360 projections for creative purposes within a flat video.

GoPro Software
On the software front, GoPro has also been developing tools to help its camera users process and share their footage. One of the inherent issues of action-camera footage is that there is basically no trigger discipline. You hit record long before anything happens, and then get back to the camera after the event in question is over. I used to get one-hour roll-outs that had 10 seconds of usable footage within them. The same is true when recording many attempts to do something before one of them succeeds.

Remote control of the recording process has helped with this a bit, but regardless you end up with tons of extra footage that you don’t need. GoPro is working on software tools that use AI and machine learning to sort through your footage and find the best parts automatically. The next logical step is to start cutting together the best shots, which is what Quikstories in their mobile app is beginning to do. As someone who edits video for a living, and is fairly particular and precise, I have a bit of trouble with the idea of using something like that for my videos, but for someone to whom the idea of “video editing” is intimidating, this could be a good place to start. And once the tools get to a point where their output can be trusted, automatically sorting footage could make even very serious editing a bit easier when there is a lot of potential material to get through. In the meantime though, I find their desktop tool Quik to be too limiting for my needs and will continue to use Premiere to edit my GoPro footage, which is the response I believe they expect of any professional user.

There are also a variety of new camera mount options available, including small extendable tripod handles in two lengths, as well as a unique “Bite Mount” (pictured, left) for POV shots. It includes a colorful padded float in case it pops out of your mouth while shooting in the water. The tripods are extra important for the forthcoming Fusion, to support the camera with minimal obstruction of the shot. And I wouldn’t recommend the using Fusion on the Bite Mount, unless you want a lot of head in the shot.

Ease of Use
Ironically, as someone who has processed and edited hundreds of hours of GoPro footage, and even worked for GoPro for a week on paper (as an NAB demo artist for Cineform during their acquisition), I don’t think I had ever actually used a GoPro camera. The fact that at this event we were all handed new cameras with zero instructions and expected to go out and shoot is a testament to how confident GoPro is that their products are easy to use. I didn’t have any difficulty with it, but the engineer within me wanted to know the details of the settings I was adjusting. Bouncing around with water hitting you in the face is not the best environment for learning how to do new things, but I was able to use pretty much every feature the camera had to offer during that ride with no prior experience. (Obviously I have extensive experience with video, just not with GoPro usage.) And I was pretty happy with the results. Now I want to take it sailing, skiing and other such places, just like a “normal” GoPro user.

I have pieced together a quick highlight video of the various features of the Hero6:


Mike McCarthy is an online editor/workflow consultant with 10 years of experience on feature films and commercials. He has been involved in pioneering new solutions for tapeless workflows, DSLR filmmaking and multi-screen and surround video experiences. Check out his site.

Making the jump to 360 Video (Part 1)

By Mike McCarthy

VR headsets have been available for over a year now, and more content is constantly being developed for them. We should expect that rate to increase as new headset models are being released from established technology companies, prompted in part by the new VR features expected in Microsoft’s next update to Windows 10. As the potential customer base increases, the software continues to mature, and the content offerings broaden. And with the advances in graphics processing technology, we are finally getting to a point where it is feasible to edit videos in VR, on a laptop.

While a full VR experience requires true 3D content, in order to render a custom perspective based on the position of the viewer’s head, there is a “video” version of VR, which is called 360 Video. The difference between “Full VR” and “360 Video,” is that while both allow you to look around every direction, 360 Video is pre-recorded from a particular point, and you are limited to the view from that spot. You can’t move your head to see around behind something, like you can in true VR. But 360 video can still offer a very immersive experience and arguably better visuals, since they aren’t being rendered on the fly. 360 video can be recorded in stereoscopic or flat, depending on the capabilities of the cameras used.

Stereoscopic is obviously more immersive, less of a video dome and inherently supported by the nature of VR HMDs (Head Mounted Displays). I expect that stereoscopic content will be much more popular in 360 Video than it ever was for flat screen content. Basically the viewer is already wearing the 3D glasses, so there is no downside, besides needing twice as much source imagery to work with, similar to flat screen stereoscopic.

There are a variety of options for recording 360 video, from a single ultra-wide fisheye lens on the Fly360, to dual 180-degree lens options like the Gear 360, Nikon KeyMission, and Garmin Virb. GoPro is releasing the Fusion, which will fall into this category as well. The next step is more lens, with cameras like the Orah4i or the Insta360 Pro. Beyond that, you are stepping into the much more expensive rigs with lots of lenses and lots of stitching, but usually much higher final image quality, like the GoPro Omni or the Nokia Ozo. There are also countless rigs that use an array of standard cameras to capture 360 degrees, but these solutions are much less integrated than the all-in-one products that are now entering the market. Regardless of the camera you use, you are going to be recording one or more files in a pixel format fairly unique to that camera that will need to be processed before it can be used in the later stages of the post workflow.

Affordable cameras

The simplest and cheapest 360 camera option I have found is the Samsung Gear 360. There are two totally different models with the same name, usually differentiated by the year of their release. I am using the older 2016 model, which has a higher resolution sensor, but records UHD instead of the slightly larger full 4K video of the newer 2017 model.

The Gear 360 records two fisheye views that are just over 180 degrees, from cameras situated back to back in a 2.5-inch sphere. Both captured image circles are recorded onto a single frame, side by side, resulting in a 2:1 aspect ratio files. These are encoded into JPEG (7776×3888 stills) or HEVC (3840×1920 video) at 30Mb and saved onto a MicroSD card. The camera is remarkably simple to use, with only three buttons, and a tiny UI screen to select recording mode and resolution. If you have a Samsung Galaxy phone, there are a variety of other functions that allows, like remote control and streaming the output to the phone as a viewfinder and such. Even without a Galaxy phone, the camera did everything I needed to generate 360 footage to stitch and edit with but it was cool to have a remote viewfinder for the driving shots.

Pricier cameras

One of the big challenges of shooting with any 360 camera is how to avoid getting gear and rigging in the shot since the camera records everything around it. Even the tiny integrated tripod on the Gear 360 is visible in the shots, and putting it on the plate of my regular DSLR tripod fills the bottom of the footage. My solution was to use the thinnest support I could to keep the rest of the rigging as far from the camera as possible, and therefore smaller from its perspective. I created a couple options to shoot with that are pictured below. The results are much less intrusive in the resulting images that are recorded. Obviously besides the camera support, there is the issue of everything else in the shot including the operator. Since most 360 videos are locked off, an operator may not be needed, but there is no “behind the camera” for hiding gear or anything else. Your set needs to be considered in every direction, since it will all be visible to your viewer. If you can see the camera, it can see you.

There are many different approaches to storing 360 images, which are inherently spherical, as a video file, which is inherently flat. This is the same issue that cartographers have faced for hundreds of years — creating flat paper maps of a planet that is inherently curved. While there are sphere map, cube map and pyramid projection options (among others) based on the way VR headsets work, the equirectangular format has emerged as the standard for editing and distribution encoding, while other projections are occasionally used for certain effects processing or other playback options.

Usually the objective of the stitching process is to get the images from all of your lenses combined into a single frame with the least amount of distortion and the fewest visible seams. There are a number of software solutions that do this, from After Effects plugins, to dedicated stitching applications like Kolor AVP and Orah VideoStitch-Studio to unique utilities for certain cameras. Once you have your 360 video footage in the equirectangular format, most of the other steps of the workflow are similar to their flat counterparts, besides VFX. You can cut, fade, title and mix your footage in an NLE and then encode it in the standard H.264 or H.265 formats with a few changes to the metadata.

Technically, the only thing you need to add to an existing 4K editing workflow in order to make the jump to 360 video is a 360 camera. Everything else could be done in software, but the other thing you will want is a VR headset or HMD. It is possible to edit 360 video without an HMD, but it is a lot like grading a film using scopes but no monitor. The data and tools you need are all right there, but without being able to see the results, you can’t be confident of what the final product will be like. You can scroll around the 360 video in the view window, or see the whole projected image all distorted, but it won’t have the same feel as experiencing it in a VR headset.

360 Video is not as processing intensive as true 3D VR, but it still requires a substantial amount of power to provide a good editing experience. I am using a Thinkpad P71 with an Nvidia Quadro P5000 GPU to get smooth performance during all these tests.

Stay tuned for Part 2 where we focus on editing 360 Video.


Mike McCarthy is an online editor/workflow consultant with 10 years of experience on feature films and commercials. He has been working on new solutions for tapeless workflows, DSLR filmmaking and multi-screen and surround video experiences. Check out his site.

Review: Lenovo’s ThinkPad P71 mobile workstation

By Mike McCarthy

Lenovo was nice enough to send me their newest VR-enabled mobile workstation to test out on a VR workflow project I am doing. The new ThinkPad P71 is a beast with a 17-inch UHD IPS screen. The model they sent to me was equipped with the fastest available processor, a Xeon E5-1535M v6 with four cores processing eight threads at an official speed of 3.1GHz. It has 32GB of DDR4-2400 ECC RAM, with two more slots allowing that to be doubled to 64GB if desired.

The system’s headline feature is the Nvidia Quadro P5000 mobile GPU, with 2,048 CUDA cores, fed by another 16GB of dedicated DDR5 memory. The storage configuration is a single NVMe 1TB SSD populating one of two available M.2 slots. This configuration is currently available for $5,279, discounted to $4,223.20 on Lenovo.com right now. So while it is not cheap, it is one of the most powerful mobile workstations you can buy right now.

Connectivity wise, it has dual Thunderbolt 3 ports, which can also be used for USB 3.1 Type C devices. It has four more USB 3.1 Type A ports and a Gigabit Ethernet port. You have a number of options for display connectivity. Besides the Thunderbolt ports, there is a MiniDP 1.2 port and an HDMI 1.4 port (1.4 based on Intel graphics limitations). It has an SDXC slot, an ExpressCard34 slot, and a single 1/8-inch headphone mic combo jack. The system also has a docking connector and a rectangular port for the included 230W power adaptor.

It has the look and feel of a traditional ThinkPad, which goes back to the days when they were made by IBM. It has the customary TrackPoint as well as a touchpad. Both have three mouse buttons, which I like in theory, but I constantly find myself trying to click with the center button to no avail. I would either have to get used to it, or set the center action to click as well, defeating the purpose of the third button. The Fn key in the bottom corner will take some getting used to as well, as I keep hitting that instead of CTRL, but I adapted to a similar configuration on my current laptop.

I didn’t like the combo jack at first, because it required a cheap adapter, but now that I have gotten one, I see why that is the future, once all the peripherals support it. I had plugged my mic and headphones in backwards as recently as last week, so it is an issue when the ports aren’t clearly labeled and the combo jack solves that once and for all. It is a similar jack to most cell phones, and you only need an adapter for the mic functionality, regular headphones work by default.

The system doesn’t weigh as much as I expected, probably due to the lack of spinning disks or optical drive, which can be added if desired. It came relatively clean, with Windows 10 Pro installed, without too many other applications or utilities pre-installed. It had all of the needed drivers and a simple utility for operating the integrated X-Rite Pantone color calibrator for the screen. There was a utility for adding any other applications that would normally be included, which I used to download the Lenovo Performance Tuner. I use the Performance Tuner more for monitoring usage than adjusting settings, but can be nice to have everything in one place, especially in Windows 10.

The system boots up in about 10 seconds, and shuts down even faster. Hibernating takes twice as long, which is to be expected with that much RAM to be cached to disk, even with an NVMe SSD. But that may be worth the extra time to keep your applications open. My initial tests of the SSD showed a 1700MB/s write speed with 2500MB/s reads. Longer endurance tests resulted in write speeds decreasing to 1200MB/s, but the read speeds remained consistently above 2500MB/s. That should be more than enough throughput for most media work, even allowing me to playback uncompressed 6K content, and should allow 4K uncompressed media capture if you connect an I/O device to the Thunderbolt bus.

The main application I use on a daily basis is Adobe Premiere Pro, so most of my performance evaluation revolves around that program, although I used a few others as well. I was able to load a full feature film off of a USB3 drive with no issues. The 6K Cineform and DNxHR media played back at ½ res without issue. The 6K R3D files played at ¼ res without dropping frames, which is comparable to my big tower.

My next playback test was fairly unique to my workflow, but a good benchmark of what is possible. I was able to connect three 1080p televisions to the MiniDP port, using an MST (Multi-Stream Transport) hub, with three HDMI ports. Using the Nvidia Mosaic functionality offered by the Quadro P5000 card, I can span them into a single display, which Premiere can send output to, via the Adobe’s Mercury Playback engine. This configuration allows me to playback 6K DNxHR 444 files to all three screens, directly off the timeline, at half res, without dropping frames. My 6K H.265 files playback at full res outside Premiere. That is a pretty impressive display for a laptop. Once I had maxed out the possibilities for playback, I measured a few encodes. In general, the P71 takes about twice as long to encode things in Adobe Media Encoder as my 20-core desktop workstation, but is twice as fast as my existing quad Core i7 4860 laptop.

The other application I have been taxing my system with recently is DCP-O-Matic. It takes 30 hours to render my current movie to a 4K DCP on my desktop, which is 18x the runtime, but I know most of my system’s 20 cores are sitting idle based on the software threading. Doing a similar encode on the Lenovo system took 12.5x the run time, so that means my 100-minute film should take 21 hours. The higher base frequency of the quad core CPU definitely makes a difference in this instance.

The next step was to try my HMD headset with it to test out the VR capability. My Oculus Rift installed without issues, which is saying something, based on the peculiarities of Oculus’ software. Maybe there is something to that “VR-ready” program, but I did frequently have issues booting up the system with the Rift connected, so I recommend plugging it in after you have your system up and running. Everything VR-related ran great, except for the one thing I actually wanted to do, which was edit 360 video in Premiere, with the HMD. There was some incompatibility between the drivers for the laptop and the software. (Update: Setting the graphics system to Discrete instead of Hybrid in the BIOS solves this problem. This solution works with both PPro11’s Skybox Player, and PPro12’s new SteamVR based approach.)

There are a variety of ways to test battery life, but since this is a VR-ready system that seemed to be the best approach. How long would it support using a VR headset before needing to plug in? I got just short of an hour of heavy 3D VR usage before I started getting low battery warnings. I was hoping to be able to close the display to save power, since I am not looking at it while using the headset. (I usually set the Close Lid action to Do Nothing on all my systems because I want to be able to walk into the other room to show someone something on my timeline without effecting the application. If I want to sleep the system, I can press the button.) But whenever the Rift is active, closing the lid puts the machine to sleep immediately, regardless of the settings. So you have to run the display and the HMD anytime you are working in VR. And don’t plan on doing extensive work without plugging in.

Now to be fair, setting up to use VR involves preparing the environment and configuring sensors, so adding power to that mix is a reasonable requirement and very similar to 3D gaming. Portable doesn’t always mean untethered. But for browsing the Internet, downloading project files and editing articles, I would expect about four hours of battery life from the system before needing to recharge. It is really hard to accurately estimate run time when the system’s performance and power needs scale so much depending on the user’s activities. The GPU alone scales from 5 watts to 100 watts depending on what is being processed, but the run time is not out of line with what is to be expected from products in this class of performance.

Summing Up
All in all, the P71 is an impressive piece of equipment, and one of only a few ways you can currently get a portable professional VR solution. I recognize that most of my applications aren’t using all of the power I would be carrying around in a P71, so for my own work, I would probably hope to find a smaller and lighter-weight system at the expense of some of that processing power. But for people who have uncompromising needs for the fastest system they can possibly get, the Lenovo P71 fits the bill. It is a solid performer that can do an impressive amount of processing, while still being able to come with you wherever you need to go.


Mike McCarthy is an online editor/workflow consultant with 10 years of experience on feature films and commercials. He has been working on new solutions for tapeless workflows, DSLR filmmaking and multi-screen and surround video experiences. Check out his site.