Tag Archives: Mike McCarthy

Nvidia, AMD and Intel news from Computex

By Mike McCarthy

A number of new technologies and products were just announced at this year’s Computex event in Taipei, Taiwan. Let’s take a look at ones that seem relevant to media creation pros.

Nvidia released a line of mobile workstation GPUs based on its newest Turing architecture. Like the GeForce lineup, the Turing line has versions without the RTX designation. The Quadro RTX 5000, 4000 and 3000 have raytracing and Tensor cores, while the Quadro T2000 and T1000 do not, similar to the GeForce 16 products. The RTX 5000 matches the desktop version, with slightly more CUDA cores than the GeForce RTX 2080, although at lower clock speeds for reduced power consumption.

Nvidia’s new RTX 5000

The new Quadro RTX 3000 has similar core configuration to the desktop Quadro RTX 4000 and GeForce RTX 2070. This leaves the new RTX 4000 somewhere in between, with more cores than the desktop variant, aiming to provide similar overall performance at lower clock speeds and power consumption. While I can respect the attempt to offer similar performance at given tiers, doing so makes it more complicated than just leaving consistent naming for particular core configurations.

Nvidia also announced a new “RTX Studio” certification program for laptops targeted at content creators. These laptops are designed to support content creation applications with “desktop-like” performance. RTX Studio laptops will include an RTX GPU (either GeForce or Quadro), an H-Series or better Intel CPU, at least 16GB RAM and 512GB SSD, and at least a 1080p screen. Nvidia also announced a new line of studio drivers that are supposed to work with both Quadro and GeForce hardware. They are optimized for content creators and tested for stability with applications from Adobe, Autodesk, Avid, and others. Hopefully these drivers will simplify certain external GPU configurations that mix Quadro and GeForce hardware. It is unclear whether or not these new “Studio” drivers will replace the previously announced “Creator Ready” series of drivers.

Intel announced a new variant of its top end 9900K CPU. The i9-9900KS has a similar configuration, but runs at higher clock speeds on more cores, with a 4GHz base frequency and allowing 5GHz boost speeds on all eight cores. Intel also offered more details on its upcoming 10nm Ice Lake products with Gen 11 integrated graphics, which offers numerous performance improvements and VNNI support to accelerate AI processing. Intel is also integrating support for Thunderbolt 3 and Wi-Fi 6 into the new chipsets, which should lead to wider support for those interfaces. The first 10nm products to be released will be the lower-power chip for tablets and ultra portable laptops with higher power variants coming further in the future.

AMD took the opportunity to release new generations of both CPUs and GPUs. On the CPU front, AMD has a number of new third-generation 7nm Ryzen processors, with six to 12 cores in the 4GHz range and supporting 20 lanes of fourth-gen PCIe. Priced between $200 and $500, they are targeted at consumers and gamers and are slated to be available July 7th. These CPUs compete with Intel’s 9900K and similar CPUs, which have been offering top performance for Premiere and After Effects users due to their high clock speed. It will be interesting to see if AMD’s new products offer competitive performance at that price point.

AMD also finally publicly released its Navi generation GPU architecture, in the form of the new Radeon 5700. The 5000 series has an entirely new core design, which they call Radeon DNA (RDNA) to replace the GCN architecture first released seven years ago. RDNA is supposed to offer 25% more performance per clock cycle and 50% more performance per watt. This is important, because power consumption was AMD’s weak point compared to competing products from Nvidia.

AMD president and CEO Dr. Lisa Su giving her keynote.

While GPU power consumption isn’t as big of a deal for gamers using it a couple hours a day, commercial compute tasks that run 24/7 see significant increases in operating costs for electricity and cooling when power consumption is higher. AMD’s newest Radeon 5700 is advertised to compete performance-wise with the GeForce RTX 2070, meaning that Nvidia still holds the overall performance crown for the foreseeable future. But the new competition should drive down prices in the mid-range performance segment, which are the cards most video editors need.


Mike McCarthy is an online editor/workflow consultant with over 10 years of experience on feature films and commercials. He has been involved in pioneering new solutions for tapeless workflows, DSLR filmmaking and multi-screen and surround video experiences. Check out his site.

Review: Razer Blade 15-inch mobile workstation

By Mike McCarthy

I am always looking for the most powerful tools in the smallest packages, so I decided to check out the Razer Blade 15-inch laptop with an Nvidia GeForce RTX 2080 Max-Q graphics card. The Max-Q variants are optimized for better thermals and power usage — at the potential expense of performance — in order to allow more powerful GPUs to be used in smaller laptops. The RTX 2080 is Nvidia’s top-end mobile GPU, with 2,944 CUDA cores and 8GB of DDR6 memory, running at 384GB/s with 13.6 billion transistors on the chip.

The new Razer Blade has a six-core Intel i7-8750H processor with 16GB RAM and a 512GB SSD. It has mDP 1.4, HDMI 2.0b, Thunderbolt 3 and three USB 3.1 ports. Its 15.6-inch screen can run at 144Hz refresh rate but only supports full HD 1920×1080, which is optimized for gaming, not content creation. The past four laptops I have used have all been UHD resolution at various sizes, which gives far more screen real estate for creative applications and better resolution to review your imagery.

I also prefer to have an Ethernet port, but I am beginning to accept that a dongle might be acceptable for that, especially since it opens up the possibility of using 10 Gigabit Ethernet. We aren’t going to see 10GigE on laptops anytime soon due to the excessive power consumption, but you only need 10GigE when at certain locations that support it, so a dongle or docking station is reasonable for those use cases.

Certain functionality on the system required a free account to be registered with Razer, which is annoying, but I’ve found this requirement is becoming the norm these days. That gives access to the Razer Synapse utility for customizing the system settings, setting fan speed and even remapping keyboard functionality. Any other Razer peripherals would be controlled here as well. As part of a top-end modern gaming system, the keyboard has fully controllable color back lighting. While I find most of the default “effects” to be distracting, the option to color code your shortcut keys is interesting. And if you really want to go to the next level, you can customize it further.

For example, when you press the FN key, by default the keys that have function behaviors connected with them light up white, which impressed me. The colors and dimming are generated by blinking the LEDs, but I was able to perceive the flicker when moving my eyes, so I stuck with colors that didn’t involve dimming channels. But that still gave me six options (RGB, CYM) plus white.

This is the color config I was running in the photos, but the camera does not reflect how it actually looks. In pictures, the keys look washed out, but in person they are almost too bright and vibrant. But we are here for more than looks, so it was time to put it through its paces and see what can happen under the hood.

Testing
I ran a number of benchmarks, starting with Adobe Premiere Pro. I now have a consistent set of tests to run on workstations in order to compare each system. The tests involve Red, Sony Venice and ARRI Alexa source files, with various GPU effects applied and exported to compressed formats. It handled the 4K and 8K renders quite well — pretty comparable to full desktop systems — showcasing the power of the RTX GPU. Under the sustained load of rendering for 30 minutes, it did get quite warm, so you will want adequate ventilation … and you won’t want it sitting on your lap.

My next test was RedCine-X Pro, with its new CUDA playback acceleration of files up to 8K. But what is the point of decoding 8K if you can’t see all the pixels you are processing? So for this test, I also connected my Dell UP3218K screen to the Razer Blade’s Mini DisplayPort 1.4 output. Outputting to the monitor does affect performance a bit, but that is a reasonable expectation. It doesn’t matter if you can decode 8K in real time if you can’t display it. Nvidia provides reviewers with links to some test footage, but I have 40TB to choose from, in addition to test clips from all different settings on the various cameras from my Large Format Camera test last year.

The 4K Red files worked great at full res to the external monitor — full screen or pixel for pixel — while the system barely kept up with the 6K and 8K anamorphic files. 8K full frame required half-res playback to view smoothly on the 8K display. Full-frame 8K was barely realtime with the external monitor disabled, but that is still very impressive for a laptop (I have yet to accomplish that on my desktop). The rest of the files played back solidly to the local display. Disabling the CUDA GPU acceleration requires playing back below 1/8th res to do anything on a laptop, so this is where having a powerful GPU makes a big difference.

Blackmagic Resolve is the other major video editing program to consider, and while I do not find it intuitive to use myself, I usually recommend it to others who are looking for a high level of functionality but aren’t ready to pay for Premiere. I downloaded and rendered a test project from Nvidia, which plays Blackmagic Raw files in real time with a variety of effects and renders to H.264 in 40 seconds, but it takes 10 times longer with CUDA disabled in Resolve.

Here, as with the other tests, the real-world significance isn’t how much faster it is with a GPU than without, but how much faster is it with this RTX GPU compared to with other options. Nvidia clams this render takes 2.5 times as long on a Radeon-based MacBook Pro, and 10% longer on a previous-generation GTX 1080 laptop, which seems consistent with my previous experience and tests.

The primary differentiation of Nvidia’s RTX line of GPUs is the inclusion of RT cores to accelerate raytracing and Tensor cores to accelerate AI inferencing, so I wanted to try tasks that used those accelerations. I started by testing Adobe’s AI-based image enhancement in Lightroom Classic CC. Nvidia claims that the AI image enhancement uses the RTX’s Tensor cores, and it is four times faster with the RTX card. The visual results of the process didn’t appear to be much better than I could have achieved with manual development in Photoshop, but it was a lot faster to let the computer figure out what to do to improve the images. I also ran into an issue where certain blocks of the image got corrupted in the process, but I am not sure if Adobe or Nvidia is at fault here.

Raytracing
While I could have used this review as an excuse to go play Battlefield V to experience raytracing in video games, I stuck with the content-creation focus. In looking for a way to test raytracing, Nvidia pointed me to OctaneRender. Otoy has created a utility called OctaneBench for measuring the performance of various hardware configurations with its render engine. It reported that the RTX’s raytracing acceleration was giving me a 3x increase in render performance.

I also tested ProRender in Maxon Cinema 4D, which is not a raytracing renderer but does use GPU acceleration through OpenCL. Apparently, there is a way to use the Arnold ray-tracing engine in Cinema 4D, but I was reaching the limits of my 3D animation expertise and resources, so I didn’t pursue that path, and I didn’t test Maya for the same reason.

With ProRender, I was able to render views of various demo scenes 10 to 20 times faster than I could with a CPU only. I will probably include this as a regular test in future reviews, allowing me to gauge render performance far better than I can with Cinebench (which returned a CPU score of 836). And compiling a list of comparison render times will add more context to raw data. But, for now, I was able to render the demo “Bamboo” scene in 39 seconds and the more complex “Coffee Bean” scene in 188 seconds, beating even the Nvidia marketing team’s expected results.

VR
No test of a top-end GPU would be complete without trying out its VR performance. I connected my Windows-based Lenovo Explorer Mixed Reality headset, installed SteamVR and tested both 360 video editing in Premiere Pro and the true 3D experiences available in Steam. As would be expected, the experience was smooth, making this one of the most portable solutions for full-performance VR.

The RTX 2080 is a great GPU, and I had no issues with it. Outside of true 3D work, the upgrade from the Pascal-based GTX 1080 is minor, but for anyone upgrading from systems older than that, or doing true raytracing or AI processing, you will see a noticeable improvement in performance.

The new Razer Blade is a powerful laptop for its size, and while I did like it, that doesn’t mean I didn’t run into a few issues along the way. Some of those, like the screen resolution, are due to its focus on gaming instead of content creation, but I also had an issue with the touch pad. Touch pad issues are common when switching between devices constantly, but in this case, right-clicking instead of left-clicking and not registering movement when the mouse button was pressed were major headaches. The problems were only alleviated by connecting a mouse and sticking with that, which I frequently do anyway. The power supply has a rather large connector on a cumbersome thick and stiff cord, but it isn’t going to be falling out once you get it inserted. Battery life will vary greatly depending on how much processing power you are using.

These RTX chips are the first mobile GPUs with dedicated RT cores and with Tensor cores, since Volta-based chips never came to laptops. So for anyone with processing needs that are accelerated by those developments, the new RTX chip is obviously worth the upgrade. If you want the fastest thing out there, this is it. (Or at least it was, until Razer added options for 9th Generation Intel processors this week and a 4K OLED screen (an upgrade I would highly recommend for content creators). The model I reviewed goes for $3,000. The new 9th Gen version with a 240Hz screen is the same price, while the 4K OLED Touch version costs an extra $300.

Summing Up
If you are looking for a more balanced solution or are on a more limited budget, you should definitely compare the new Razer Blade to the new Nvidia GTX 16 line of mobile products that was just announced. Then decide which option is a better fit for your particular needs and budget.

The development of eGPUs has definitely shifted this ideal target for my usage. While this system has a Thunderbolt 3 port, it is fast enough that you won’t see significant gains from an eGPU, but that advantage comes at the expense of battery life and price. I am drawn to eGPUs because I only need maximum performance at my desk, but if you need top-end graphics performance totally untethered, RTX Max-Q chips are the solution for you.


Mike McCarthy is an online editor/workflow consultant with over 10 years of experience on feature films and commercials. He has been involved in pioneering new solutions for tapeless workflows, DSLR filmmaking and multi-screen and surround video experiences. Check out his site.

NAB 2019: First impressions

By Mike McCarthy

There are always a slew of new product announcements during the week of NAB, and this year was no different. As a Premiere editor, the developments from Adobe are usually the ones most relevant to my work and life. Similar to last year, Adobe was able to get their software updates released a week before NAB, instead of for eventual release months later.

The biggest new feature in the Adobe Creative Cloud apps is After Effects’ new “Content Aware Fill” for video. This will use AI to generate image data to automatically replace a masked area of video, based on surrounding pixels and surrounding frames. This functionality has been available in Photoshop for a while, but the challenge of bringing that to video is not just processing lots of frames but keeping the replaced area looking consistent across the changing frames so it doesn’t stand out over time.

The other key part to this process is mask tracking, since masking the desired area is the first step in that process. Certain advances have been made here, but based on tech demos I saw at Adobe Max, more is still to come, and that is what will truly unlock the power of AI that they are trying to tap here. To be honest, I have been a bit skeptical of how much AI will impact film production workflows, since AI-powered editing has been terrible, but AI-powered VFX work seems much more promising.

Adobe’s other apps got new features as well, with Premiere Pro adding Free-Form bins for visually sorting through assets in the project panel. This affects me less, as I do more polishing than initial assembly when I’m using Premiere. They also improved playback performance for Red files, acceleration with multiple GPUs and certain 10-bit codecs. Character Animator got a better puppet rigging system, and Audition got AI-powered auto-ducking tools for automated track mixing.

Blackmagic
Elsewhere, Blackmagic announced a new version of Resolve, as expected. Blackmagic RAW is supported on a number of new products, but I am not holding my breath to use it in Adobe apps anytime soon, similar to ProRes RAW. (I am just happy to have regular ProRes output available on my PC now.) They also announced a new 8K Hyperdeck product that records quad 12G SDI to HEVC files. While I don’t think that 8K will replace 4K television or cinema delivery anytime soon, there are legitimate markets that need 8K resolution assets. Surround video and VR would be one, as would live background screening instead of greenscreening for composite shots. No image replacement in post, as it is capturing in-camera, and your foreground objects are accurately “lit” by the screens. I expect my next major feature will be produced with that method, but the resolution wasn’t there for the director to use that technology for the one I am working on now (enter 8K…).

AJA
AJA was showing off the new Ki Pro Go, which records up to four separate HD inputs to H.264 on USB drives. I assume this is intended for dedicated ISO recording of every channel of a live-switched event or any other multicam shoot. Each channel can record up to 1080p60 at 10-bit color to H264 files in MP4 or MOV and up to 25Mb.

HP
HP had one of their existing Z8 workstations on display, demonstrating the possibilities that will be available once Intel releases their upcoming DIMM-based Optane persistent memory technology to the market. I have loosely followed the Optane story for quite a while, but had not envisioned this impacting my workflow at all in the near future due to software limitations. But HP claims that there will be options to treat Optane just like system memory (increasing capacity at the expense of speed) or as SSD drive space (with DIMM slots having much lower latency to the CPU than any other option). So I will be looking forward to testing it out once it becomes available.

Dell
Dell was showing off their relatively new 49-inch double-wide curved display. The 4919DW has a resolution of 5120×1440, making it equivalent to two 27-inch QHD displays side by side. I find that 32:9 aspect ratio to be a bit much for my tastes, with 21:9 being my preference, but I am sure there are many users who will want the extra width.

Digital Anarchy
I also had a chat with the people at Digital Anarchy about their Premiere Pro-integrated Transcriptive audio transcription engine. Having spent the last three months editing a movie that is split between English and Mandarin dialogue, needing to be fully subtitled in both directions, I can see the value in their tool-set. It harnesses the power of AI-powered transcription engines online and integrates the results back into your Premiere sequence, creating an accurate script as you edit the processed clips. In my case, I would still have to handle the translations separately once I had the Mandarin text, but this would allow our non-Mandarin speaking team members to edit the Mandarin assets in the movie. And it will be even more useful when it comes to creating explicit closed captioning and subtitles, which we have been doing manually on our current project. I may post further info on that product once I have had a chance to test it out myself.

Summing Up
There were three halls of other products to look through and check out, but overall, I was a bit underwhelmed at the lack of true innovation I found at the show this year.

Full disclosure, I was only able to attend for the first two days of the exhibition, so I may have overlooked something significant. But based on what I did see, there isn’t much else that I am excited to try out or that I expect to have much of a serious impact on how I do my various jobs.

It feels like most of the new things we are seeing are merely commoditized versions of products that may originally have been truly innovative when they were initially released, but now are just slightly more fleshed out versions over time.

There seems to be much less pioneering of truly new technology and more repackaging of existing technologies into other products. I used to come to NAB to see all the flashy new technologies and products, but now it feels like the main thing I am doing there is a series of annual face-to-face meetings, and that’s not necessarily a bad thing.

Until next year…


Mike McCarthy is an online editor/workflow consultant with over 10 years of experience on feature films and commercials. He has been involved in pioneering new solutions for tapeless workflows, DSLR filmmaking and multi-screen and surround video experiences. Check out his site.

Review: HP’s double-hinged ZBook Studio x360 mobile workstation

By Mike McCarthy

I recently had the opportunity to test HP’s ZBook Studio x360 mobile workstation over the course of a few weeks. HP’s ZBook mobile workstation division has really been thinking outside the box lately, with the release of the ZBook X2 tablet, the HP Z-VR backpack-mounted system and now the ZBook Studio x360.

The ZBook Studio x360 is similar in design functionality to HP’s other x360 models — the Pavilion, Spectre, Envy, ProBook and Elitebook x360 — in that the display is double-hinged. The keyboard can be folded all the way behind the screen, allowing it to be used similarly to a tablet or placed in “tent” or “presentation” mode with the keyboard partially folded behind it. But the ZBook is clearly the top-end option of the systems available in that form factor. And it inherits all of the engineering from the rest of HP’s extensive product portfolio, in regards to security, serviceability, and interface.

Performance-wise, this Studio x360 model sits somewhere in the middle of HP’s extensive ZBook mobile workstation lineup. It is above the lightweight ZBook 14U and 15U and X2 tablet with their low-voltage U-Series CPUs and the value-oriented 15v. It is similar to the more traditional clamshell ultrabook ZBook Studio, and has less graphics power and RAM than the top-end ZBook 15 and 17.

It is distinguished from the ZBook Studio by its double-hinged 360 folding chassis, and its touch and pen inking capability. It is larger than the ZBook X2 with more powerful internal hardware. This model is packed with processing power in the form of a 6-core 8th generation Xeon processor, 32GB RAM and an Nvidia Quadro P1000 GPU. The 15-inch UHD screen boosts up to 400 nits at full brightness and, of course, supports touch and pen input.

Configuration Options
The unit has a number of interesting configuration options with two M.2 slots and a 2.5-inch bay allowing up to 6TB of internal storage, but most users will forgo the 2.5-inch SATA bay for an extended 96whr battery. There is the option of choosing between a 4G WWAN card or DreamColor display, giving users a wide selection of possible capabilities.

Because of the work I do, I am mostly interested in answering the question: “How small and light can I go, and still get my work done effectively?” In order to answer that question, I am reviewing a system with most of the top-end options. I started at a 17-inch Lenovo P71 last year, then tried a large 15-inch PNY PrevailPro and now am trying out this much lighter 15-inch book. There is no compromise with the 6-core CPU, as that is the same as in a 17-inch beast. So the biggest difference is in the GPU, with the mobile Quadro P1000 only having the 512 CUDA core, one third the power of the Quadro P4000 I last tested. So VR is not going to work, but besides heavy color grading, most video editing tasks should be supported. And 32GB of RAM should be enough for most users, but I installed a second NVMe drive, giving me a total of 2TB of storage.

Display
The 15.6-inch display is available in a number of different options, all supporting touch and digital pen input. The base-level full-HD screen can be upgraded to a Sure View screen, allowing the user to selectively narrow the viewing angle at the press of a key in order to increase their privacy. Next up is the beautiful 400-nit UHD screen that my unit came with. And the top option is a 600-nit DreamColor calibrated UHD panel. All of the options fully support touch and pen input.

Connectivity
The unit has dual-Thunderbolt 3 ports, supporting DisplayPort 1.3, as well as HDMI, dual-USB3.1 Type-A ports, an SDXC card slot and an audio jack. The main feature I am missing is an RJ-45 jack for Gigabit Ethernet. I get that there are trade-offs to be made in any configuration, but that is the item I am missing from this unit. On the flip side, with the release of affordable Thunderbolt-based 10GbE adapters, that is probably what I would pair with this unit if I was going to be using it to edit assets I have stored on my network. So that is a solvable problem.

Serviceability
Unlike the heavier ZBook 15 and 17 models, it does not have a tool-less chassis, but that is an understandable a compromise to reduce size and weight, and totally reasonable. I was able to remove the bottom cover with a single torx screwdriver, giving me access to the RAM, wireless cards, and M.2 slots I was populating with a second NVMe drive to test. The battery can also be replaced that way should the need arise, but the 96whr long-life battery is fully covered by the system warranty, be that three or five years depending on your service level.

Security
There are a number of unique features that this model shares with many others in HP’s lineup. The UEFI-based HP Sure Start BIOS and pre-boot environment provide a host of options for enterprise-level IT management, and make it less likely that the boot process will get corrupted. HP Sure Click is a security mechanism that isolates each Chromium browser tab in its own virtual machine, protecting the rest of your system from any malware that it might otherwise be exposed to. Sure Run and Sure Recover are designed to prevent and recover from security failures that render the system unusable.

The HP Client Security Manager brings the controls for all of this functionality into one place and uses the system’s integrated fingerprint reader. HP Workwise is a utility for integrating the laptop with one’s cell phone, allowing automatic system lock and unlock when the cell phone leaves or enters Bluetooth range and phone notifications from the other “Sure” security applications.

Thunderbolt Dock
HP also supplied me with their new Thunderbolt dock. The single most important feature on that unit from my perspective is the Gigabit Ethernet port, since there isn’t one built into the laptop. It also adds two DisplayPorts and one VGA output and includes five more USB ports. I was able to connect my 8K display to the DisplayPort output and it ran fine at 30Hz, as is to be expected from a single Thunderbolt connection. The dock should run anything smaller than that at 60Hz, including two 4K displays.

The dock also supports an optional audio module to facilitate better conference calls, with a built-in speaker, microphone and call buttons. It is a nice idea but a bit redundant since the laptop has a “world-facing” microphone for noise cancellation or group calling and even has “Collaboration Keys” for controlling calls built into the top of the keyboard. Apparently, HP sees this functionality totally replacing office phones.

I initially struggled to get the dock to work — besides the DisplayPorts — but this was because I connected it before boot-up. Unlike docking stations from back in the day, Thunderbolt is fully hot-swappable and actually needs to be powered on the first time it is connected in order to trigger the dialog box, which gives it low-level access to your computer for security reasons. Once I did that, it has worked seamlessly.

The two-part cable integrates a dedicated power port and Thunderbolt 3 connection, magnetically connected for simple usage while maintaining flexibility for future system compatibility. The system can receive power from the Thunderbolt port, but for maximum power and performance uses a 130W dedicated power plug as well, which appears to be standardized across much of HP’s line of business products.

Touchscreens and Pens
I had never seriously considered tablets or touchscreen solutions for my own work until one of HP’s reps showed me an early prototype of the ZBook X2 a few years ago. I initially dismissed it until he explained how much processing power they had packed into it. Only then did I recognize that HP had finally fulfilled two of my very different and long-standing requests in a way that I hadn’t envisioned. I had been asking the display team for a lightweight battery-powered DreamColor display, and I had been asking the mobile workstation team for a 12- or 14-inch Nvidia-powered model — this new device was both.

I didn’t end up reviewing the X2 during its initial release last year, although I plan to soon. But once the X2 shifted my thinking about tablet and touch-based tools, I saw this ZBook Studio x360 as an even more powerful implementation of that idea, in a slightly larger form factor. While I have used pens on other people’s systems in the past, usually when doing tech support for other editors, this is my first attempt to do real work with a pen instead of a mouse and keyboard.

One of the first obstacles I encountered was getting the pen to work at all. Unlike the EMR-based pens from Wacom tablets and the ZBook X2, the x360 uses an AES-based pen, which requires power and a Bluetooth connection to communicate with the system. I am not the only user to be confused by this solution, but I have been assured by HP that the lack of documentation and USB-C charging cable have been remedied in currently shipping systems.

It took me a while (and some online research) to figure out that there was a USB-C port hidden in the pen and that it needed to be charged and paired with the system. Once I did that, it has functioned fine for me. The pen itself works great, with high precision and 4K levels of pressure sensitivity and tilt support. I am not much of a sketcher or painter, but I do a lot of work in Photoshop, either cleaning images up or creating facial expressions for my Character Animator puppets. The pen is a huge step up from the mouse for creating smooth curves and natural lines. And the various buttons worked well for me once I got used to them. But I don’t do a lot of work that benefits from having the pen support, and trying to adapt other tasks to the pen-based input was more challenging than I anticipated.

The other challenge I encountered was with the pen holder, which fits into the SD card slot. The design is good and works better than I would have expected, but removing the original SD plug that protects the slot was far more difficult than it should be. I assume the plug is necessary for the system to pass the 13 MilSpec type tests that HP runs all of its ZBooks through, but I probably won’t be wedging it back in that slot as long as I have the system.

Inking
I am not much of a tablet user as of yet since this was my first foray into that form factor, but the system is a bit large and bulky when folded back into tablet mode. I have hit the power button by accident on multiple occasions, hibernating the system while I was trying to use it. This has primarily been an issue when I am using it in tablet mode and holding it with my left hand in that area by default. But the biggest limitation I encountered in tablet mode was recognizing just how frequently I use the keyboard during the course of my work. While Windows Inking does allow for an onscreen keyboard to be brought up for text entry, functions like holding Alt for anchor-based resizing are especially challenging. I am curious to see if some of these issues are alleviated on the X2 by the buttons they built into the edge of the display. As long as I have easy access to Shift, Ctrl, Alt, C, V and a couple others, I think I would be good to go, but it is one of those things that you can’t know for sure until you try it yourself. And different people with varying habits and preferences might prefer different solutions to the same tasks. In my case, I have not found the optimal touch and inking experience yet.

Performance
I was curious to see what level of performance I would get from the Quadro P1000, as I usually use systems with far more GPU power. But I was impressed with how well it was able to handle the animating and editing of the 5K assets for my Grounds of Freedom animated series. I was even able to dynamically link between the various Adobe apps with a reasonable degree of interactive feedback. That is where you start to see a difference between this mobile system and a massive desktop workstation.

eGPU
Always looking for more power, I hooked up Sonnet’s Breakaway Box 550 with a variety of different Nvidia GPUs to accelerate the graphics performance of the system. The Quadro P6000 was the best option, as it used the same Quadro driver and Pascal architecture as the integrated P1000 GPU but greatly increased performance.

It allowed me to use my Lenovo Explorer WMR headset to edit 360 video in VR with Premiere Pro, and I was able to playback 8K DNxHR files at full resolution in Premiere to my Dell 8K LCD display. I was also able to watch 8K HEVC files in Windows movie player smoothly. Pretty impressive for a 15-inch convertible laptop, but the 6-Core Xeon processor pairs well with the desktop GPU, making this an ideal system to harness the workflow possibilities offered by eGPU solutions.

Media Export Benchmarks
I did extensive benchmark testing, measuring the export times of various media at different settings with different internal and external GPU options. The basic conclusion was that currently simple transcodes and conversions are not much different with an eGPU, but that once color correction and other effects are brought into the equation, increasing GPU power makes processing two to five times faster.

I also tested DCP exports with Quvis’ Wraptor plugin for AME and found the laptop took less than twice as long as my top-end desktop to make DCPs, which I consider to be a good thing. You can kick out a 4K movie trailer in under 10 minutes. And if you want to export a full feature film, I would recommend a desktop, but this will do it in a couple of hours.

Final Observations
The ZBook Studio x360 is a powerful machine and an optimal host for eGPU workflows. While it exceeded my performance expectations, I did not find the touch and ink solution to be optimal for my needs as I am a heavy keyboard user, even when doing artistic tasks. (To be clear, I haven’t found a better solution. This just doesn’t suitably replace my traditional mouse and keyboard approach to work.) So if buying one for myself, I would personally opt for the non-touch ZBook Studio model. But for anyone to whom inking is a critical part of their artistic workflow, who needs a powerful system on the go, this is a very capable model that doesn’t appear to have too many similar alternatives. It blends the power of the ZBook Studio with the inking experience of HP’s other x360 products.


Mike McCarthy is an online editor/workflow consultant with over 10 years of experience on feature films and commercials. He has been involved in pioneering new solutions for tapeless workflows, DSLR filmmaking and multi-screen and surround video experiences. Check out his site.

Review: eGPUs and the Sonnet Breakaway Box

By Mike McCarthy

As a laptop user and fan of graphics performance, I have always had to weigh the balance between performance and portability when selecting a system. And this usually bounces back and forth, as neither option is totally satisfactory. Systems are always too heavy or not powerful enough.

My first laptop when I graduated high school was the 16-inch Sony Vaio GRX570, with the largest screen available at the time, running 1600×1200 pixels. After four years carrying that around, I was eager to move to the Dell XPS M1210, the smallest laptop with a discrete GPU. That was followed by a Quadro-based Dell Precision M4400 workstation, which was on the larger side. I then bounced to the lightweight Carbon Fiber 13-inch Sony Vaio Z1 in 2010, which my wife still uses. This was followed by my current Aorus X3 Plus, which has both power (GF870M) and a small form factor (13 inch), but at the expense of everything else.

Some More History
The Vaio Z1 was one of the first hybrid graphics solutions to allow users to switch between different GPUs. Its GeForce 330M was powerful enough to run Adobe’s Mercury CUDA Playback engine in CS5, but was at the limit of its performance. It didn’t support my 30-inch display, and while the SSD storage solution had the throughput for 2K DPX playback, the GPU processing couldn’t keep up.

Other users were upgrading the GPU with an ExpressCard-based ViDock external PCIe enclosure, but a single-lane of PCIe 1.0 bandwidth (2Gb/s) wasn’t enough to make is worth the effort for video editing. (3D gaming requires less source bandwidth than video processing.) Sony’s follow-on Z2 model offered the first commercial eGPU, connected via LightPeak, the forerunner to Thunderbolt. It allowed the ultra-light Z series laptop to use an AMD Radeon 6650M GPU and Blu-ray drive in the proprietary Media Dock, presumably over a PCIe x4 1.0 (8Gb/s) connection.

Thunderbolt 3
Alienware also has a propriety eGPU solution for their laptops, but Thunderbolt is really what makes eGPUs a marketable possibility, giving direct access to the PCIe bus at x4 speed, in a standardized connection. The first generation offered a dedicated 10Gb connection, while Thunderbolt 2 increased that to a 20Gb shared connection. The biggest thing holding back eGPUs at that point was lack of PC adoption of the Apple technology licensed from Intel, and OS X limitations on eGPUs.

Thunderbolt 3 changed all of that, increasing the total connection bandwidth to 40Gb, the same as first-generation PCIe x16 cards. And far more systems support Thunderbolt 3 than the previous iterations. Integrated OS support for GPU switching in Windows 10 and OS X (built on laptop GPU power saving technology) further paved the path to eGPU adoption.

Why eGPUs Now?
Even with all of this in my favor, I didn’t take the step into eGPU solutions until very recently. I bought my personal system in 2014. This was just before Thunderbolt 3 hit the mainstream. The last two systems I reviewed had Thunderbolt 3, but didn’t need eGPUs with their mobile Quadro P4000 and P5000 internal GPUs. So I hadn’t had the opportunity to give it a go until I received an HP Zbook Studio x360 to review. Now, its integrated Quadro P1000 is nothing to scoff at, but there was significantly more room for performance gains from an external GPU.

Sonnet Breakaway Box
I have had the opportunity to review the 550W version of Sonnet’s Breakaway Box PCIe enclosure over the course of a few weeks, allowing me to test out a number of different cards, including four different GPUs, as well as my Red-Rocket-X and 10GbE cards. Sonnet has three different eGPU enclosure options, depending on the power requirements of your GPU.

They sent me the mid-level 550 model, which should support every card on the market, aside from AMD’s power-guzzling Vega 64-based GPUs. The base 350 model should support GF1080 or 2080 cards, but not overclocked Titanium or Titan versions. The 550 model includes two PCIe power cables that can be used in 6- or 8-pin connectors. This should cover any existing GPU on the market, and I have cards requiring nearly every possible combo — 6-pin, 8-pin, both, and dual 8-pin. Sonnet has a very thorough compatibility list available, for more specific details.

Installation
I installed my Quadro P6000 into the enclosure, because it used the same drivers as my internal Quadro P1000 GPU and would give me the most significant performance boost. I plugged the Thunderbolt connector into the laptop while it was booted. It immediately recognized the device, but only saw it as a “Microsoft Basic Display Adapter” until I re-installed my existing 411.63 Quadro drivers and rebooted. After that, it worked great, I was able to run my benchmarks and renders without issue, and I could see which GPU was carrying the processing load just by looking in the task manager performance tab.

Once I had finished my initial tests, safely removed the hardware in the OS and disconnected the enclosure, I swapped the installed card with my Quadro P4000 and plugged it back into the system without rebooting. It immediately detected it, and after a few seconds the new P4000 was recognized and accelerating my next set of renders. When I attempted to do the same procedure with my GeForce 2080TI, it did make me install the GeForce driver (416.16) and reboot before it would function at full capacity (subsequent transitions between Nvidia cards were seamless).

The next step was to try an AMD GPU, since I have a new RadeonPro WX8200 to test, which is a Pro version of the Vega 56 architecture. I was a bit more apprehensive about this configuration due to the integrated Nvidia card, and having experienced those drivers not co-existing well in the distant past. But I figured: “What’s the worst that could happen?”

Initially, plugging it in gave me the same Microsoft Basic Display Adapter device until I installed the RadeonPro drivers. Installing those drivers caused the system to crash and refuse to boot. Startup repair, system restore and OS revert all failed to run, let alone fix the issue. I was about to wipe the entire OS and let it reinstall from the recovery partition when I came across one more idea online. I was able to get to a command line in the pre-boot environment and run a Deployment Image Servicing and Management (DISM) command to see which drivers were installed — DISM /image:D:\ /Get-Drivers|more.

This allowed me to see that the last three drivers — oem172.inf through oem174.inf —were the only AMD-related ones on the system. I was able to remove them via the same tool — DISM /Image:D:\ /Remove-Driver /Driver:oem172.inf”) — and when I restarted, the system booted up just fine.

I then pulled the card from the eGPU box, wiped all the AMD files from the system, and vowed never to do something like that again. Lesson of the day: Don’t mix AMD and Nvidia cards and drivers. To AMDs credit, the WX8200 does not officially support eGPU installations, but extraneous drivers shouldn’t cause that much problem.

Performance Results
I tested Adobe Media Encoder export times with a variety of different sources and settings. Certain tests were not dramatically accelerated by the eGPU, while other renders definitely were. The main place we see differences between the integrated P1000 and a more-powerful external GPU is when effects are applied to high-res footage. That is when the GPU is really put to work, so those are the tests that improve with more GPU power. I had a one-minute sequence of Red clips with lots of effects (Lumetri, selective blur and mosaic: all GPU FX) that took 14 minutes to render internally, but finished in under four minutes with the eGPU attached. Exporting the same sequence with the effects disabled took four minutes internally and three minutes with the GPU. So the effects cost 10 minutes of render time internally, but under one minute of render time (35 seconds to be precise) when a powerful GPU is attached.

So if you are trying to do basic cuts-only editorial, an eGPU may not improve your performance much, but if you are doing VFX or color work, it can make a noticeable difference.

VR Headset Support
The external cards, of course, do increase performance in a measurable way, especially since I am using such powerful cards. It’s not just a matter of increasing render speeds, but about enabling functionality that was previously unavailable on the system. I connected my Lenovo Explorer WMR headset to the RTX2080TI in the Breakaway Box and gave it a shot. I was able to edit 360 video in VR in Premiere Pro, which is not supported on the included Quadro P1000 card. I did experience some interesting ghosting on occasion, where if I didn’t move my head everything looked perfect, but movement caused a double image — as if one eye was a frame behind the other — but the double image was appearing in each eye, as if there was an excessive motion blur applied to the rendered frames.

I thought this might be a delay based on extra latency in the Thunderbolt bus, but other times the picture looked crisp regardless of how quickly I moved my head. So it can work great, but there may need to be a few adjustments made to smooth things out. Lots of other users online report it working just fine, so there is probably a solution available out there.

Full-Resolution 8K Tests
I was able to connect my 8K display to the card as well, and while the x360 happens to support that display already (DP1.3 over Thunderbolt), most notebooks do not — and it increased the refresh rate from 30Hz to the full 60Hz. I was able to watch HEVC videos smoothly at 8K in Windows, and was able to playback 8K DNxHR files in Premiere at full res, as long as there were no edits or effects.

Just playing back footage at full 8K taxed the 2080TI at 80% compute utilization. But this is 8K we are talking about, playing back on a laptop, at full resolution. 4K anamorphic and 6K Venice X-OCN footage played back smoothly at half res in Premiere, and 8K Red footage played back at quarter. This is not the optimal solution for editing 8K footage, but it should have no problem doing serious work at UHD and 4K.

Other Cards and Functionality
GPUs aren’t the only PCIe cards that can be installed in the Breakaway Box, so I can add a variety of other functionality to my laptop if desired. Thunderbolt array controllers minimize the need for SATA or SAS cards in enclosures, but that is a possibility. I installed an Intel X520-DA2 10GbE card into the box and was copying files from my network at 700MB/s within a minute, without even having to install any new drivers. But unless you need to have SFP ports, most people looking for 10GbE functionality would be better served to look into Sonnet’s Solo 10G for smaller form factor, lower power use, and cheaper price. There are a variety of other options for Thunderbolt 3 to 10GbE hitting the market as well.

The Red-Rocket-X card has been a popular option for external PCIe enclosures over the last few years, primarily for on-set media transcoding. I installed mine in the Breakaway Box to give that functionality a shot as well.

I ran into two issues, both of which I was able to overcome, but are worth noting. First, the 6-pin power connector is challenging to fit into the poorly designed Rocket power port, due to the retention mechanism being offset for 8-pin compatibility. But it can fit if you work at it a bit, although I prefer to keep a 6-pin extension cable plugged into my Rocket since I move it around so much. Once I had all of the hardware hooked up, it was recognized in the OS, but installing the drivers from Red resulted in a Code-52 error that is usually associated with USB devices. The recommended solution online was to disable Windows 10 driver signing, in the pre-boot environment, and that did the trick. (My theory is that my HP’s SureStart security functionality was hesitating to give direct memory access to an external device, as that is the level of access Thunderbolt devices get to your system, and the Red Rocket-X driver wasn’t signed for that level of security.)

Anyhow, the card worked fine after that, and I verified that it accelerated my renders in Premiere Pro and AME. I am looking forward to a day when CUDA acceleration allows me to get that functionality out of my underused GPU power instead of requiring a dedicated card.

I did experience an issue with the Quadro P4000, where the fans spun up to 100% when the laptop went to shut off, hibernated, or went to sleep. None of the other cards had that issue, instead they shut off when the host system did and turned back on automatically when I booted up the system. I have no idea why the P4000 acted differently than the architecturally very similar P6000. Manually turning off the Breakaway Box or disconnecting the Thunderbolt cable solves the problem with the P4000, but then you have to remember to reconnect again when you are booting up.

In the process of troubleshooting the fan issue, I did a few other driver installs and learned a few tricks. First off, I already knew Quadro drivers can’t run GeForce cards (otherwise why pay for a Quadro), but GeForce drivers can run on Quadro cards. So it makes sense you would want to install GeForce drivers when mixing both types of GOUs. But I didn’t realize that apparently GeForce drivers take preference when they are installed. So when I had an issue with the internal Quadro card, reinstalling the Quadro drivers had no effect, since the GeForce drivers were running the hardware. Removing them (with DDU just to be thorough) solved the issue, and got everything operating seamlessly again. Sonnet’s support people were able to send me the solution to the problem on the first try. That was a bit of a hiccup, but once it was solved I could again swap between different GPUs without even rebooting. And most users will always have the same card installed when they connect their eGPU, further simplifying the issue.

Do you need an eGPU?
I really like this unit, and I think that eGPU functionality in general will totally change the high-end laptop market for the better. For people who only need high performance at their desk, there will be a class of top-end laptop with high-end CPU, RAM and storage, but no GPU to save on space and weight (CPU can’t be improved by external box, and needs to keep up with GPU).

There will be another similar class with mid-level GPUs to support basic 3D work on the road, but massive increases at home. I fall in the second category, as I can’t forego all GPU acceleration when I am traveling or even walking around the office. But I don’t need to be carrying around an 8K rendering beast all the time either. I can limit my gaming, VR work and heavy renders to my desk. That is the configuration I have been able to use with this ZBook x360.: enough power to edit un-tethered, but combining the internal 6-core CPU with a top -end external GPU gives great performance when attached to the Breakaway Box. As always, I still want to go smaller, and plan to test with an even lighter weight laptop as soon as the opportunity arises.

Summing Up
The Breakaway Box is a simple solution to a significant issue. No bells and whistles, which I initially appreciated. But the eGPU box is inherently a docking station, so there is an argument to be made for adding other functionality. In my case, once I am setup at my next project, using a 10GbE adapter in the second TB3 port on my laptop will be a better solution for top performance and bandwidth anyway.

So I am excited about the possibilities that eGPUs bring to the table, now that they are fully supported by the OS and applications I use, and I don’t imagine buying a laptop setup without one anytime in the foreseeable future. The Sonnet Breakaway Box meets my needs and has performed very well for me over the last few weeks.


Mike McCarthy is an online editor/workflow consultant with over 10 years of experience on feature films and commercials. He has been involved in pioneering new solutions for tapeless workflows, DSLR filmmaking and multi-screen and surround video experiences. Check out his site.

New codec, workflow options via Red, Nvidia and Adobe

By Mike McCarthy

There were two announcements last week that will impact post production workflows. The first was the launch of Red’s new SDK, which leverages Nvidia’s GPU-accelerated CUDA framework to deliver realtime playback of 8K Red footage. I’ll get to the other news shortly. Nvidia was demonstrating an early version of this technology at Adobe Max in October, and I have been looking forward to this development since I am about to start post on a feature film shot on the Red Monstro camera. This should effectively render the RedRocket accelerator cards obsolete, replacing them with cheaper, multipurpose hardware that can also accelerate other computational tasks.

While accelerating playback of 8K content at full resolution requires a top-end RTX series card from Nvidia (Quadro RTX 6000, Titan RTX or GeForce RTX 2080Ti), the technology is not dependent on RTX’s new architecture (RT and Tensor cores), allowing earlier generation hardware to accelerate smooth playback at smaller frame sizes. Lots of existing Red footage is shot at 4K and 6K, and playback of these files will be accelerated on widely deployed legacy products from previous generations of Nvidia GPU architecture. It will still be a while before this functionality is in the hands of end users, because now Adobe, Apple, Blackmagic and other software vendors have to integrate the new SDK functionality into their individual applications. But hopefully we will see those updates hitting the market soon (targeting late Q1 of 2019).

Encoding ProRes on Windows via Adobe apps
The other significant update, which is already available to users as of this week, is Adobe’s addition of ProRes encoding support on its video apps in Windows. Developed by Apple, ProRes encoding has been available on Mac for a long time, and ProRes decoding and playback has been available on Windows for over 10 years. But creating ProRes files on Windows has always been a challenge. Fixing this was less a technical challenge than a political one, as Apple owns the codec and it is not technically a standard. So while there were some hacks available at various points during that time, Apple has severely restricted the official encoding options available on Windows… until now.

With the 13.0.2 release of Premiere Pro and Media Encoder, as well as the newest update to After Effects, Adobe users on Windows systems can now create ProRes files in whatever flavor they happen to need. This is especially useful since many places require delivery of final products in the ProRes format. In this case, the new export support is obviously a win all the way around.

Adobe Premiere

Now users have yet another codec option for all of their intermediate files, prompting another look at the question: Which codec is best for your workflow? With this release, Adobe users have at least three major options for high-quality intermediate codecs: Cineform, DNxHR and now ProRes. I am limiting the scope to integrated cross-platform codecs supporting 10-bit color depth, variable levels of image compression and customizable frame sizes. Here is a quick overview of the strengths and weaknesses of each option:

ProRes
ProRes was created by Apple over 10 years ago and has become the de-facto standard throughout the industry, regardless of the fact that it is entirely owned by Apple. ProRes is now fully cross-platform compatible, has options for both YUV and RGB color and has six variations, all of which support at least 10-bit color depth. The variable bit rate compression scheme scales well with content complexity, so encoding black or static images doesn’t require as much space as full-motion video. It also supports alpha channels with compression, but only in the 444 variants of the codec.

Recent tests on my Windows 10 workstation resulted in ProRes taking 3x to 5x as much CPU power to playback as similar DNxHR of Cineform files, especially as frame sizes get larger. The codec supports 8K frame sizes but playback will require much more processing power. I can’t even playback UHD files in ProRes 444 at full resolution, while the Cineform and DNxHR files have no problem, even at 444. This is less of concern if you are only working at 1080p.

Multiply those file sizes by four for UHD content (and by 16 for 8K content).

Cineform
Cineform, which has been available since 2004, was acquired by GoPro in 2011. They have licensed the codec to Adobe, (among other vendors) and it is available as “GoPro Cineform” in the AVI or QuickTime sections of the Adobe export window. Cineform is a wavelet compression codec, with 10-bit YUV and 12-bit RGB variants, which like ProRes support compressed alpha channels in the RGB variant. The five levels of encoding quality are selected separately from the format, so higher levels of compression are available for 4444 content compared to the limited options available in the other codecs.

It usually plays back extremely efficiently on Windows, but my recent tests show that encoding to the format is much slower than it used to be. And while it has some level of support outside of Adobe applications, it is not as universally recognized as ProRes or DNxHD.

DNxHD
DNxHD was created by Avid for compressed HD playback and has now been extended to DNxHR (high resolution). It is a fixed bit rate codec, with each variant having a locked multiplier based on resolution and frame rate. This makes it easy to calculate storage needs but wastes space for files that are black or contain a lot of static content. It is available in MXF and Mov wrappers and has five levels of quality. The top option is 444 RGB, and all variants support alpha channels in Mov but uncompressed, which takes a lot of space. For whatever reason, Adobe has greatly optimized DNxHR playback in Premiere Pro, of all variants, in both MXF and Mov wrappers. On my project 6Below, I was able to get 6K 444 files to playback, with lots of effects, without dropping frames. The encodes to and from DNxHR are faster in Adobe apps as well.

So for most PC Adobe users, DNxHR-LB (low bandwidth) is probably the best codec to use for intermediate work. We are using it to offline my current project, with 2.2K DNxHR-LB, Mov files. People with a heavy Mac interchange may lean toward ProRes, but up your CPU specs for the same level of application performance.


Mike McCarthy is an online editor/workflow consultant with 10 years of experience on feature films and commercials. He has been involved in pioneering new solutions for tapeless workflows, DSLR filmmaking and multi-screen and surround video experiences. Check out his site.

Making an animated series with Adobe Character Animator

By Mike McCarthy

In a departure from my normal film production technology focus, I have also been working on an animated web series called Grounds of Freedom. Over the past year I have been directing the effort and working with a team of people across the country who are helping in various ways. After a year of meetings, experimentation and work we finally started releasing finished episodes on YouTube.

The show takes place in Grounds of Freedom, a coffee shop where a variety of animated mini-figures gather to discuss freedom and its application to present-day cultural issues and events. The show is created with a workflow that weaves through a variety of Adobe Creative Cloud apps. Back in October I presented our workflow during Adobe Max in LA, and I wanted to share it with postPerspective’s readers as well.

When we first started planning for the series, we considered using live action. Ultimately, after being inspired by the preview releases of Adobe Character Animator, I decided to pursue a new digital approach to brick filming (a film made using Legos), which is traditionally accomplished through stop-motion animation. Once everyone else realized the simpler workflow possibilities and increased level of creative control offered by that new animation process, they were excited to pioneer this new approach. Animation gives us more control and flexibility over the message and dialog, lowers production costs and eases collaboration over long distances, as there is no “source footage” to share.

Creating the Characters
The biggest challenge to using Character Animator is creating digital puppets, which are deeply layered Photoshop PSDs with very precise layer naming and stacking. There are ways to generate the underlying source imagery in 3D animation programs, but I wanted the realism and authenticity of sourcing from actual photographs of the models and figures. So we took lots of 5K macro shots of our sets and characters in various positions with our Canon 60D and 70D DSLRs and cut out hundreds of layers of content in Photoshop to create our characters and all of their various possible body positions. The only thing that was synthetically generated was the various facial expressions digitally painted onto their clean yellow heads, usually to match an existing physical reference character face.

Mike McCarthy shooting stills.

Once we had our source imagery organized into huge PSDs, we rigged those puppets in Character Animator with various triggers, behaviors and controls. The walking was accomplished by cycling through various layers, instead of the default bending of the leg elements. We created arm movement by mapping each arm position to a MIDI key. We controlled facial expressions and head movement via webcam, and the mouth positions were calculated by the program based on the accompanying audio dialog.

Animating Digital Puppets
The puppets had to be finished and fully functional before we could start animating on the digital stages we had created. We had been writing the scripts during that time, parallel to generating the puppet art, so we were ready to record the dialog by the time the puppets were finished. We initially attempted to record live in Character Animator while capturing the animation motions as well, but we didn’t have the level of audio editing functionality we needed available to us in Character Animator. So during that first session, we switched over to Adobe Audition, and planned to animate as a separate process, once the audio was edited.

That whole idea of live capturing audio and facial animation data is laughable now, looking back, since we usually spend a week editing the dialog before we do any animating. We edited each character audio on a separate track and exported those separate tracks to Character Animator. We computed lipsync for each puppet based on their dedicated dialog track and usually exported immediately. This provided a draft visual that allowed us to continue editing the dialog within Premiere Pro. Having a visual reference makes a big difference when trying to determine how a conversation will feel, so that was an important step — even though we had to throw away our previous work in Character Animator once we made significant edit changes that altered sync.

We repeated the process once we had a more final edit. We carried on from there in Character Animator, recording arm and leg motions with the MIDI keyboard in realtime for each character. Once those trigger layers had been cleaned up and refined, we recorded the facial expressions, head positions and eye gaze with a single pass on the webcam. Every re-record to alter a particular section adds a layer to the already complicated timeline, so we limited that as much as possible, usually re-recording instead of making quick fixes unless we were nearly finished.

Compositing the Characters Together
Once we had fully animated scenes in Character Animator, we would turn off the background elements, and isolate each character layer to be exported in Media Encoder via dynamic link. I did a lot of testing before settling on JPEG2000 MXF as the format of choice. I wanted a highly compressed file, but need alpha channel support, and that was the best option available. Each of those renders became a character layer, which was composited into our stage layers in After Effects. We could have dynamically linked the characters directly into AE, but with that many layers that would decrease performance for the interactive part of the compositing work. We added shadows and reflections in AE, as well as various other effects.

Walking was one of the most challenging effects to properly recreate digitally. Our layer cycling in Character Animator resulted in a static figure swinging its legs, but people (and mini figures) have a bounce to their step, and move forward at an uneven rate as they take steps. With some pixel measurement and analysis, I was able to use anchor point keyframes in After Effects to get a repeating movement cycle that made the character appear to be walking on a treadmill.

I then used carefully calculated position keyframes to add the appropriate amount of travel per frame for the feet to stick to the ground, which varies based on the scale as the character moves toward the camera. (In my case the velocity was half the scale value in pixels per seconds.) We then duplicated that layer to create the reflection and shadow of the character as well. That result can then be composited onto various digital stages. In our case, the first two shots of the intro were designed to use the same walk animation with different background images.

All of the character layers were pre-comped, so we only needed to update a single location when a new version of a character was rendered out of Media Encoder, or when we brought in a dynamically linked layer. It would propagate all the necessary comp layers to generate updated reflections and shadows. Once the main compositing work was finished, we usually only needed to make slight changes in each scene between episodes. These scenes were composited at 5K, based on the resolution off the DSLR photos of the sets we had built. These 5K plates could be dynamically linked directly into Premiere Pro, and occasionally used later in the process to ripple slight changes through the workflow. For the interactive work, we got far better editing performance by rendering out flattened files. We started with DNxHR 5K assets, but eventually switched to HEVC files since they were 30x smaller and imperceptibly different in quality with our relatively static animated content.

Editing the Animated Scenes
In Premiere Pro, we had the original audio edit, and usually a draft render of the characters with just their mouths moving. Once we had the plate renders, we placed them each in their own 5K scene sub-sequence and used those sequences as source on our master timeline. This allowed us to easily update the content when new renders were available, or source from dynamically linked layers instead if needed. Our master timeline was 1080p, so with 5K source content we could push in two and a half times the frame size without losing resolution. This allowed us to digitally frame every shot, usually based on one of two rendered angles, and gave us lots of flexibility all the way to the end of the editing process.

Collaborative Benefits of Dynamic Link
While Dynamic Link doesn’t offer the best playback performance without making temp renders, it does have two major benefits in this workflow. It ripples change to the source PSD all the way to the final edit in Premiere just by bringing each app into focus once. (I added a name tag to one character’s PSD during my presentation, and 10 seconds later, it was visible throughout my final edit.) Even more importantly, it allows us to collaborate online without having to share any exported video assets. As long as each member of the team has the source PSD artwork and audio files, all we have to exchange online are the Character Animator project (which is small once the temp files are removed), the .AEP file and the .PrProj file.

This gives any of us the option to render full-quality visual assets anytime we need them, but the work we do on those assets is all contained within the project files that we sync to each other. The coffee shop was built and shot in Idaho, our voice artist was in Florida, our puppets faces were created in LA. I animate and edit in Northern California, the AE compositing was done in LA, and the audio is mixed in New Jersey. We did all of that with nothing but a Dropbox account, using the workflow I have just outlined.

Past that point, it was a fairly traditional finish, in that we edited in music and sound effects, and sent an OMF to Steve, our sound guy at DAWPro Studios http://dawpro.com/photo_gallery.html for the final mix. During that time we added other b-roll visuals or other effects, and once we had the final audio back we rendered the final result to H.264 at 1080p and uploaded to YouTube.


Mike McCarthy is an online editor/workflow consultant with over 10 years of experience on feature films and commercials. He has been involved in pioneering new solutions for tapeless workflows, DSLR filmmaking and multi-screen and surround video experiences. Check out his site.

A Technologist’s Data Storage Primer

By Mike McCarthy

Storage is the concept of keeping all of the files for a particular project or workflow, but they may not all be stored in the same place — different types of data have different requirements and different storage solutions have different strengths and features.

At a fundamental level, most digital data is stored on HDDs or SSDs. HDDs, or hard disk drives, are mechanical devices that store the data on a spinning magnetic surface and move read/write heads over that surface to access the data. They currently max out around 200MB/s and 5ms latency.

SSDs, or solid-state drives, involve no moving parts. SSDs can be built with a number of different architectures and interfaces, but most are based on the same basic Flash memory technology as the CF or SD card in your camera. Some SSDs are SATA drives that use the same interface and form factor as a spinning disk for easy replacement in existing HDD-compatible devices. These devices are limited to SATA’s bandwidth of 600MB/s. Other SSDs use the PCIe interface, either in full-sized PCIe cards or the smaller M.2 form factor. These have much higher potential bandwidths, up to 3000MB/s.

Currently, HDDs are much cheaper for storing large quantities of data but require some level of redundancy for security. SSDs are also capable of failure, but it is a much more rare occurrence. Data recovery for either is very expensive. SSDs are usually cheaper for achieving high bandwidth, unless large capacities are also needed.

RAIDs
Traditionally, hard drives used in professional contexts are grouped together for higher speeds and better data security. These are called RAIDs, which stands for redundant array of independent disks. There are a variety of different approaches to RAID design that are very different from one another.

RAID-0 or striping is technically not redundant, but every file is split across each disk, so each disk only has to retrieve its portion of a requested file. Since these happen in parallel, the result is usually faster than if a single disk had read the entire file, especially for larger files. But if one disk fails, every one of your files will be missing a part of its data, making the remaining partial information pretty useless. The more disks in the array, the higher the chances of one failing, so I rarely see striped arrays composed of more than four disks. It used to be popular to create striped arrays for high-speed access to restorable data, like backed-up source footage, or temp files, but now a single PCIe SSD is far faster, cheaper, smaller and more efficient in most cases.

Sugar

RAID-1 or mirroring is when all of the data is written to more than one drive. This limits the array’s capacity to the size of the smallest source volume, but the data is very secure. There is no speed benefit to writes since each drive must write all of the data, but reads can be distributed across the identical drives with similar performance as RAID-0.

RAID-3, -5 and -6 try to achieve a balance between those benefits for larger arrays with more disks (minimum three). They all require more complicated controllers, so they are more expensive for the same levels of performance. RAID-3 stripes data across all but one drive and then calculates and stores parity (odd/even) data across the data drives and stores it on the last drive. This allows the data from any single failed drive to be restored, based on the parity data. RAID-5 is similar, but the parity volume is alternated depending on the block, allowing the reads to be shared across all disks, not just the “data drives.”

So the capacity of a RAID-3 or RAID-5 array will be the minimum individual disk capacity times the number of disks minus one. RAID-6 is similar but stores two drives worth of parity data, which via some more advanced math than odd/even, allows it to restore the data even if two drives fail at the same time. RAID-6 capacity will be the minimum individual disk capacity times the number of disks minus two, and is usually only used on arrays with many disks. RAID-5 is the most popular option for most media storage arrays, although RAID-6 becomes more popular as the value of the data stored increases and the price of extra drives decreases over time.

Storage Bandwidth
Digital data is stored as a series of ones and zeros, each of which is a bit. One byte is 8 bits, which frequently represents one letter of text, or one pixel of an image (8-bit single channel). Bits are frequently referenced in large quantities to measure data rates, while bytes are usually referenced when measuring stored data. I prefer to use bytes for both purposes, but it is important to know the difference. A Megabit (Mb) is one million bits, while a Megabyte (MB) is one million bytes, or 8 million bits. Similar to metric, Kilo is thousand, Mega is million, Giga is billion, and Tera is trillion. Anything beyond that you can learn as you go.

Networking speeds are measured in bits (Gigabits), but with headers and everything else, it is safer to divide by 10 when converting speed into bytes per second. Estimate 100MB/s for Gigabit, up to 1000MB/s on 10GB, and around 500MB/s for the new N-BaseT standard. Similarly, when transferring files over a 30Mb Internet connection, expect around 3MB/s, then multiple by 60 or 3,600 to get to minutes or hours (180MB/min or 9600MB/hr in this case). So if you have to download a 10GB file on that connection, come back to check on it in an hour.

Magnopus

Because networking standards are measured in bits, and because networking is so important for sharing video files, many video file types are measured in bits as well. An 8Mb H.264 stream is 1MB per second. DNxHD36 is 36Mb/s (or 4.5MB/s when divided by eight), DV and HDV are 25Mb, DVCProHD is 100Mb, etc. Other compression types have variable bit rates depending on the content, but there are still average rates we can make calculations from. Any file’s size divided by its duration will reveal its average data rate. It is important to make sure that your storage has the bandwidth to handle as many streams of video as you need, which will be that average data rate times the number of streams. So 10 streams of DNxHD36 will be 360Mb or 45MB/s.

The other issue to account for is IO requests and drive latency. Lots of small requests require not just high total transfer rates, but high IO performance as well. Hard drives can only fulfill around 100 individual requests per second, regardless of how big those requests are. So while a single drive can easily sustain a 45MB/s stream, satisfying 10 different sets of requests may keep it so busy bouncing between the demands that it can’t keep up. You may need a larger array, with a higher number of (potentially) smaller disks to keep up with the IO demands of multiple streams of data. Audio is worse in this regard in that you are dealing with lots of smaller individual files as your track count increases, even though the data rate is relatively low. SSDs are much better at handling larger numbers of individual requests, usually measured in the thousands or tens of thousands per second per drive.

Storage Capacity
Capacity on the other hand is simpler. Megabytes are usually the smallest increments of data that we have to worry about calculating. A media type’s data rate (in MB/sec) times its duration (in seconds) will give you its expected file size. If you are planning to edit a feature film with 100 hours of offline content in DNxHD36, that is 3600×100 seconds, times 4.5MB/s, equaling 1620000MB, 1620GB, or simply about 1.6TB. But you should add some headroom for unexpected needs, and then a 2TB disk is about 1.8TB when formatted, so it will just barely fit. It is probably worth sizing up to at least 3TB if you are planning to store your renders and exports on there as well.

Once you have a storage solution of the required capacity there is still the issue of connecting it to your system. The most expensive options connect through the network to make them easier to share (although more is required for true shared storage), but that isn’t actually the fastest option or the cheapest. A large array can be connected over USB3 or Thunderbolt, or via the SATA or SAS protocol directly to an internal controller.

There are also options for Fibre Channel, which can allow sharing over a SAN, but this is becoming less popular as 10GbE becomes more affordable. Gigabit Ethernet and USB3 won’t be fast enough for high-bandwidth files to playback, but 10GbE, multichannel SAS, Fibre Channel and Thunderbolt can all handle almost anything up to uncompressed 4K.

Direct attached storage will always have the highest bandwidth and lowest latency, as it has the fewest steps between the stored files and the user. Using Thunderbolt or USB adds another controller and hop, Ethernet even more so.

Different Types of Project Data
Now that we know the options for storage, let’s look at the data we anticipate needing to store. First off we will have lots of video footage of source media (either camera original files, transcoded editing dailies, or both). This is usually in the Terabytes, but the data rates vary dramatically — from 1Mb H.264 files to 200Mb ProRes files to 2400Mb Red files. The data rate for the files you are playing back, combined with the number of playback streams you expect to use, will determine the bandwidth you need from your storage system. These files are usually static in that they don’t get edited or written to in any way after creation.

The exceptions would be sidecar files like RMD and XML files, which will require write access to the media volume. If a certain set of files is static, as long as a backup of the source data exists, they don’t need to be backed up on a regular basis and don’t even necessarily need redundancy. Although if the cost of restoring that data would be high, in regards to lost time during that process, some level of redundancy is still recommended.

Another important set of files we will have is our project files, which actually record the “work” we do in our application. They contain instructions for manipulating our media files during playback or export. The files are usually relatively small, and are constantly changing as we use them. That means they need to be backed up on a regular basis. The more frequent the backups, the less work you lose when something goes wrong.

We will also have a variety of exports and intermediate renders over the course of the project. Whether they are flattened exports for upload and review, VFX files or other renders, these are a more dynamic set of files than our original source footage. And they are generated on our systems instead of being imported from somewhere else. These can usually be regenerated from their source projects, if necessary, but the time and effort required usually makes it worth it to invest in protecting or backing them up. In most workflows, these files don’t change once they are created, which makes it easier to back them up if desired.

There will also be a variety of temp files generated by most editing or VFX programs. Some of these files need high-speed access for best application performance, but they rarely need to be protected or backed up because they can be automatically regenerated by the source applications on the fly if needed.

Choosing the Right Storage for Your Needs
Ok, so we have source footage, project files, exports and temp files that we need to find a place for. If you have a system or laptop with a single data volume, the answer is simple: It all goes on the C drive. But we can achieve far better performance if we have the storage infrastructure to break those files up onto different devices. Newer laptops frequently have both a small SSD and a larger hard disk. In that case we would want our source footage on the (larger) HDD, while the project files should go on the (safer) SSD.

Usually your temp file directories should be located on the SSD as well since it is faster, and your exports can go either place, preferably the SSD if they fit. If you have an external drive of source footage connected, you can back all files up there, but you should probably work from projects stored on the local system, playing back media from the external drive.

A professional workstation can have a variety of different storage options available. I have a system with two SSDs and two RAIDs, so I store my OS and software on one SSD, my projects and temp files on the other SSD, my source footage on one RAID and my exports on the other. I also back up my project folder to the exports RAID on a daily basis, since the SSDs have no redundancy.

Individual Store Solution Case Study Examples
If you are natively editing a short film project shot on Red, then R3Ds can be 300MB/s. That is 1080GB/hour, so five hours of footage will be just over 5TB. It could be stored on a single 6TB external drive, but that won’t give you the bandwidth to play back in real-time (hard drives usually top out around 200MB/s).

Striping your data across two drives in one of those larger external drives would probably provide the needed performance, but with that much data you are unlikely to have a backup elsewhere. So data security becomes more of a concern, leading us toward a RAID-5-based solution. A four-disk array of 2TB drives provides 6TB of usable storage at RAID-5 (4x2TB = 8TB raw capacity, minus 2TB of parity data equals 6TB of usable storage capacity). Using an array of 8 1TB drives would provide higher performance, and 7TB of space before formatting (8x1TB = 8TB raw capacity, minus 1TB of parity, because a single drive failure would only lose 1TB of data in this configuration) but will cost more. (eight-port RAID controller, eight-bay enclosure, and two 1TB drives are usually more expensive than one 2TB drive.)

Larger projects deal with much higher numbers. Another project has 200TB of Red footage that needs to be accessible on a single volume. A 24-bay enclosure with 12TB drives provides 288TB of space, minus two drives worth of data for RAID-6 redundancy (288TB raw-[2x12TB for parity]=264TB usable capacity), which will be more like 240TB of space available in Windows once it is formatted.

Sharing Storage and Files With Others
As Ethernet networking technology has improved, the benefits of expensive SAN (storage area network) solutions over NAS (network attached storage) solutions has diminished. 10Gigabit Ethernet (10GbE) transfers over 1GB of data a second and is relatively cheap to implement. NAS has the benefit of a single host system controlling the writes, usually with software included in the OS. This prevents data corruption and also isolates the client devices from the file system allowing PC, Mac and Linux devices to all access the same files. This comes at the cost of slightly increased latency and occasionally lower total bandwidth, but the prices and complexity of installation are far lower.

So now all but the largest facilities and most demanding workflows are being deployed with NAS-based shared storage solutions. This can be as simple as a main editing system with a large direct attached array sharing its media with an assistant station, over a direct 10GbE link, for about $50. This can be scaled up by adding a switch and connecting more users to it, but the more users sharing the data, the greater the impact on the host system, and the lower the overall performance. Over 3-4 users, it becomes prudent to have a dedicated host system for the storage, for both performance and stability. Once you are buying a dedicated system, there are a variety of other functionalities offered by different vendors to improve performance and collaboration.

Bin Locking and Simultaneous Access
The main step to improve collaboration is to implement what is usually referred to as a “bin locking system.” Even with a top-end SAN solution and strict permissions controls there is still the possibility of users overwriting each other’s work, or at the very least branching the project into two versions that can’t easily be reconciled.

If two people are working on the same sequence at the same time, only one of their sets of changes is going to make it to the master copy of the file without some way of combining the changes (and solutions are being developed). But usually the way to avoid that is to break projects down into smaller pieces and make sure that no two people are ever working on the exact same part. This is accomplished by locking the part (or bin) of the project that a user is editing so that no one else may edit it at the same time. This usually requires some level of server functionality because it involves changes that are not always happening at the local machine.

Avid requires specific support for that from the storage host in order for it to enable that feature. Adobe on the other hand has implemented a simpler storage-based solution, which is effective but not infallible, that works on any shared storage device that offers users write access.

A Note on iSCSI
iSCSI arrays offer some interesting possibilities for read-only data, like source footage, as iSCSI gives block-level access for maximum performance and runs on any network without expensive software. The only limit is that only one system can copy new media to the volume, and there must be a secure way to ensure the remaining systems have read-only access. Projects and exports must be stored elsewhere, but those files require much less capacity and bandwidth than source media. I have not had the opportunity to test out this hybrid SAN theory since I don’t have iSCSI appliances to test with.

A Note on Top-End Ethernet Options
40Gb Ethernet products have been available for a while and we are now seeing 25GB and 100Gb Ethernet products as well. 40Gb cards can be gotten quite cheaply, and I was tempted to use them for direct connect, hoping to see 4GB/s to share fast SSDs between systems. But 40Gb Ethernet is actually a trunk of four parallel 10Gb links and each individual connection is limited to 10Gb. It is easy to share the 40Gb of aggregate bandwidth across 10 systems accessing a 40Gb storage host, but very challenging to get more than 10Gb to a single client system. Having extra lanes on the highway doesn’t get you to work any faster if there are no other cars on the road, it only helps when there is lots of competing traffic.

25Gb Ethernet on the other hand will give you access to nearly 3GB/s for single connections, but as that is newer technology, the prices haven’t come down yet ($500 instead of $50 for a 10GbE direct link). 100Gb Ethernet is four 25Gb links trunked together, and subject to the same aggregate limitations as 40Gb.

Main Image: Courtesy of Sugar Studios LA


Mike McCarthy is an online editor/workflow consultant with 10 years of experience on feature films and commercials. He has been involved in pioneering new solutions for tapeless workflows, DSLR filmmaking and multi-screen and surround video experiences. Check out his site.

Adobe Max 2018: Creative Cloud updates and more

By Mike McCarthy

I attended my first Adobe Max 2018 last week in Los Angeles. This huge conference takes over the LA convention center and overflows into the surrounding venues. It began on Monday morning with a two-and-a-half-hour keynote outlining the developments and features being released in the newest updates to Adobe’s Creative Cloud. This was followed by all sorts of smaller sessions and training labs for attendees to dig deeper into the new capabilities of the various tools and applications.

The South Hall was filled with booths from various hardware and software partners, with more available than any one person could possibly take in. Tuesday started off with some early morning hands-on labs, followed by a second keynote presentation about creative and career development. I got a front row seat to hear five different people, who are successful in their creative fields — including director Ron Howard — discuss their approach to work and life. The rest of the day was so packed with various briefings, meetings and interviews that I didn’t get to actually attend any of the classroom sessions.

By Wednesday, the event was beginning to wind down, but there was still a plethora of sessions and other options for attendees to split their time. I presented the workflow for my most recent project Grounds of Freedom at Nvidia’s booth in the community pavilion, and spent the rest of the time connecting with other hardware and software partners who had a presence there.

Adobe released updates for most of its creative applications concurrent with the event. Many of the most relevant updates to the video tools were previously announced at IBC in Amsterdam last month, so I won’t repeat those, but there are still a few new video ones, as well as many that are broader in scope in regards to media as a whole.

Adobe Premiere Rush
The biggest video-centric announcement is Adobe Premiere Rush, which offers simplified video editing workflows for mobile devices and PCs.  Currently releasing on iOS and Windows, with Android to follow in the future, it is a cloud-enabled application, with the option to offload much of the processing from the user device. Rush projects can be moved into Premiere Pro for finishing once you are back on the desktop.  It will also integrate with Team Projects for greater collaboration in larger organizations. It is free to start using, but most functionality will be limited to subscription users.

Let’s keep in mind that I am a finishing editor for feature films, so my first question (as a Razr-M user) was, “Who wants to edit video on their phone?” But what if the user shot the video on their phone? I don’t do that, but many people do, so I know this will be a valuable tool. This has me thinking about my own mentality toward video. I think if I was a sculptor I would be sculpting stone, while many people are sculpting with clay or silly putty. Because of that I would have trouble sculpting in clay and see little value in tools that are only able to sculpt clay. But there is probably benefit to being well versed in both.

I would have no trouble showing my son’s first-year video compilation to a prospective employer because it is just that good — I don’t make anything less than that. But there was no second-year video, even though I have the footage because that level of work takes way too much time. So I need to break free from that mentality, and get better at producing content that is “sufficient to tell a story” without being “technically and artistically flawless.” Learning to use Adobe Rush might be a good way for me to take a step in that direction. As a result, we may eventually see more videos in my articles as well. The current ones took me way too long to produce, but Adobe Rush should allow me to create content in a much shorter timeframe, if I am willing to compromise a bit on the precision and control offered by Premiere Pro and After Effects.

Rush allows up to four layers of video, with various effects and 32-bit Lumetri color controls, as well as AI-based audio filtering for noise reduction and de-reverb and lots of preset motion graphics templates for titling and such.  It should allow simple videos to be edited relatively easily, with good looking results, then shared directly to YouTube, Facebook and other platforms. While it doesn’t fit into my current workflow, I may need to create an entirely new “flow” for my personal videos. This seems like an interesting place to start, once they release an Android version and I get a new phone.

Photoshop Updates
There is a new version of Photoshop released nearly every year, and most of the time I can’t tell the difference between the new and the old. This year’s differences will probably be a lot more apparent to most users after a few minutes of use. The Undo command now works like other apps instead of being limited to toggling the last action. Transform operates very differently, in that they made proportional transform the default behavior instead of requiring users to hold Shift every time they scale. It allows the anchor point to be hidden to prevent people from moving the anchor instead of the image and the “commit changes” step at the end has been removed. All positive improvements, in my opinion, that might take a bit of getting used to for seasoned pros. There is also a new Framing Tool, which allows you to scale or crop any layer to a defined resolution. Maybe I am the only one, but I frequently find myself creating new documents in PS just so I can drag the new layer, that is preset to the resolution I need, back into my current document. For example, I need a 200x300px box in the middle of my HD frame — how else do you do that currently? This Framing tool should fill that hole in the features for more precise control over layer and object sizes and positions (As well as provide its easily adjustable non-destructive masking.).

They also showed off a very impressive AI-based auto selection of the subject or background.  It creates a standard selection that can be manually modified anywhere that the initial attempt didn’t give you what you were looking for.  Being someone who gives software demos, I don’t trust prepared demonstrations, so I wanted to try it for myself with a real-world asset. I opened up one of my source photos for my animation project and clicked the “Select Subject” button with no further input and got this result.  It needs some cleanup at the bottom, and refinement in the newly revamped “Select & Mask” tool, but this is a huge improvement over what I had to do on hundreds of layers earlier this year.  They also demonstrated a similar feature they are working on for video footage in Tuesday night’s Sneak previews.  Named “Project Fast Mask,” it automatically propagates masks of moving objects through video frames and, while not released yet, it looks promising.  Combined with the content-aware background fill for video that Jason Levine demonstrated in AE during the opening keynote, basic VFX work is going to get a lot easier.

There are also some smaller changes to the UI, allowing math expressions in the numerical value fields and making it easier to differentiate similarly named layers by showing the beginning and end of the name if it gets abbreviated.  They also added a function to distribute layers spatially based on the space between them, which accounts for their varying sizes, compared to the current solution which just evenly distributes based on their reference anchor point.

In other news, Photoshop is coming to iPad, and while that doesn’t affect me personally, I can see how this could be a big deal for some people. They have offered various trimmed down Photoshop editing applications for iOS in the past, but this new release is supposed to be based on the same underlying code as the desktop version and will eventually replicate all functionality, once they finish adapting the UI for touchscreens.

New Apps
Adobe also showed off Project Gemini, which is a sketch and painting tool for iPad that sits somewhere between Photoshop and Illustrator. (Hence the name, I assume) This doesn’t have much direct application to video workflows besides being able to record time-lapses of a sketch, which should make it easier to create those “white board illustration” videos that are becoming more popular.

Project Aero is a tool for creating AR experiences, and I can envision Premiere and After Effects being critical pieces in the puzzle for creating the visual assets that Aero will be placing into the augmented reality space.  This one is the hardest for me to fully conceptualize. I know Adobe is creating a lot of supporting infrastructure behind the scenes to enable the delivery of AR content in the future, but I haven’t yet been able to wrap my mind around a vision of what that future will be like.  VR I get, but AR is more complicated because of its interface with the real world and due to the variety of forms in which it can be experienced by users.  Similar to how web design is complicated by the need to support people on various browsers and cell phones, AR needs to support a variety of use cases and delivery platforms.  But Adobe is working on the tools to make that a reality, and Project Aero is the first public step in that larger process.

Community Pavilion
Adobe’s partner companies in the Community Pavilion were showing off a number of new products.  Dell has a new 49″ IPS monitor, the U4919DW, which is basically the resolution and desktop space of two 27-inch QHD displays without the seam (5120×1440 to be exact). HP was displaying their recently released ZBook Studio x360 convertible laptop workstation, (which I will be posting a review of soon), as well as their Zbook X2 tablet and the rest of their Z workstations.  NVidia was exhibiting their new Turing-based cards with 8K Red decoding acceleration, ray tracing in Adobe Dimension and other GPU accelerated tasks.  AMD was demoing 4K Red playback on a MacBookPro with an eGPU solution, and CPU based ray-tracing on their Ryzen systems.  The other booths spanned the gamut from GoPro cameras and server storage devices to paper stock products for designers.  I even won a Thunderbolt 3 docking station at Intel’s booth. (Although in the next drawing they gave away a brand new Dell Precision 5530 2-in-1 convertible laptop workstation.)   Microsoft also garnered quite a bit of attention when they gave away 30 MS Surface tablets near the end of the show.  There was lots to see and learn everywhere I looked.

The Significance of MAX
Adobe MAX is quite a significant event, especially now that I have been in the industry long enough to start to see the evolution of certain trends — things are not as static as we may expect.  I have attended NAB for the last 12 years, and the focus of that show has shifted significantly away from my primary professional focus. (No Red, Ncidia, or Apple booths, among many other changes)  This was the first year that I had the thought “I should have gone to Sundance,” and a number of other people I know had the same impression. Adobe Max is similar, although I have been a little slower to catch on to that change.  It has been happening for over ten years, but has grown dramatically in size and significance recently.  If I still lived in LA, I probably would have started attending sooner, but it was hardly on my radar until three weeks ago.  Now that I have seen it in person, I probably won’t miss it in the future.


Mike McCarthy is an online editor/workflow consultant with 10 years of experience on feature films and commercials. He has been involved in pioneering new solutions for tapeless workflows, DSLR filmmaking and multi-screen and surround video experiences. Check out his site.

The benefits of LTO

By Mike McCarthy

LTO stands for Linear Tape Open, and was initially developed nearly 20 years ago as an “open” format technology that allows manufacturing by any vendor that wishes to license the technology. It records any digital files onto half-inch magnetic tapes, stored in square single reel cartridges. The capacity started at 100GB and has increased by a factor of two nearly every generation; the most recent LTO-8 cartridges store 12TB of uncompressed data.

If you want to find out more about LTO, you should check out the LTO Consortium, which is made up of Hewlett Packard Enterprises, IBM and Quantum, although there are other companies that make LTO drives and tape cartridges. You might be familiar with their LTO Ultrium logo.

‘Tapeless’ Workflows
While initially targeting server markets, with the introduction of “tapeless workflows” in the media and entertainment industry, there became a need for long-term media storage. Since the first P2 cards and SxS sticks were too expensive for single write operations, they were designed to be reused repeatedly once their contents had been offloaded to hard drives. But hard drives are not ideal for long-term data storage, and insurance and bonding companies wanted their clients to have alternate data archiving solutions.

So, by the time the Red One and Canon 5D were flooding post facilities with CF cards, LTO had become the default archive solution for most high-budget productions. But this approach was not without limitations and pitfalls. The LTO archiving solutions being marketed at the time were designed around the Linux-based Tar system of storing files, while most media work is done on Windows and Mac OS X. Various approaches were taken by different storage vendors to provide LTO capabilities to M&E customers. Some were network appliances running Linux under the hood, while others wrote drivers and software to access the media from OS X or, in one case, Windows. Then there was the issue that Tar isn’t a self-describing file system, so you needed a separate application to keep track of what was on each tape in your library. All of these aspects cost lots of money, so the initial investment was steep, even though the margin cost of tape cartridges was the cheapest way to store data per GB.

LTFS
Linear Tape File System (LTFS) was first introduced with LTO-5 and was intended to make LTO tapes easier to use and interchange between systems. A separate partition on the tape stores the index of data in XML and other associated metadata. It was intended to be platform independent, although it took a while for reliable drivers and software to be developed for use in Windows and OS X.

At this point, LTFS-formatted tapes in LTO tape drives operate very similarly to old 3.5-inch floppy drives. You insert a cartridge, it makes some funny noises, and then after a minute it asks you to format a new tape, or it displays the current contents of the tape as a new drive letter. If you drag files into that drive, it will start copying the data to the tape, and you can hear it grinding away. The biggest difference is when you hit eject it will take the computer a minute or two to rewind the tape, write the updated index to the first partition and then eject the cartridge for you. Otherwise it is a seamless drag and drop, just like any other removable data storage device.

LTO Drives
All you need in order to use LTO in your media workflow — for archive or data transfer — is an LTO drive. I bought one last year on Amazon for $1,600, which was a bit of a risk considering that I didn’t know if I was going to be able to get it to work on my Windows 7 desktop. As far as I know, all tape drives are SAS devices, although you can buy ones that have adapted the SAS interface to Thunderbolt or Fibre Channel.

Most professional workstations have integrated SAS controllers, so internal LTO drives fit into a 5.25-inch bay and can just connect to those, or any SAS card. External LTO drives usually use Small Form Factor cables (SFF-8088) to connect to the host device. Internal SAS ports can be easily adapted to SFF-8088 ports, or a dedicated eSAS PCIe card can be installed in the system.

Capacity & Compression
How much data do LTO tapes hold? This depends on the generation… and the compression options. The higher capacity advertised on any LTO product assumes a significant level of data compression, which may be achievable with uncompressed media files (DPX, TIFF, ARRI, etc.) The lower value advertised is the uncompressed data capacity, which is the more accurate estimate of how much data it will store. This level of compression is achieved using two different approaches, eliminating redundant data segments and eliminating the space between files. LTO was originally designed for backing up lots of tiny files on data servers, like credit card transactions or text data, and those compression approaches don’t always apply well to large continuous blocks of unique data found in encoded video.

Using data compression on media files which are already stored in a compressed codec doesn’t save much space (there is little redundancy in the data, and few gaps between individual files).

Uncompressed frame sequences, on the other hand, can definitely benefit from LTO’s hardware data compression. Regardless of compression, I wouldn’t count on using the full capacity of each cartridge. Due to the way the drives are formatted, and the way storage vendors measure data, I have only been able to copy 2.2TB of data from Windows onto my 2.5TB LTO-6 cartridges. So keep that in mind when estimating real-world capacity, like with any other data storage medium.

Choosing the ‘Right’ Version to Use
So which generation of LTO is the best option? That depends on how much data you are trying to store. Since most media files that need to be archived these days are compressed, either as camera source footage or final deliverables, I will be calculating based on the uncompressed capacities. VFX houses using DPX frames, or vendors using DCDMs might benefit from calculating based on the compressed capacities.

Prices are always changing, especially for the drives, but these are the numbers as of summer 2018. On the lowest end, we have LTO-5 drives available online for $600-$800, which will probably store 1-1.2TB of data on a $15 tape. So if you have less than 10TB of data to backup at a time, that might be a cost-effective option. Any version lower than LTO-5 doesn’t support the partitioning required for LTFS, and is too small to be useful in modern workflows anyway.

As I mentioned earlier, I spent $1,600 on an LTO-6 drive last year, and while that price is still about the same, LTO-7 and LTO-8 drives have come down in cost since then. My LTO-6 drive stores about 2.2TB of data per $23 tape. That allowed me to backup 40TB of Red footage onto 20 tapes in 90 hours, or an entire week. Now I am looking at using the same drive to ingest 250TB of footage from a production in China, but that would take well over a month, so LTO-6 is not the right solution for that project. But the finished deliverables will probably be a similar 10TB set of DPX and TIFF files, so LTO-6 will still be relevant for that application.

I see prices as low as $2,200 for LTO-7 drives, so they aren’t much more expensive than LTO-6 drives at this point, but the 6TB tapes are. LTO-7 switched to a different tape material, which increased the price of the media. At $63 they are just over $10 per TB, but that is higher than the two previous generations.

LTO-8 drives are available for as low as $2,600, and store up to 12TB on a single $160 tape. LTO-8 drives can also write up to 9TB onto a properly formatted LTO-7 tape in a system called “LTO-7 Type M” This is probably the cheapest cost per TB approach at the moment, since 9TB on a $63 tape is $7/TB.

Compatibility Between Generations
One other consideration is backwards compatibility. What will it take to read your tapes back in the future? The standard for LTO has been that drives can write the previous generation tapes and read tapes from two generations back.

So if you invested in an LTO-2 drive and have tons of tapes, they will still work when you upgrade to an LTO-4 drive. You can then copy them to newer cartridges with the same hardware at a 4:1 ratio since the capacity will have doubled twice. The designers probably figured that after two generations (about five years) most data will have been restored at some point, or be irrelevant (the difference between backups and archives).

If you need your media archived longer than that, it would probably be wise to transfer it to fresh media of a newer generation to ensure it is readable in the future. The other issue is transfer if you are using LTO cartridges to move data from one place to another. You must use the same generation of tape and be within one generation to go both ways. If I want to send data to someone who has an LTO-5 drive, I have to use an LTO-5 tape, but I can copy the data to the tape with my LTO-6 drive (and be subject to the LTO-5 capacity and performance limits). If they then sent that LTO-5 tape to someone with an LTO-7 drive, they would be able to read the data, but wouldn’t be able to write to the tape. The only exception to this is that the LTO-8 drives won’t read LTO-6 tapes (of course, because I have a bunch of LTO-6 tapes now, right?).

So for my next 250TB project, I have to choose between a new LTO-7 drive with backwards compatibility to my existing gear or an LTO-8 drive that can fit 50% more data on a $63 cartridge, and use the more expensive 12TB ones as well. Owning both LTO-6 and LTO-8 drives would allow me to read or write to any LTFS cartridge (until LTO-9 is released), but the two drives couldn’t exchange tapes with each other.

Automated Backup Software & Media Management
I have just been using HPE’s free StoreOpen Utility to operate my internal LTO drive and track what files I copy to which tapes. There are obviously much more expensive LTO-based products, both in hardware with robotic tape libraries and in software with media and asset management programs and automated file backup solutions.

I am really just exploring the minimum investment that needs to be made to take advantage of the benefits of LTO tape, for manually archiving your media files and backing up your projects. The possibilities are endless, but the threshold to start using LTO is much lower than it used to be, especially with the release of LTFS support.


Mike McCarthy is an online editor/workflow consultant with 10 years of experience on feature films and commercials. He has been involved in pioneering new solutions for tapeless workflows, DSLR filmmaking and multi-screen and surround video experiences. Check out his site.