The cloud and production storage

By Tom Coughlin

The network of connected data centers known as “the cloud” is playing a greater role in many media and entertainment applications. This includes collaborative workflows and proxy viewing, rendering, content distribution and archiving. Cloud services can include processing power as well as various types of digital storage.

In the figure below, you will see Coughlin Associates’ projections (2015 Digital Storage in Media and Entertainment Report), for the growth of professional media and entertainment cloud storage out to 2020. Note that archiving is the biggest projected market for media and entertainment cloud storage.

Media and Entertainment Cloud Storage Capacity Projections

At the 2016 NAB show, there were many companies offering cloud storage and online services for the media and entertainment industry. Some of these were cloud-only offerings and some are hybrid cloud services with some on-premises storage.

In this piece, we will review some of these cloud storage offerings and take a look at how to move content around the cloud, as well as to and from the cloud and on-premise storage. There were also some interesting object storage infrastructure implementations that were of interest at the NAB show.

Archiving in the Cloud
Archive is the biggest application for cloud storage in media and entertainment and several companies have products geared toward these applications, some of them with magnetic tape storage in the cloud. Oracle’s DIVA content storage management software allows the integration of on-premises storage and the Oracle cloud. The recently announced DIVAnet 2.0 allows a converged infrastructure for rich media using single namespace access to DIVArchive on-premises sites and Oracle DIVA Cloud storage as a service.
The Oracle Archive Cloud using DIVA content management offers archive storage for about $0.01/GB/month. This equates to storing 1 petabyte (PB) of content for just $12,000 per year. That is less than the on-premises costs for many content archives. At this price, rich media content companies are considering trusting their long-term content archives to the cloud.

Fujifilm’s Dternity tape-based archive, which offers online access to your data and integrates with applications already in your workflow, had an exhibit at NAB again this year. IBM also offers tape storage in the cloud. In addition to archiving on tape there are HDD cloud storage offerings as well. Major cloud companies such as Google, Azure and AWS offer tape and HDD- based cloud storage.

Quantum showcased its new Q-Cloud Vault long-term cloud storage service. This is fully integrated within workflows powered by StorNext 5.3, Q-Cloud Vault will provide low-cost, Quantum-managed “cold storage” in the public cloud. Because StorNext 5.3 enables end-to-end encryption, users can leverage the cloud as a part of their storage infrastructure to facilitate secure, cost-effective storage of their media content, both on-site and off-site.

In addition to supporting Q-Cloud Vault (pictured above), StorNext 5.3 gives users greater control and flexibility in optimizing their collaborative media workflows for maximum efficiency and productivity.

Cloud-Assisted Media Workflows
In addition to archive-focused cloud storage, some companies at NAB were talking about cloud and hybrid storage focused on non-archive applications.

Fast-paced growth and strong demand for scale-out storage clouds have propelled DDN’s WOS to one of the industry’s top solutions based on the number of objects in production and have fortified DDN’s position as a strong market leader in object storage. Continuing to fuel the pace of its object storage momentum, DDN also announced the availability of its latest WOS platform release. WOS is also an important component in the company’s MediaScaler Converged Media Workflow Storage Platform.

WOS possesses a combination of high-performance, flexible protection, multi-site capabilities and storage efficiencies that make it the perfect solution for a wide range of use cases, including active archive repositories, OpenStack Swift, data management, disaster recovery, content distribution, distributed collaboration workflows, enterprise content repositories, file sync and share, geospatial images, video surveillance, scale-out web and cloud services, and video post-production.

EMC, Pixspan, Aspera and Nvidia are bringing uncompressed 4K workflows to IT infrastructures, advancing digital media workflows with full resolution content over standard 10GbE networks. Customers can now achieve savings and performance increases of 50-80 percent in storage and bandwidth throughout the entire workflow — from on-set through post to final assets. Artists and facilities using creative applications for compositing, visual effects, DI and more can now work faster with camera raw, DPX, EXR, TIFF and Cineon files. Content can be safely stored on EMC’s Isilon scale-out NAS for shared collaborative access to project data in the data center, around the world, or to the cloud.

NetApp Webscale

NetApp StorageGrid Webscale

At NAB, NetApp promoted new features in its StorageGrid Webscale (appliance or software-defined) object storage. The object store has been widely adopted by media sites and media cloud providers who are managing tens of billions of media objects. Now, a majority of the key MAM, file-delivery and archive systems have integrated to StorageGrid Webscale’s Amazon S3 object interface. 

StorageGrid Webscale is a next-generation solution for multi-petabyte distributed content repositories. It provides erasure coding or, alternatively, automatic file copies to remote locations depending on the value of the media and the needs of the workflow.

Scality Ring storage scales linearly across multiple active sites and thousands of servers and can host an unlimited number of objects, providing high performance across a variety of workloads with file or object storage access. The company says the product enables organizations to build Exabyte-scale active archives and scalable content distribution systems, including network DVR/PVR. The product can be used to make a private storage cloud with file and object access and to provide customized web services.

Avere FlashCloud is a hybrid cloud and on-premise storage offering advertised as providing unlimited capacity scaling in the Cloud with unlimited performance scaling to the edge with up to 480 TB of data on FXT Series Edge filers. The dynamic tiering of active data to the edge hides the latency of cloud storage while NFS and SMB access provide file-based storage with a global namespace including public objects, private objects and NAS.

Avere’s FlashMove software transparently moves live online data to the cloud and between cloud providers. FlashMiror replicates data to the cloud for disaster recovery. AES-256 encryption with FIPS 140-2 compliance provides data security with on premise encryption key management. It should be noted that Avere worked with Google to provide the storage cluster used to stream the video showed at NAB during the Lytro Cinema demonstration.

Moving and delivering in the cloud

Moving and delivering in the cloud

SAN Solutions SAN Metro Media ultra-low-latency cloud for media extends a customer’s studio to the cloud with its SMM Ultra-Connect dedicated, secure direct connect, low-latency circuit from the customer’s site to one of SAN Metro Media’s data centers in a metropolitan area. The SMM Ultra-Connect circuit can operate completely off the Internet and transport media at the bandwidth and latencies that large studio applications and workflows require.

Moving and Delivering Content in the Cloud
There are several companies offering data transport services to and from cloud services as well as from on-premise storage to the cloud and back.

At NAB show Aspera (a division of IBM) introduced FASPStream, a turnkey application-software line designed to enable live streaming of broadcast-quality video globally over commodity Internet networks with glitch-free playout and negligible startup time, reducing the need for expensive and limited satellite-based backhaul, transport and distribution.

The FASPStream software uses the FASP bulk data protocol to transport live multicast, unicast UDP, PCP or other file source video, providing timely arrival of live video and data independent of network round-trip delay and packet loss. The company says that less than five seconds of startup delay is required for 50Mbps video streams transported with 250ms round-trip latency and three percent packet loss. These properties are sufficient for 4K streaming between continents.

Aspera is part of a broader group of IBM acquisitions with a strong focus on the media and entertainment industry, including object storage provider Cleversafe.

Signiant announced the integration of its Manager+Agents product with the Avid Interplay | MAM system. Customers can now initiate accelerated file transfers from within Interplay | MAM, making it easier than ever to use the power of Signiant technology in support of global creative processes. Users can now initiate and monitor Signiant file transfers via the Export capability within Avid Interplay|MAM.

FileCatalyst had some media and entertainment case studies, including involvement with NBC’s 2016 Rio Olympics preparation.

Snowball is a PB-scale data transport solution that uses secure appliances to transfer large amounts of data into and out of the AWS cloud. With Snowball, you don’t need to write any code or purchase any hardware to transfer your data. Simply create a job in the AWS Management Console and a Snowball appliance will be automatically shipped to you. Once it arrives, attach the appliance to your local network, download and run the Snowball client to establish a connection, then use the client to select the file directories that you want to transfer to the appliance.

The client will then encrypt and transfer the files to the appliance at high speed. Once the transfer is complete and the appliance is ready to be returned, the E Ink shipping label will automatically update and you can track the job status via Amazon Simple Notification Service (SNS), text messages or directly in the Console.

EMC and Imagine Communications provide live channel playout with the Versio solution in an offering with EMC’s converged VCE Vblock system and EMC’s Isilon scale-out NAS storage system. EMC’s technology and Versio, Imagine’s cloud-capable channel playout solution, help to enable broadcasters to securely fulfill channel playout across geographically dispersed network to help engage customers with content tailored to their respective operations. EMC also talked about Cloud DVR solutions with Anevia.

Object Storage Infrastructure
A start-up company named Fixstars Solutions provides an innovative storage server (called Olive) with dual core-CPU, FPGA, 512MB RAM, gigabit Ethernet and up to 13TB of non-volatile flash memory storage in a 2.5-inch form factor. The company announced Ceph running on Olive, building high-performance, scalable storage systems at low cost that it feels can provide solutions for broadcasters, studios, cable providers and Internet delivery networks.

Dr. Tom Coughlin, president of Coughlin Associates, is a storage analyst and consultant with over 30 years in the data storage industry. He is the founder and organizer of the Annual Storage Visions Conference as well as the Creative Storage Conference. The 2016 Creative Storage Conference is June 23 in Culver City. It features conferences and exhibits focused on the growing storage demands of HD, UltraHD, 4K and HDR film production and how it is affecting every stage of the production.


Leave a Reply

Your email address will not be published. Required fields are marked *