Tag Archives: cloud storage

The cloud and production storage

By Tom Coughlin

The network of connected data centers known as “the cloud” is playing a greater role in many media and entertainment applications. This includes collaborative workflows and proxy viewing, rendering, content distribution and archiving. Cloud services can include processing power as well as various types of digital storage.

In the figure below, you will see Coughlin Associates’ projections (2015 Digital Storage in Media and Entertainment Report), for the growth of professional media and entertainment cloud storage out to 2020. Note that archiving is the biggest projected market for media and entertainment cloud storage.

Media and Entertainment Cloud Storage Capacity Projections

At the 2016 NAB show, there were many companies offering cloud storage and online services for the media and entertainment industry. Some of these were cloud-only offerings and some are hybrid cloud services with some on-premises storage.

In this piece, we will review some of these cloud storage offerings and take a look at how to move content around the cloud, as well as to and from the cloud and on-premise storage. There were also some interesting object storage infrastructure implementations that were of interest at the NAB show.

Archiving in the Cloud
Archive is the biggest application for cloud storage in media and entertainment and several companies have products geared toward these applications, some of them with magnetic tape storage in the cloud. Oracle’s DIVA content storage management software allows the integration of on-premises storage and the Oracle cloud. The recently announced DIVAnet 2.0 allows a converged infrastructure for rich media using single namespace access to DIVArchive on-premises sites and Oracle DIVA Cloud storage as a service.
The Oracle Archive Cloud using DIVA content management offers archive storage for about $0.01/GB/month. This equates to storing 1 petabyte (PB) of content for just $12,000 per year. That is less than the on-premises costs for many content archives. At this price, rich media content companies are considering trusting their long-term content archives to the cloud.

Fujifilm’s Dternity tape-based archive, which offers online access to your data and integrates with applications already in your workflow, had an exhibit at NAB again this year. IBM also offers tape storage in the cloud. In addition to archiving on tape there are HDD cloud storage offerings as well. Major cloud companies such as Google, Azure and AWS offer tape and HDD- based cloud storage.

Quantum showcased its new Q-Cloud Vault long-term cloud storage service. This is fully integrated within workflows powered by StorNext 5.3, Q-Cloud Vault will provide low-cost, Quantum-managed “cold storage” in the public cloud. Because StorNext 5.3 enables end-to-end encryption, users can leverage the cloud as a part of their storage infrastructure to facilitate secure, cost-effective storage of their media content, both on-site and off-site.

In addition to supporting Q-Cloud Vault (pictured above), StorNext 5.3 gives users greater control and flexibility in optimizing their collaborative media workflows for maximum efficiency and productivity.

Cloud-Assisted Media Workflows
In addition to archive-focused cloud storage, some companies at NAB were talking about cloud and hybrid storage focused on non-archive applications.

Fast-paced growth and strong demand for scale-out storage clouds have propelled DDN’s WOS to one of the industry’s top solutions based on the number of objects in production and have fortified DDN’s position as a strong market leader in object storage. Continuing to fuel the pace of its object storage momentum, DDN also announced the availability of its latest WOS platform release. WOS is also an important component in the company’s MediaScaler Converged Media Workflow Storage Platform.

WOS possesses a combination of high-performance, flexible protection, multi-site capabilities and storage efficiencies that make it the perfect solution for a wide range of use cases, including active archive repositories, OpenStack Swift, data management, disaster recovery, content distribution, distributed collaboration workflows, enterprise content repositories, file sync and share, geospatial images, video surveillance, scale-out web and cloud services, and video post-production.

EMC, Pixspan, Aspera and Nvidia are bringing uncompressed 4K workflows to IT infrastructures, advancing digital media workflows with full resolution content over standard 10GbE networks. Customers can now achieve savings and performance increases of 50-80 percent in storage and bandwidth throughout the entire workflow — from on-set through post to final assets. Artists and facilities using creative applications for compositing, visual effects, DI and more can now work faster with camera raw, DPX, EXR, TIFF and Cineon files. Content can be safely stored on EMC’s Isilon scale-out NAS for shared collaborative access to project data in the data center, around the world, or to the cloud.

NetApp Webscale

NetApp StorageGrid Webscale

At NAB, NetApp promoted new features in its StorageGrid Webscale (appliance or software-defined) object storage. The object store has been widely adopted by media sites and media cloud providers who are managing tens of billions of media objects. Now, a majority of the key MAM, file-delivery and archive systems have integrated to StorageGrid Webscale’s Amazon S3 object interface. 

StorageGrid Webscale is a next-generation solution for multi-petabyte distributed content repositories. It provides erasure coding or, alternatively, automatic file copies to remote locations depending on the value of the media and the needs of the workflow.

Scality Ring storage scales linearly across multiple active sites and thousands of servers and can host an unlimited number of objects, providing high performance across a variety of workloads with file or object storage access. The company says the product enables organizations to build Exabyte-scale active archives and scalable content distribution systems, including network DVR/PVR. The product can be used to make a private storage cloud with file and object access and to provide customized web services.

Avere FlashCloud is a hybrid cloud and on-premise storage offering advertised as providing unlimited capacity scaling in the Cloud with unlimited performance scaling to the edge with up to 480 TB of data on FXT Series Edge filers. The dynamic tiering of active data to the edge hides the latency of cloud storage while NFS and SMB access provide file-based storage with a global namespace including public objects, private objects and NAS.

Avere’s FlashMove software transparently moves live online data to the cloud and between cloud providers. FlashMiror replicates data to the cloud for disaster recovery. AES-256 encryption with FIPS 140-2 compliance provides data security with on premise encryption key management. It should be noted that Avere worked with Google to provide the storage cluster used to stream the video showed at NAB during the Lytro Cinema demonstration.

Moving and delivering in the cloud

Moving and delivering in the cloud

SAN Solutions SAN Metro Media ultra-low-latency cloud for media extends a customer’s studio to the cloud with its SMM Ultra-Connect dedicated, secure direct connect, low-latency circuit from the customer’s site to one of SAN Metro Media’s data centers in a metropolitan area. The SMM Ultra-Connect circuit can operate completely off the Internet and transport media at the bandwidth and latencies that large studio applications and workflows require.

Moving and Delivering Content in the Cloud
There are several companies offering data transport services to and from cloud services as well as from on-premise storage to the cloud and back.

At NAB show Aspera (a division of IBM) introduced FASPStream, a turnkey application-software line designed to enable live streaming of broadcast-quality video globally over commodity Internet networks with glitch-free playout and negligible startup time, reducing the need for expensive and limited satellite-based backhaul, transport and distribution.

The FASPStream software uses the FASP bulk data protocol to transport live multicast, unicast UDP, PCP or other file source video, providing timely arrival of live video and data independent of network round-trip delay and packet loss. The company says that less than five seconds of startup delay is required for 50Mbps video streams transported with 250ms round-trip latency and three percent packet loss. These properties are sufficient for 4K streaming between continents.

Aspera is part of a broader group of IBM acquisitions with a strong focus on the media and entertainment industry, including object storage provider Cleversafe.

Signiant announced the integration of its Manager+Agents product with the Avid Interplay | MAM system. Customers can now initiate accelerated file transfers from within Interplay | MAM, making it easier than ever to use the power of Signiant technology in support of global creative processes. Users can now initiate and monitor Signiant file transfers via the Export capability within Avid Interplay|MAM.

FileCatalyst had some media and entertainment case studies, including involvement with NBC’s 2016 Rio Olympics preparation.

Snowball is a PB-scale data transport solution that uses secure appliances to transfer large amounts of data into and out of the AWS cloud. With Snowball, you don’t need to write any code or purchase any hardware to transfer your data. Simply create a job in the AWS Management Console and a Snowball appliance will be automatically shipped to you. Once it arrives, attach the appliance to your local network, download and run the Snowball client to establish a connection, then use the client to select the file directories that you want to transfer to the appliance.

The client will then encrypt and transfer the files to the appliance at high speed. Once the transfer is complete and the appliance is ready to be returned, the E Ink shipping label will automatically update and you can track the job status via Amazon Simple Notification Service (SNS), text messages or directly in the Console.

EMC and Imagine Communications provide live channel playout with the Versio solution in an offering with EMC’s converged VCE Vblock system and EMC’s Isilon scale-out NAS storage system. EMC’s technology and Versio, Imagine’s cloud-capable channel playout solution, help to enable broadcasters to securely fulfill channel playout across geographically dispersed network to help engage customers with content tailored to their respective operations. EMC also talked about Cloud DVR solutions with Anevia.

Object Storage Infrastructure
A start-up company named Fixstars Solutions provides an innovative storage server (called Olive) with dual core-CPU, FPGA, 512MB RAM, gigabit Ethernet and up to 13TB of non-volatile flash memory storage in a 2.5-inch form factor. The company announced Ceph running on Olive, building high-performance, scalable storage systems at low cost that it feels can provide solutions for broadcasters, studios, cable providers and Internet delivery networks.

Dr. Tom Coughlin, president of Coughlin Associates, is a storage analyst and consultant with over 30 years in the data storage industry. He is the founder and organizer of the Annual Storage Visions Conference as well as the Creative Storage Conference. The 2016 Creative Storage Conference is June 23 in Culver City. It features conferences and exhibits focused on the growing storage demands of HD, UltraHD, 4K and HDR film production and how it is affecting every stage of the production.

Public, Private, Hybrid Cloud: the basics and benefits

By Alex Grossman

The cloud is everywhere, and media facilities are constantly being inundated with messages about the benefits the cloud offers in every area, from production to delivery. While it is easy to locate information on how the cloud can be used for ingest, storage, post operations, transcoding, rendering, archive and, of course, delivery, many media facilities still have questions about the public, private and hybrid clouds, and how each of these cloud models can relate to their business. The following is a brief guide intended to answer these questions.

Public
Public cloud is the cloud as most people see it: a set of services hosted outside a facility and accessed through the Web, either securely through a gateway appliance or simply through a browser. The public nature of this cloud model does not mean that content from one person or one company can be accessed by another. It simply means that the same physical hardware is being shared by multiple users — a “multi-tenant” arrangement in which data from different users resides on one system. Through this approach, users get to take advantage of the scale of many servers and storage systems at the cloud facility. This scale can also improve accessibility and performance, which can be key considerations for many content creators.

Public cloud is the most versatile type of cloud, and it can offer a range of services, from hosted applications to “compute” capabilities. For media companies these services range from transcoding, rendering and animation to business services such as project tracking, billing and, in some cases, file sharing. (Box and Dropbox are good examples of file sharing enabled by public cloud.) Services may be generic or branded, and they are most often offered by a software vendor using a public cloud, or by the public cloud vendor itself. Public clouds are popular for content and asset storage, both for short-term transcode to delivery or project-in-process storage and for longer-term “third copy” off-site archive.

Public clouds can be very appealing due to the OPEX or pay-as-you-go nature of billing and the lack of any capital expense with ongoing hardware purchase and refresh, but the downside of this is that public clouds remove control over workflow. While most public cloud vendors today are large and financially stable, it remains important to choose carefully.

Moreover, taking advantage of public cloud is rarely easy. This path involves dealing with new vendors, and possibly with unfamiliar applications and hardware gateways, and there can be unexpected charges for simple operations such as retrieving data. Although content security concerns are mostly overblown, they nevertheless are a source of apprehension for many potential public cloud users. These uncertainties have a lot of media companies looking to private cloud.

Private
Private cloud can most simply be defined for media companies as a walled machine room environment with workflow compute and storage capabilities offering outside connectivity, while at the same time preventing outside intrusions into the facility.

A well-designed private cloud will allow facilities to extend most of their production and archive capabilities to remote users. The main difference between this approach and most current (non-cloud) storage and compute operations in a facility today is simply that a private cloud can isolate the current workflow from the outside world while extending a portion to remote users based on preferences and polices.

The idea of remote access is not confined to private cloud. It is possible to provide facility access to external users through normal networking protocols, but the private cloud takes this a step further through easier access for authorized users and greater security for others. The media facility remains in complete control of its content and assets. Furthermore, it can host its applications, and its content remains in its on-site storage, safe and secure, with no retrieval fees.

A facility that embraces private cloud cannot take advantage of the scale or pay-as-you-go benefits of public cloud. Thus, in order to provide greater accessibility and flexibility, some media companies have adopted a private cloud model as an extension of their online operations. Private cloud can effectively replace much of the current hardware used today in post and archive operations, so it is a more cost-effective solution for many considering cloud benefits.

Hybrid
Hybrid cloud is an interesting proposition. In the enterprise IT world, hybrid cloud implementations are seen as way to bridge private and public and realize the best of both worlds — lower OPEX for certain functions such as SAAS (software as a service) and the security of keeping valuable data back in their data centers.

For media professionals, hybrid cloud may have even greater benefits. Considering the changing delivery requirements facing the industry and the sheer volume of content being created and reviewed — and, of course, keeping in mind the value of re-monetization — hybrid cloud has exciting potential. Well-designed hybrid cloud can provide the benefits of public and private cloud while taking advantage of the cost savings and reduced complexity that come with maintaining on-premise end-to end-hardware. By sharing the load between the hardware at a facility and the massive scale of a public cloud, a media company can extend its workflow easily while controlling every stage — even on a project-by-project basis.

Choosing between public, private and hybrid cloud can be a daunting task. It is a decision that must start with understanding the unique needs and goals of the media company and its operations, and then involve careful mapping of the various vendors’ offering solutions — with cost considerations always in mind. In the end, a facility may choose neither public cloud, private cloud nor hybrid cloud, but then it may miss out on the many and growing benefits enabled by the cloud.

Alex Grossman, a cloud workflow expert, is VP, Media and Entertainment at Quantum. You can follow him on Twitter @activeguy.

Quantum intros Q-Cloud storage solutions

Quantum has been an early adopter when it comes to creating cloud-based storage. Recently they took that to a new level, offering three new product solutions that integrate the cloud into multi-tier, hybrid storage architectures for data workloads.

The Quantum Q-Cloud solutions allow users to leverage Quantum’s intelligent data management software to store data in the cloud when it makes the most sense for a given workflow or application – essentially customizing a user’s needs.

The new Q-Cloud Archive and Q-Cloud Vault incorporate the power of the public cloud as an off-site tier within a Quantum StorNext 5 workflow environment, while Q-Cloud Protect for AWS enables customers using Quantum’s DXi deduplication appliances to replicate data to the Amazon Web Services (AWS) cloud. With all three offerings, users can get the benefits of the cloud without making changes to existing applications or processes.

Taking into account both the benefits and challenges of the cloud, Quantum says their approach is to enable customers to combine public cloud storage with on-premises storage in a multi-tier, intelligently managed, application-centric architecture. So instead of treating the cloud as a passive repository, they integrate the cloud as an active tier in a hybrid storage infrastructure driven by application requirements.

Key features include:
• Full integration as a tier in StorNext 5-managed workflows, with automated, policy-based data movement
• No additional hardware, separate applications or programming needed
• Agility in addressing changing workload demands
• Reliability of global public cloud infrastructure
• Straightforward pricing and billing directly from Quantum
• Secure, encrypted data transmission and at rest

StorNext users can also employ Q-Cloud Archive and Q-Cloud Vault in conjunction with Quantum Lattus extended online storage, moving data from a Lattus object storage technology-based private cloud on their premises to Q-Cloud.

Offered as a subscription service through the Amazon Marketplace, Q-Cloud Protect for AWS enables users to replicate data from either a physical or virtual DXi appliance on premises to a virtual DXi instance in the AWS cloud.

Key features include:
• Easy access to the Amazon public cloud and its benefits, with no changes to applications or processes
• Cost savings enabled by the most efficient method of data deduplication

Q-Cloud Archive is available immediately in the Americas, EMEA and APAC, while Q-Cloud Vault will be available in the second half of this year. Q-Cloud Protect for AWS will be available next quarter. You can see an overview of the products here.