Tag Archives: LTO

ATTO XstreamCore for remote access to DAS and sharing of devices

ATTO, which provides storage, network connectivity and infrastructure solutions, is now offering XstreamCore storage controllers to add remote Fibre Channel or Ethernet connectivity to SATA optical disc and SAS LTO tape devices, including the newly available LTO-8 tape format. XstreamCore is designed to provide the benefits of remote access to direct attached storage (DAS) technologies as well as the connection of multiple devices to be shared by multiple clients.

XstreamCore is a rack-scale flash and capacity storage controller that bridges 12Gb and 6Gb SAS storage devices and SATA devices to share and remotely connect them to Fibre Channel or Ethernet networks. XstreamCore includes exclusive ATTO developed features including xCore data acceleration to rapidly move data, the eCore control engine to add services and management features to storage while not affecting performance and SpeedWrite, a tape and optical performance feature that significantly boosts write-performance by effectively managing commands between attached clients and tape and optical devices.

By directly being able to connect up to 16 SATA optical or SAS tape drives with the possibility of additional drives being connected through SAS expanders, XstreamCore enables a lower cost of ownership versus native Ethernet or Fibre Channel tape devices. Fewer switch ports are required when using XstreamCore and power, cooling, cabling and weight requirements can be better managed as the ATTO controller allows separation of racks of client servers, storage and archive devices. ATTO XstreamCore FC 7500 currently supports connectivity for SAS tape devices and SATA optical devices to Fibre Channel connected servers or fabrics. ATTO Ethernet to SAS controllers will be available in early 2018 to support connectivity for these devices to Ethernet connected servers or fabrics.

 

Grading & Compositing Storage: Northern Lights

Speed is key for artist Chris Hengeveld.

By Beth Marchant

For Flame artist Chris Hengeveld of Northern Lights in New York City, high-performance file-level storage and a Fibre Channel connection mean it’s never been easier for him to download original source footage and share reference files with editorial on another floor. But Hengeveld still does 80 percent of his work the old-fashioned way: off hand-delivered drives that come in with raw footage from production.

Chris Hengeveld

The bicoastal editorial and finishing facility Northern Lights — parent company to motion graphics house Mr. Wonderful, the audio facility SuperExploder and production boutique Bodega — has an enviably symbiotic relationship with its various divisions. “We’re a small company but can go where we need to go,” says colorist/compositor Hengeveld. “We also help each other out. I do a lot of compositing, and Mr. Wonderful might be able to help me out or an assistant editor here might help me with After Effects work. There’s a lot of spillover between the companies, and I think that’s why we stay busy.”

Hengeveld, who has been with Northern Lights for nine years, uses Flame Premium, Autodesk’s visual effects finishing bundle of Flame and Flare with grading software Lustre. “It lets me do everything from final color work, VFX and compositing to plain-old finishing to get it out of the box and onto the air,” he says. With Northern Lights’ TV-centric work now including a growing cache of Web content, Hengeveld must often grade and finish in parallel. “No matter how you send it out, chances are what you’ve done is going to make it to the Web in some way. We make sure that what we make look good on TV also looks good on the Web. It’s often just two different outputs. What looks good on broadcast you often have to goose a bit to get it to look good on the Web. Also, the audio specs are slightly different.”

Hengeveld provided compositing and color on this spot for Speedo.

Editorial workflows typically begin on the floor above Hengeveld in Avid, “and an increasing number, as time goes by, in Adobe Premiere,” he says. Editors are connected to media through a TerraBlock shared storage system from Facilis. “Each room works off a partition from the TerraBlock, though typically with files transcoded from the original footage,” he says. “There’s very little that gets translated from them to me, in terms of clip-based material. But we do have an Aurora RAID from Rorke (now Scale Logic) off which we run a HyperFS SAN — a very high-performance, file-level storage area network — that connects to all the rooms and lets us share material very easily.”

The Avids in editorial at Northern Lights are connected by Gigabit Ethernet, but Hengeveld’s room is connected by Fibre. “I get very fast downloading of whatever I need. That system includes Mr. Wonderful, too, so we can share what we need to, when we need to. But I don’t really share much of the Avid work except for reference files.” For that, he goes back to raw camera footage. “I’d say bout 80 percent of the time, I’m pulling that raw shoot material off of G-Technology drives. It’s still sneaker-net on getting those source drives, and I don’t think that’s ever going to change,” he says. “I sometimes get 6TB of footage in for certain jobs and you’re not going to copy that all to a centrally located storage, especially when you’ll end up using about a hundredth of that material.”

The source drives are typically dupes from the production company, which more often than not is sister company Bodega. “These drives are not made for permanent storage,” he says. “These are transitional drives. But if you’re storing stuff that you want to access in five to six years, it’s really got to go to LTO or some other system.” It’s another reason he’s so committed to Flame and Lustre, he says. Both archive every project locally with its complete media, which can be then be easily dropped onto an LTO for safe long-term storage.

Time or money constraints can shift this basic workflow for Hengeveld, who sometimes receives a piece of a project from an editor that has been stripped of its color correction. “In that case, instead of loading in the raw material, I would load in the 15- or 30-second clip that they’ve created and work off of that. The downside with that is if the clip was shot with an adjustable format camera like a Red or Arri RAW, I lose that control. But at least, if they shoot it in Log-C, I still have the ability to have material that has a lot of latitude to work with. It’s not desirable, but for better stuff I almost always go back to the original source material and do a conform. But you sometimes are forced to make concessions, depending on how much time or budget the client has.”

A recent spot for IZod, with color by Hengeveld.

Those same constraints, paired with advances in technology, also mean far fewer in-person client meetings. “So much of this stuff is being evaluated on their computer after I’ve done a grade or composite on it,” he says. “I guess they feel more trust with the companies they’re working with. And let’s be honest: when you get into these very detailed composites, it can be like watching paint dry. Yet, many times when I’m grading,  I love having a client here because I think the sum of two is always greater than one. I enjoy the interaction. I learn something and I get to know my client better, too. I find out more about their subjectivity and what they like. There’s a lot to be said for it.”

Hengeveld also knows that his clients can often be more efficient at their own offices, especially when handling multiple projects at once, influencing their preferences for virtual meetings. “That’s the reality. There’s good and bad about that trade off. But sometimes, nothing beats an in-person session.”

Our main image is from NBC’s Rokerthon.

Bling Digital: proving the value of LTO data archiving

Bling Digital designs and manages digital workflows for film and television productions. They also provide digital lab services, edit systems and post services. Working with their parent company SIM Group, Bling has the ability to provide complete production packages — from the camera through final delivery — so they understand the importance of archiving. More on that later.

Bling provides services for film and television productions everywhere, and the company now has offices in Toronto, Vancouver, Los Angeles, Atlanta and New York. Additionally, Bling has virtual teams around the world and last year worked on jobs in 21 different cities. Their reach is long.

Bling’s workflow supervisors and workflow producers have come to play a major role in decision-making regarding metadata and color management for their clients — with the aim of maximizing production efficiency. Bling typically has about 30 scripted, long-format jobs running at any given time, split between television shows and feature films, and the goal is to safeguard its post customers against surprise bills after they begin their work.

To help meet that goal in terms of data archiving, Bling uses PreRoll Post from Imagine Products for all master media in its LTO archives. PreRoll Post automatically indexes camera originals — complete with thumbs, proxies and metadata — while backing up full-resolution media to local disk and long-term LTO tapes, a Sony Optical Disc Archive, or a local disk or server. PreRoll Post uses an LTFS system, which mounts a tape as a volume (hard disk) within OS X or Windows. This makes it possible to write and read from the tape as if it were a disk, without proprietary software or formatting.

Bling made the transition to PreRoll Post after its clients began asking for LTFS tapes, which have since become the standard for all of the studios Bling works with.

Jesse Korosi

For one thing, PreRoll Post lets Bling scale up or down quickly, which is hugely important when dealing with as many as 30 jobs at any given time, each with its own unique requirements. For example, one production might travel to a new city every couple of days, which necessitates backups every day at every location. Another production might be managed from an office that is always connected to local storage, while yet another might be run out of one of Bling’s facilities off of a SAN or NAS.

“With the number of remote jobs we do and how quickly we have expanded, we needed software that was easy for new crews to learn,” explains Jesse Korosi, director of workflow services at Bling. “We have found that one of the biggest benefits of using PreRoll Post is how easily we can roll it out. When you’re talking about servicing 30 shows at a time, all in different cities or countries, we’re constantly hiring new, local crews, and knowing that they will be able to learn the software easily is a huge relief.”

Narcos
Bling provides dailies, LTO archival and edit systems on the Netflix television series Narcos. The show has an on-set data manager who backs up the master files from the Red Epic camera to three hard drives that are part of a backup system on the camera truck. One of those hard drives is a rotation drive that gets sent to the Bling lab at both break and wrap, depending on that day’s production locations.

Once Bling’s dailies tech receives the drive, he mounts it and backs up the files to two LTFS tapes simultaneously using PreRoll Post. One of these copies will eventually be sent to the studio and the other will be held in Los Angeles at the finishing facility. When writing to the LTO tapes is complete, the Bling technician notifies the camera crew that the files are secure. Then, using Imagine’s ShotPut Pro automated offloading application, the camera crew can wipe the camera cards and put them back into rotation.

“After all of the money the production company spends on crew, locations, props, wardrobe, cameras — all of the millions of dollars — eventually everything is left only on these tapes. So it is very, very important that we trust the software writing them without a shadow of a doubt,” says Korosi.

Quantum intros next-gen shared workflow storage product Xcellis

Quantum is offering a next-generation, high-performance, end-to-end shared workflow storage solution, called Xcellis. Consolidating media and metadata management, extending connectivity options for both Fibre Channel and Ethernet clients, and supporting hosted applications, the new system aids productivity in collaborative media environments by integrating the most important components of workflow storage into a single, easy-to-manage, fully scalable hardware solution.

Xcellis supports online work in process, ingest and delivery, and archive through Quantum’s portfolio of Lattus object storage, LTO tape and Q-Cloud services, all running on the powerful StorNext 5 media workflow platform.

Xcellis integrates realtime and “non-realtime” media production support elements in the same system. At the heart of the new solution, the Xcellis Workflow Director controls shared client access privileges and provides simultaneous client connectivity across all popular network types, including Fibre Channel and IP for SMB, CIFS, NFS and Quantum’s higher performing DLC (Distributed LAN Client) — while also supporting hosted applications. By bringing the functionality of multiple disparate components into a single system that occupies just four rack units of space, the Xcellis Workflow Director simplifies the overall storage architecture and streamlines operation and management.

Xcellis also recognizes that many operations do not require high-speed Fibre Channel and can instead take advantage of low-cost IP NAS connectivity. Xcellis improves productivity by eliminating the need for separate islands of storage and allowing all clients and workflow automation operations such as transcode, QA, and delivery to share storage with complete transparent access to all assets and content regardless of interface type or operating system.

Check out Quantum’s Alex Grossman giving us highlights at SMPTE 2015 below:

Powered by StorNext 5, Xcellis offers:
• Streaming performance needed for ingest, transcoding and delivery, important for today’s high-resolution and HDR-enabled workflows.
• Optimized high-speed Fibre Channel SAN connectivity, along with cost-effective LAN, SMB and NFS sharing options, with IP connectivity for easy deployment and integration into all broadcast and post environments.
• Intelligent, policy-driven file movement across LTO tape (LTO-6 and LTO-7), LTFS, object storage and cloud archive systems for an optimal balance of media access and storage TCO.
• High-speed collaboration across Mac OS X, Windows and Linux.

Users can start with the smallest system offered and scale smoothly to the largest system with no replacement of original hardware investment and no interruption. While nearly doubling the number of files that Quantum workflow storage can handle, Xcellis also enables users to:

• Scale capacity and bandwidth independently to realize the performance characteristics ideal for their operations, tuning performance to the roles of different individuals within the workflow.
• Protect and maximize investment by paying only for what they require, and eliminating disruptive and costly hardware changes as their system grows.
• Independently and economically scaled performance and capacity, now up to 10 billion unmanaged files or 1.4 billion managed files in up to 64 file system volumes.

Xcellis also features StorNext Connect, a full-featured management tool that supports users in configuring, deploying and scaling the entire storage system and clients and also helps them to identify and resolve issues quickly for optimal performance and maximum uptime. Quantum is continually updating StorNext Connect, and Xcellis users will benefit from a newly enhanced customer self-install feature, as well as NAS (IP client) configuration within the StorNext Connect UI.

IBC 2015: Adventures in archiving

By Tom Coughlin

Once you have your content and have completed that award-winning new project, Oscar-nominated film or brilliant and effective commercial, where does your data go? Today, video content can be reused and repurposed a number of times, producing a continuing revenue stream by providing viewing for many generations of people. That makes video archives valuable and also requires changes from in-active to more active archives. This article looks at some of the archiving products on display at the 2015 IBC.

The figure to the right shows our estimate of revenue spent on various media and entertainment storage markets in 2014 (from the Digital Storage in Media and Entertainment Report from Coughlin Associates). Note that although almost 96 percent of all M&E storage capacity is used for archiving about 45 percent of the spending is for archiving.

Quantum showcased its StorNext 5 shared storage architecture, which includes high-performance online storage, extended online storage and tape- and cloud-based archives. The company also highlighted the StorNext Connect, a management and monitoring console that provides an at-a-glance dashboard of the entire StorNext environment. At IBC, Quantum introduced their Q-Cloud Archive that extends StorNext workflow capabilities to the cloud, allowing end-to-end StorNext environments to leverage cloud storage fully with no additional hardware, separate applications or programming while maintaining full compatibility with existing software applications.

The Quantum Storage Manager migrates data from online storage to its object-based Lattus, allowing secure, long-term storage with greater durability than RAID and extremely high scalability. Content can be migrated from Lattus to tape archives or Q-Cloud archives automatically. In addition Quantum’s Artico intelligent NAS archive appliance was on display, offering low cost scale-out storage for active media archives that can scale to PBs of content across HDDs, extended online storage, tape and cloud storage.

Also during IBC, the LTO Program Technology Provider Companies — HP, IBM and Quantum —announced the LTO-7 tape format that will be available in late 2015. The native capacity of this drive is 6TB, while 2.5:1 compression provides 15TB of storage with up to 750MB/s data rates. This product will provide over twice the capacity of the LTO-6 drive generation. The LTO roadmap goes out to a generation 10 product with up to 120TB of compressed content and about 48TB native capacity.

LTO proponents said that tape has some advantages over hard disk drives for archiving, despite the difference in latency to access content. In particular, they said tape has and error rate two orders of magnitude lower than HDDs, providing more accurate recording and reading of content. Among the interesting LTO developments at IBC were the M-Logic Thunderbolt interface tape drives.

Tape can also be combined with capacity SATA HDDs to provide storage systems with performance approaching hard disk drive arrays and costs approaching magnetic tape libraries. Crossroads has teamed up with Fujifilm to provide NAS systems combining HDDs and tape and including cloud storage combining tape and HDDs. In fact archiving is becoming one of the biggest growing applications in the media and entertainment industry, according to the 2015 Digital Storage in Media and Entertainment Report from Coughlin Associates.

Oracle was also showing its tape storage systems with 8TB native storage capacity in a half-inch tape form factor. Oracle now includes Front Porch Digital with its cloud archiving platform as well as digital ingest solutions for older analog and digital format media.

Some companies also use flash memory as a content cache in order to match the high speeds of data transfers to and from a tape library system. Companies such as Xendata provide LTO tape and optical disc libraries for media and entertainment customers. Spectra Logic has made a big push into HDD-based archiving, using shingled magnetic recording (SMR) 3.5-inch HDDs in their DPE storage system to provide unstructured storage costs as low as 9 cents/GB. This system can provide up to 7.4PB of raw capacity in a single rack with 1GB/s data rates. This sort of system is best for data that is seldom or never overwritten because of the use of SMR HDDS.

Sony was showing its 300GB Blu-ray optical WORM discs, although it was not clear if the product is being shipped in storage cartridges in 2015. Archiving is a significant driver of M&E storage demand. This is because all video eventually ends up in an archive. Because of more frequent access of archived content, the performance requirements of many archives are more demanding than in the past. This has led to the use of HDD-based archives and archives combining HDDs and magnetic tape. Even flash memory can play a role as a write and read cache in a tape based system.

Dr. Tom Coughlin, president of Coughlin Associates, has over 35 years in the data storage industry. Coughlin is also the founder and organizer of the annual Storage Visions Conference, a partner to the International Consumer Electronics Show, as well as the Creative Storage Conference

IBC Blog: scanners, LTO and SDI

By Tim Spitzer

Although we all know “Film is Dead,” film scanners are popping up all over in an effort to resuscitate the medium.  Three of the new players to my circus of knowledge are:

MWA Nova GmbH, which make realtime scanners in 2.5K and 4K at 5fps, with the 4K, are soon to be realtime flavors. The scanner is capstan-driven (without sprockets), with room for PTR rollers, so looks to be easy on archival film. A laser senses the perf position to modulate the speed of the transport to deal with shrunken perforation pitch. Their gates are available for most known film formats from 35mm down to Super 8.

Blackmagic is offering a 35mm scanner with stabilization software and Resolve thrown in for free. The machine comes with a 35mm gate, and the rollers are milled to also handle S-16mm, Continue reading