Tag Archives: Fraunhofer

Digging Deeper: Fraunhofer’s Dr. Siegfried Foessel

By Randi Altman

If you’ve been to NAB, IBC, AES or regional conferences involving media and entertainment technology, you have likely seen Fraunhofer exhibiting or heard one of their representatives speaking on a panel.

Fraunhofer first showed up on my radar years ago at an AES show in New York City when they were touting the new MP3 format, which they created. From that moment on, I’ve made it a point to keep up on what Fraunhofer has been doing in other areas of the industry, but for some, what Fraunhofer is and does is a mystery.

We decided to help with that mystery by throwing some questions at Dr. Siegfried Foessel, Fraunhofer IIS Department Moving Picture Technologies.

Can you describe Fraunhofer?
Fraunhofer-Gesellschaft is an organization for applied research that has 67 institutes and research units at locations throughout Germany. At present, there are around 24,000 people. The majority are qualified scientists and engineers who work with an annual research budget of more than 2.1 billion euros.

More than 70 percent of the Fraunhofer-Gesellschaft’s research revenue is derived from contracts with industry and from publicly financed research projects. Almost 30 percent is contributed by the German federal and Länder governments in the form of base funding. This enables the institutes to work ahead on solutions to problems that will become relevant to industry and society within the next five or ten years from now.

How did it all begin? Is it a think tank of sorts? Tell us about Fraunhofer’s business model.
The Fraunhofer-Gesellschaft was founded in 1949 and is a recognized non-profit organization that takes its name from Joseph von Fraunhofer (1787–1826), the illustrious Munich researcher, inventor and entrepreneur. Its focus was clearly defined to do application-oriented research and to develop future-relevant key technologies. Through their research and development work, the Fraunhofer Institutes help to reinforce the competitive strength of the economy. They do so by promoting innovation, strengthening the technological base, improving the acceptance of new technologies and helping to train the urgently needed future generation of scientists and engineers.

What is Fraunhofer IIS?
The Fraunhofer Institute for Integrated Circuits IIS is an application-oriented research institution for microelectronic and IT system solutions and services. With the creation of MP3 and the co-development of AAC, Fraunhofer IIS has reached worldwide recognition. In close cooperation with partners and clients, the ISS institute provides research and development services in the following areas: audio and multimedia, imaging systems, energy management, IC design and design automation, communication systems, positioning, medical technology, sensor systems, safety and security technology, supply chain management and non-destructive testing. About 880 employees conduct contract research for industry, the service sector and public authorities.

Fraunhofer IIS partners with companies as well as public institutions?
We develop, implement and optimize processes, products and equipment until they are ready for use in the market. Flexible interlinking of expertise and capacities enables us to meet extremely broad project requirements and complex system solutions. We do contracted research for companies of all sizes. We license our technologies and developments. We work together with partners in publicly funded research projects or carry out commercial and technical feasibility studies.

IMF transcoding.

What is the focus of Fraunhofer IIS’ Department of Moving Picture Technologies?
For more than 15 years, our Department Moving Picture Technologies has driven developments for digital cinema and broadcast solutions focused on imaging systems, post production tools, formats and workflow solutions. The Department Moving Picture Technologies was chosen by the Digital Cinema Initiatives (DCI) to develop and implement the first certification test plan for digital cinema as the main reference for all systems in this area. As a leader in the ISO standardization committee for digital cinema within JPEG, my team and I are driving standardization for JPEG 2000 and formats, such as DCP and the Interoperable Master Format (IMF.)

We also are working together with SMPTE and other standardization bodies worldwide. Renowned developments for the department that are highly respected are the Arri D20/D21 camera, the easyDCP post production suite for DCP and IMF creation and playback, as well as the latest developments and results of multi-camera/light-field technology.

What are some of the things you are working on and how does that work find its way to post houses and post pros?
The engineers and scientists of the Department Moving Picture Technologies are working on tools and workflow solutions for new media file formats like IMF to enable smooth integration and use in existing workflows and to optimize performance and quality. As an example, we always enhance and augment the features available through the post production easyDCP suite. The team discusses and collaborates with customers, industry partners and professionals in the post production and digital cinema industries to identify the “most wanted and needed” requirements.

easyDCP

We preview new technologies and present developments that meet these requirements or facilitate process steps. Examples of this include the acceleration process of IMF or DCP creation by using an approach based on a hybrid JPEG 2000 functionality or introducing a media asset management tool for DCP/IMF or dailies. We present our ideas, developments and results at exhibitions such as NAB, the HPA Tech Retreat and IBC, as well as SMPTE conferences and plugfests all around the world.

Together with distribution partners who are selling the products like easyDCP, Fraunhofer IIS licenses those developments and puts them into the market. Therefore, the team always looks for customer feedback for their developments that is supported by a very active community.

Who are some of your current customers and partners?
We have more than 1,500 post houses as customers, managed by our licensing partner easyDCP GmbH. Nearly all of the Hollywood studios and post houses on all continents are our customers. We also work together with integration partners like Blackmagic and Quantel. Most of the names of our partners in the contract research area are confidential, but to name some partners from the past and present: Arri, DCI, IHSE GmbH.

Which technologies are available for license now?
• Tools for creation and playback of DCPs and IMPs, as standalone tools and for integration into third party tools
• Tools for quality control of DCPs and IMPs
• Tools for media asset management of DCPs and IMPs
• Plug-ins for light-field-processing and depth map generation
• Codecs for mezzanine compression of images

Lightfield tech

What are you working on now that people should know about?
We are developing new tools and plug-ins for bringing lightfield technology to the movie industry to enhance creativity opportunities. This includes system aspects in combination with existing post tools. We are chairing and actively participating on adhoc groups for lightfield-related standardization efforts in the JPEG/MPEG Joint Adhoc Group for digital representations of light/sound fields for immersive media applications (see https://jpeg.org/items/20160603_pleno_report.html).

We are also working together with DIN on a proposal to standardize digital long-term archive formats for movies. Basic work is done with German archives and service providers at DIN NVBF3 and together with CST from France at SMPTE with IMF App#4. Furthermore, we are developing mezzanine image compression formats for the transmission of video over IP in professional broadcast environments and GPU accelerated tools for creation and playback of JPEG 2000 code streams.

How do you pick what you will work on?
The employees at Fraunhofer IIS are very creative people. By observation of the market, research in joint projects and cooperation with universities, ideas are created and evaluated. Employees and our student scientists are discussing with industry partners what might be possible in the near future and which ideas have the greatest potential. Selected ideas will then be evaluated with respect to the business opportunities and transformed into internal projects or proposed as research projects. Our employees are tasked with working much like our eponym Joseph von Fraunhofer, as researchers, inventors and entrepreneurs — all at the same time.

What other “hats” do you wear in the industry?
As mentioned earlier, Fraunhofer is involved in standardization bodies and industry associations. For example, I chair the Systems Group within ISO SC29WG1 (JPEG) and the post production group within ISO TC36 (Cinematography). I am also a SMPTE governor (EMEA and Central and South America region) and a SMPTE fellow, along with supporting SMPTE conferences as a program committee member.

Currently, I am president of the German Society Fernseh- und Kinotechnische Gesellschaft (FKTG) and am involved in associations like EDCF and ISDCF. Additionally, I’m a speaker for the German VDE/ITG society in the area of media technology. Last, but not least, I chair the German standardization body at DIN for NVBF3 and consult the German federal film board in questions related to new technical challenges in the film industry.

IBC: Surrounded by sound

By Simon Ray

I came to the 2016 IBC Show in Amsterdam at the start of a period of consolidation at Goldcrest in London. We had just gone through three years of expansion, upgrading, building and installing. Our flagship Dolby Atmos sound mixing theatre finished its first feature, Jason Bourne, and the DI department recently upgraded to offer 4K and HDR.

I didn’t have a particular area to research at the show, but there were two things that struck me almost immediately on arrival: the lack of drones and the abundance of VR headsets.

Goldcrest’s Atmos mixing stage.

360 audio is an area I knew a little about, and we did provide a binaural DTS Headphone X mix at the end of Jason Bourne, but there was so much more to learn.

Happily, my first IBC meeting was with Fraunhofer, where I was updated on some of the developments they have made in production, delivery and playback of immersive and 360 sound. Of particular interest was their Cingo technology. This is a playback solution that lives in devices such as phones and tablets and can already be found in products from Google, Samsung and LG. This technology renders 3D audio content onto headphones and can incorporate head movements. That means a binaural render that gives spatial information to make the sound appear to be originating outside the head rather than inside, as can be the case when listening to traditionally mixed stereo material.

For feature films, for example, this might mean taking the 5.1 home theatrical mix and rendering it into a binaural signal to be played back on headphones, giving the listener the experience of always sitting in the sweet spot of a surround sound speaker set-up. Cingo can also support content with a height component, such as 9.1 and 11.1 formats, and add that into the headphone stream as well to make it truly 3D. I had a great demo of this and it worked very well.

I was impressed that Fraunhofer had also created a tool for creating immersive content, a plug-in called Cingo Composer that could run as both VST and AAX plug-ins. This could run in Pro Tools, Nuendo and other DAWs and aid the creation of 3D content. For example, content could be mixed and automated in an immersive soundscape and then rendered into an FOA (First Order Ambisonics or B-Format) 4-channel file that could be played with a 360 video to be played on VR headsets with headtracking.

After Fraunhofer, I went straight to DTS to catch up with what they were doing. We had recently completed some immersive DTS:X theatrical, home theatrical and, as mentioned above, headphone mixes using the DTS tools, so I wanted to see what was new. There were some nice updates to the content creation tools, players and renderers and a great demo of the DTS decoder doing some live binaural decoding and headtracking.

With immersive and 3D audio being the exciting new things, there were other interesting products on display that related to this area. In the Future Zone Sennheiser was showing their Ambeo VR mic (see picture, right). This is an ambisonic microphone that has four capsules arranged in a tetrahedron, which make up the A-format. They also provide a proprietary A-B format encoder that can run as a VST or AAX plug-in on Mac and Windows to process the outputs of the four microphones to the W,X,Y,Z signals (the B-format).

From the B-Format it is possible to recreate the 3D soundfield, but you can also derive any number of first-order microphones pointing in any direction in post! The demo (with headtracking and 360 video) of a man speaking by the fireplace was recorded just using this mic and was the most convincing of all the binaural demos I saw (heard!).

Still in the Future Zone, for creating brand new content I visited the makers of the Spatial Audio Toolbox, which is similar to the Cingo Creator tool from Fraunhofer. B-Com’s Spatial Audio Toolbox contains VST plug-ins (soon to be AAX) to enable you to create an HOA (higher order ambisonics) encoded 3D sound scene using standard mono, stereo or surround source (using HOA Pan) and then listen to this sound scene on headphones (using Render Spk2Bin).

The demo we saw at the stand was impressive and included headtracking. The plug-ins themselves were running on a Pyramix on the Merging Technologies stand in Hall 8. It was great to get my hands on some “live” material and play with the 3D panning and hear the effect. It was generally quite effective, particularly in the horizontal plane.

I found all this binaural and VR stuff exciting. I am not sure exactly how and if it might fit into a film workflow, but it was a lot of fun playing! The idea of rendering a 3D soundfield into a binaural signal has been around for a long time (I even dedicated months of my final year at university to writing a project on that very subject quite a long time ago) but with mixed success. It is exciting to see now that today’s mobile devices contain the processing power to render the binaural signal on the fly. Combine that with VR video and headtracking, and the ability to add that information into the rendering process, and you have an offering that is very impressive when demonstrated.

I will be interested to see how content creators, specifically in the film area, use this (or don’t). The recreation of the 3D surround sound mix over 2-channel headphones works well, but whether headtracking gets added to this or not remains to be seen. If the sound is matched to video that’s designed for an immersive experience, then it makes sense to track the head movements with the sound. If not, then I think it would be off-putting. Exciting times ahead anyway.

Simon Ray is head of operations and engineering Goldcrest Post Production in London.

NAB: Fraunhofer Digital Cinema’s Siegfried Foessel and Ben Bross

Las Vegas — postPerspective’s first guests at NAB 2014 were Siegfried Foessel and Ben Bross from the non-profit Fraunhofer Digital Cinema, essentially a tech think tank targeting our industry.

Here are some highlights the two touched upon during our chat:
• High-profile customer announcing its packaging, playback and transcoding components for the Interoperable Master Format (IMF)
• Latest customer order for the easyDCP software suite
• Updates to its Low Complexity Codec for processing and transmission of high-resolution video
• Further advances for its light-field camera recording system, the most innovative system in the industry
Continue reading