The perils of shooting film-style in the digital era, and how to avoid your pictures looking awful.
By Bruce Devlin
Long, long ago life was simple. Premium content for television was shot on film, using Hollywood-style cameras at Hollywood-style 24 frames a second. It was cut on film, and effects were done on film. Even delivery from production to post to broadcaster was done on film.
When the time came to show it, the broadcaster laced it up on a telecine. For 25fps (frames per second), countries they simply ran the film fast. In North America, where the frame rate is 30fps, they had to do something slightly clever. They invented the 2:3 pulldown or film-cadence.You get two fields out of one film frame and three fields out of the next. It worked well enough and everyone was delighted.
(Yes, I know that sometimes 24fps is actually 23.98fps, and 30fps should really be 29.97fps or 59.94 fields per second. We’ll leave the higher math to those who care about it).
Today we shoot digitally. But cinematographers and directors love the look and language of film, so they prefer to shoot on 24fps digital cameras when they can. Which is fine: it looks great and everyone is happy. Somewhere along the line you have to convert from 24fps with our old friend the 2:3 sequence, but we’ve known that works for 80 years.
The problem is that there are a lot of stakeholders between the camera lens and the viewer’s television screen. They include the production company, the post house, the channel management and the facility playing out the final program. And every one of them has the power to break the 2:3 sequence, which will make our beautiful pictures stop looking beautiful.
Along that stakeholder chain, people might want to edit for duration, for content or censorship, to add credit rolls or for all sorts of other commercially driven processes. If you do not understand what you are doing, bad things happen to the structure of the pictures. That is why I call it the automatic garbage inserter.
Listen to Bruce’s presentation here: http://youtu.be/J_wceBcJ9Ow
Let’s go back to our 2:3 sequence. Four frames of film (ABCD) are displayed in five frames of video, by showing two fields of original frame A, three fields of original frame B, two of C, three of D and so on. Presto: 24fps original converted to 60 fields/s output, without the use of a safety net.
What happens, though, if we edit out original frame C, is that we no longer have a 2:3 sequence. Instead, we have a 2:2:2:2:3 sequence. You may not want to remove one single frame, but if you remove an odd number of frames, the same problem occurs.
Now imagine you put a credit roll on top of it, and maybe some semi-transparent graphics or effects, or even do a squeeze back. You get a lot of video motion on top of the broken 2:3 sequence. Then you try to standards convert it for Europe, or worse still, you want to send it back to 24p for iTunes because online video sensibly bans blended frames that result from interlacing.
I have seen all of this and more, and I can tell you it is not pretty. Not pretty at all. For the cinematographer who lovingly shot the original on the best 24-frame digital camera the production budget could buy it is heartbreaking.
So what do you do to repair the broken mixed video/film/garbage? Well, in order of technical complexity, here are some things you could try:
Do nothing — very fast in software, but unfortunately does not actually change the frame rate or perform any repair. For many short segments of a longer production that haven’t been broken — this is exactly the right thing to do.
Change the metadata — if you tell the receiving device the content is at a new frame rate, it will read the video at that speed. You can get away with it when converting 24fps to 25fps, but a speed bump of a quarter to 30fps is not going to be nice. Don’t forget to resample the audio otherwise the video and audio durations won’t match. If you need perfect pitch, then you’ll also need to correct the audio that is now four percent too fast.
Drop or repeat — you can increase the frame rate by repeating some frames; decrease it by dropping some frames. After all, this is sort of what we do with the 2:3 cadence. But repeating frames regularly is blindingly obvious and your audience will certainly notice. Only to be used sparingly and intelligently to repair problems.
Linear interpolation — basically, blend two pictures together to make a new one. But the sharper the original pictures, the worse the artifacts will be. Fast motion and sharp edges become really smeary — this can be very visible.
Motion compensated frame rate conversion — this is the same sort of process, but this time every pixel in the scene is measured and its motion vectors predicted. New pixels in new pictures are a linear blend of the source pixels projected along the motion vectors. Smart processing can predict with a degree of confidence whether a pixel is a moving object or noise. Works really well with a lot of picture types, but some repetitive images – like car wheels or pans across brickwork – are very difficult to process and result in very disturbing results.
Black box magic — clever inventors regularly come up with new approaches and better algorithms. The chances are, though, that like motion compensated conversion, the current gold standard, it will work well with one sort of movement and not with another.
So which do you choose? My answer is not to make any single choice, instead you should have the flexibility to choose the one that works best at the time.
If you go out and buy even the very best hardware box with the latest technology, your only options are to have the process in the circuit or bypassed. But if you do it in software you can change process as appropriate, even scene by scene or shot by shot.
We have a white paper (http://info.amberfin.com/whitepaper-standards-conversion) that talks more about the different processes, how they are implemented in software on a platform like iCR, and how you can switch transparently between them.
But the best answer is always going to be edit and post with care so that you do not break the 2:3 sequence — bypassing the automatic garbage inserter is always going to be better than fixing it when it goes horribly wrong.
Bruce Devlin is the CTO of AmberFin (www.amberfin.com), and the star of Bruce’s Shorts, a series of short videos and exclusive Webinars. He is also co-author of the MXF format.