AMD 2.1

Doctor helps patients climb mountains with 4K/5K imagery

This space is typically held for articles about broadcast, film, web or corporate projects, but this particular story was just too compelling to ignore. So to quote Monty Python, “And now for something completely different.” I think you’ll enjoy it.

Trek Visions is a company that is visually bringing the world to patients recovering from catastrophic physical and mental injuries with its proprietary software that runs on 42-inch TV monitors at rehab facilities.

What started as a hobby for Trek Visions CEO Dr. Joseph Ludwig has evolved from recording his encounters during climbing expeditions of the world’s most rugged and mountainous terrain to professional physical-rehab applications currently used at the TIRR Memorial Hermann center, a big name in rehabilitation and research, as well as other centers.

Ludwig recently discussed how he has used the latest 5K digital media camera and software technologies to bring these trek applications to market and his vision for the future.

Dr. Joseph Ludwig in Aspen.

Dr. Joseph Ludwig in Aspen.

How did Trek Visions come about?
I had the idea that my oncology (sarcoma) patients at The University of Texas MD Anderson Cancer Center would greatly advance in their rehab programs by exercising in real-world environments, such as the Himalayas, Mt. Rainier, Zion National Park and other global locations. The challenge was to create high-quality videos in which the patients would feel immersed in the environment. This could provide the motivation and involvement in the world that enables positive results for the patient in two ways: a sense of achieving goals when doing the daily workouts and enabling faster progress in the recovery process.

We’ve since expanded to creating more location videos for the benefit of a broader spectrum of patients — those suffering from catastrophic injuries and trauma like Gabrielle Giffords, Arizona’s US Representative who was shot in the head at pointblank range, as well as patients suffering from stroke, cancer, spinal-cord injuries, war injuries and a host of other critical injuries.

How did you create the videos?
In 2008 I began training to climb the Himalayas. Initially I was running on a treadmill, watching two movies per day. I then moved my workouts to Nepal to acclimate myself to the altitude and oxygen levels before I started out on the grueling trek to Mt. Everest. For this video, I hired four Sherpas to help with hauling the 300 pounds of camera gear I brought with me. We would walk roughly five miles a day, and throughout the hikes I carried a Panasonic HVX2000 camera on my chest, shooting at 1080p and recording 24 frames per second. The complete 40-pound rig used to stabilize the camera included a Steadicam with extra weight at the base for balance against the winds; two side-facing cameras; a mattebox; Anton Bauer batteries; and a GPS system. It was a little bit crazy, but the results were definitely worth it.

In this scenario, patients are able to take this same Himalayan trek on the treadmills at the rehab center at their personalized pace, and eventually reach Mt. Everest. This same methodology is used in all the rehab treks like Bryce Canyon, Aspen and others. To ensure each patient has the opportunity to reach the iconic destination of their choice, the distance covered per rehab session varies, with some patients only covering 50 feet and others completing an entire day’s trek. The physical therapist can decide to advance the starting point for each day so that even the most disabled patient can visualize progress along a trek, which is challenging to even the fittest athletes.

So that patients have the full immersion perspective, the GPS for incline/decline is recorded based on the topography algorithm. This allows the patient to have the sense of a realtime experience, moving from walking to running — which doubles or triples the frame rate – and trekking from village to village. For an even greater sense of immersion, the next-generation release will synchronize the incline/decline of the treadmills to closely match the slope of the trials shown in the video.

Right now we have eight employees and we’re actively hiring highly motivated and environmentally conscious film crews that share our passion for providing an unmatched immersive video experience. I’ll also be adding a film crew. I’ll be working with the team to establish the direction and look and feel of each video, but delegating more of the post production, such as the color grading, finishing and mastering, to others with this expertise.

How have you set up your Trek Visions studio?
At the start, I did everything — the shoot, editing, color grading and final mastering to ProRes, and eventually H.264. In 2011, I switched to the Red Epic 5K camera for its high-resolution quality and the realism I was able to impart to the final videos. Beyond just its native 5K resolution, it’s only five pounds and it’s easier to use with a Steadicam when shooting in rugged terrain. Although 4K cameras are now ubiquitous, few cameras shoot above this resolution, which we’ve found to be critical in maintaining a 4K resolution in the final video product. We use Imagineer’s Mocha software for the visual stabilization in post production.

In 2,000 square feet, we now have 100 hard drives to store the video streams and two HP Z820 workstations. I use the Red Epic to record the imagery and then Adobe Premiere Pro for nonlinear editing. I use Assimilate Scratch for the full post and DI in 4K, from conform to color grading, versioning, finishing and rendering out to the final distribution format that is often H.264.

Why Scratch?
There are several aspects to this answer. Scratch has several high-end features that I need for the precise color grading and finishing that make our videos so realistic. Also, all the tools have the common thread of time savings, which for hectic schedules — oncology, cancer research, trek video production, family — is an invaluable consideration.

It’s the ideal software for working with the native Red files, and it’s the only software that offers realtime debayering — the process of converting a Bayer pattern color-filter array image from a single sensor to a full RGB image — for 5K material in full resolution.

All the post tools work in realtime — first-rate color correction, versioning for comparison, changes on the fly — which means saving a lot of time. Assimilate is always adding upgrades, such as the recent addition of Ultra HD color space, new formats (H.265 is just around the corner), and it plays well with other software tools like After Effects, Mocha, Avid and so on. It has dual-audio synced with timecode (Sound Devices T702), which is critical to the quality of the audio/visual experience. The GUI is intuitive — like picking up an iPad — it’s easy to learn, but it also has the high-end features video professionals demand.

In its latest version, the Scratch remote feature for a cloud-based workflow should be a game-changer for my post team since it’s optimized for collaboration and ease of use. I’ve hired two film students at the University of Texas (Austin) to do color grading. By working through the Box cloud, interns and students can load their content from their hard drives into a Box account and I can pull the files for review in Houston. We could also use the remote tool for realtime collaboration sessions as well.

What other software do you find useful?
When shooting, we can be walking from eight to 10 hours a day, and it’s just not possible to shoot in a completely stable manner during this trek. In the videos, the horizon needs to be perfectly stable and for this I use Imagineer’s Mocha. I stabilize every single frame in Mocha — three to five frames per second — and this process can take six to eight hours to stabilize a single hour of data. It’s definitely time consuming, but there really is no other method for precise planar tracking and rotoscoping.

Our film crews are evaluating the latest brushless gimbals to provide 3-axis camera stability. We’ll continue to use the Steadicam arm/vest to maintain a constant camera height, and importantly, to bear the full weight of the camera rig throughout lengthy recording sessions.

Where are Trek Visions’ videos being used?
TIRR Memorial Hermann, located in Houston, is a national leader in rehabilitation medicine. In addition to therapy, the institution goes beyond the healthcare setting to assist individuals with disabling injuries or illnesses, helping people regain the skills and confidence to reintegrate into the community and continue living full and meaningful lives. TIRR Memorial Hermann recently opened the TIRR Memorial Hermann Research Center. This state-of-the-art facility provides researchers with the tools to pioneer new advances in rehabilitative medicine, including neurological research, which can lead to significant improvements in patient lives.

TIRR has developed a comprehensive community integration program called Strength Unlimited where the trek videos are set up for the patients to use when walking, and over the period of recovery, running on the treadmills. The videos can also be used during the other movement activities that encourage patients to become upright and mobile.

Sandra Lloyd, the director of business development at TIRR Memorial Hermann, and her team of professionals and therapists are the experts in making this program successful for patients. They have over 160 therapists on staff and work with over 500 outpatients a day. TIRR and its affiliate, the Memorial Hermann Rehabilitation Network, have 250 beds for patients who come from anywhere in the world. Many patients at the MD Anderson Cancer Center are referred to TIRR for cancer rehabilitation.

Trek Visions has begun working with the National Institute of Health and other medical centers to customize the TrekFit application as a tool for enabling cutting-edge research of cancer-related fatigue, chemotherapy-induced heart dysfunction and the positive effects of exercise in pediatric-cancer survivorship.

What is your vision for future applications of Trek Visions’ videos?
The videos are all about motivation through real-world environments to get people upright and moving for improved health or recovery.

The app

As time goes on, we’ll be adding videos of other global locations — French Alps, golf courses, beaches — that patients don’t have access to but can enjoy through immersion into these highly realistic environments.

We’ve developed a commercial TrekFit iPad application that can be used in other unique applications, such as by athletes in training, inner-city kids to experience more of the world or anyone who has a need or desire to improve their physical capabilities — balance, strength, endurance or performance.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.