Hey Everybody,
If you’re a regular follower of the film, you likely know that we’re a two person team making this film. And by ‘making’ this film, we pretty much mean all of it. Shooting, Editing, Graphics, Sound, Distribution, Web, travel bookings...all of it (with the HUGE exception of Jim Guthrie’s fantastic score).
As a result, we receive quite a few technical questions asking us about gear, technique etc. We are hoping to do a few posts in the upcoming little bit that covers how we put (are putting?) the film together. You can check out a previous gear-based post here. On that note...
Let’s talk about IGTM Hard Drives!
Disclaimer: We are not file management experts...at all. This post is going to explain how we handled storage for this project. It’s the kind of a cobbled together strategy that worked for us (though we’d do some things differently), but likely isn’t a gold standard.
Hard Drives on the Road:
Filming - How Much Was Shot? The majority of IGTM was shot on one big road trip from Sept 2010 - Dec 2010 (with a couple of weeks of shooting in Feb 2011). Over the course of the film, we had 39 days of shooting on the road.
Most shooting was usually (but not always) done in 2-3 day chunks, centering around a particular interview subject (e.g. The ‘Derek Yu’ or ‘Jason Rohrer’ leg of the trip)
The first shoot day was our heavy sit down interview day. An interview could last anywhere from 2 hours to the 5 hour marathon we did with Jon Blow - who was game for anything and super patient with us. The second and third days were (usually) reserved for B-roll, active interviews and scenic shooting. These days could last anywhere from 5 - 12/14 hours.
In terms of footage, this means, at the end of a shoot, we’re typically walking away with 5-10 hours of activity covered (subject interview + coverage). Now, making matters worse (but the production better) is the fact we shoot nearly everything 2 camera. Meaning that the 5-10 hours of filming actually turns into 10-20 hours of actual footage.
On our cameras of choice, the Canon 5D Mark II, that amount of footage, 10-20 hours, equals approximately 166 - 333 Gigabytes of data.
Each shoot produced between 166 - 333 GB of data.
Hotel Room Backups: Every evening after a shoot - before dinner, before unwinding, before ANYTHING - We would backup all data 3 times. For this we used...
- 2 MacBook Pro Laptops
- 1 FireWire 800 Sandisk Card Reader
- 3 Seagate GoFlex Ultraportable drives (2TB)
The GoFlex have a pretty solid build to them and have the neat bonus of being able to use FW800/400 or USB 2.0/3.0 connectors. This can be really handy when you start to load your laptop with peripherals.
After backup, 1 copy goes in Lisanne’s Camera Bag; 1 goes in James’, and the third stays in the hotel room.
As we do more shoots, we load up more Seagates. And at any time, either of us should have an entire copy of all material captured on that shoot with us.
While You Were Sleeping - Hotel Room Transcoding: In our daily schedule, after the backups, came the transcoding.
If you’re familiar with Canon HD-DSLR workflow, you’ll know that in order to (nicely) work with the Camera’s footage, you have to first transcode the footage from its native H.264 codec into something more edit-friendly.
For this film we made the choice to transcode and edit in apple’s PRORES 422 LT codec. The reasoning was pretty simple. It’s a strong, Apple friendly codec with a reasonable data rate. We did play with the idea of transcoded to XDCAM, but we didn’t want some unforeseen ‘gotcha’ popping up when it comes to any post-production processes for theatrical or broadcast distribution. And of course, we didn’t go with the higher flavours of PRORES, as it just seemed like overkill, data-rate wise.
We tried to transcode all footage while on the road. Our reasoning:
Transcoding takes a long time (it's nearly a realtime process, i.e. 1 hr of footage = nearly 1 hr transcode). We didn’t want to arrive home and have to wait for a week’s worth of transcoding into to start editing. We were also producing web videos while on the road. We wanted to be in the position of being able to edit at a moment’s (or whim’s) notice.
Soooo, this meant taking the 200+ GBs of the day’s footage and transcoding it (via MPEG Streamclip) while we slept. For this transcoded footage, we carried with us multiple (usually 4) LaCie BigDisk Quadra. Trying to maintain 2 identical copies of the transcoded material plus whatever project files may be created in producing the on-the-road segments.
A slight downside of the transcoding is a big bloat in data size.
16.69 GB (1 hr) of H.264 = 35.87 GB (1hr) of ProRes 422 LT
Meaning, transcoding increased file size (just over) twofold. Which simply meant more HDs.
Carry-On Movie: When traveling by plane, our strategy was pretty simple. No essential items get checked. Carry-on as much as humanly possible. To Hell with your personal comfort items.
We each carried a flight-approved camera bag and a laptop bag. In these bags were:
- 2 Camera bodies
- 4 Lenses
- 2 Audio Recorders
- 2 LAV mic setups
- A LOT of batteries & CF cards
- 2 laptops
- 2 COMPLETE copies of the footage shot on that trip.
Everything else could get checked. That way, if our luggage was lost forever, not only would we still have our hard-earned footage, but we’d also have enough of a setup that we could still hit the ground running and shoot whatever we needed to once off the plane.
Home Base Drop-offs: During this extended road trip, we did make Phoenix our home away from home. Using one of our families’ homes there as a kind of base camp for the vast majority of filming. Over the course of the Fall/Winter, we’d do about 2-3 shoots at a time, regroup in Arizona and head back out.
This allowed us to off-load all raw and transcoded footage to a stationary and safe collection of harddrives. After each trip, we would copy the new footage/project file to three identical backup harddrives (again, a series of LaCie Little Big Disks). One copy was a working/editing drive(s), another was its on-site backup. While the third copy was stashed off site in a secure location.
Wrapping Up: So, that was our backup workflow while on the road. It was a bit cumbersome at times, but it worked for us. We’d likely do the same again, with one possible exception - I may have incorporated the file transfer tool ‘Shot Put Pro’ into the mix for another layer of security. ShotPut Pro is a tool that verifies the proper transfer/offload of data files from your CF cards.
In all of our transfers (thousands of files), we only had ONE single file that was corrupt via transfer. Not bad at all. But, that’s not to say I wasn’t stressing all the time about whether or not things copied properly. Shot Put could’ve helped with that stress a bit.
We’ll try and do another (though hopefully shorter) blog on the harddrive management we’re doing during the edit soon.
Hopefully you’ve found this useful. Any questions, feel free to give us a shout!
-James
J + L