Creative Workflow Hacks

Apple released Final Cut Pro 5.1.2 (pdf link) which addresses a lot of the problems that I’ve been running into lately. I want to take a bit of time to dig into the extensive bug fixes, and folks are already doing just that, but one change that I was not expecting really caught my eye. A new version of the Final Cut XML interchange format introduced some new elements.

The two that I find most interesting allow for management of metadata in QuickTime files and management of project components. I’ve only read the summaries and there could be some implementation gotchas, but what these appear to do is to open up a lot of the project management possibilites I’ve been hoping for in Final Cut Pro

If you develop a rich catalog of media and a strong database you should be able to build, replace and alter projects based on your workflow using elements like replaceiffound which replaces a target component if found. and addifnotfound which adds the imported component to the project provided the target component is not found. Combine that with adding and altering metadata in Quicktime files your workflow can really be enhanced. Obvious stuff like all files for a project as well as non-obvious, fuzzier ideas like all clips in a certain color range. Very cool. Time to dig in.

UPDATE: 09-27-2006 So, the glue that will hold this together is a set of 7 apple-events to use as an interchange between your custom app and FCP. Lots of possibilites. Download the sample app here.

SECOND UPDATE: 09-27-2006 Well, it looks like this gets us part of the way there. The apple-events seem to work nicely for sending to FCP, but it still doesn’t have an applescript dictionary which I think means we won’t be sending anything (like maybe the selected clips in a sequence) from FCP with this update. What that means is you’ll be able to do cool batch processing on the days work as a chron job and have lots of interesting data, but won’t be able to do things like space elements in a sequence unless I’m missing something. There are some private and undocumented data interchange protocols, but those require reverse engineering and are going to be out of the scope of your average power user or scripter.If anybody knows more please post a comment.

Interesting post from Shane Ross about some monitoring problems in FCP.

His Workaround

So we cancelled the output and tried to figure out what to do.

The engineer…my buddy…had a solution. And this is a nifty workaround (Workaround #214 for those keeping tally of needed FCP workarounds to get things working properly) for those of you taking notes. EXPORT VIA QUICKTIME CONVERSION. Make the settings match those of your project exactly. When you re-import that footage and drop it in the timeline above the originals, you will find that the conversion has chopped off the portions that are above 100IRE, and clamped (chooped, whatever) the portions that go below black. In other words, it does what Broadcast Safe should do, but doesn’t really do.

We’ve been struggling with inconsistent monitoring in our After Effects to Final Cut through AJA Kona products workflow. Particularly interesting has been the vast differences between the digital scopes in Color Finesse in After Effects and the digital scopes in Final Cut and haven’t been sure which was going awry. I’ve seen less of a problem in our monitoring using hardware scopes and Kona throughput. I’m not sure if I’d want to do a Quicktime conversion on every element that went to tape, but it is an interesting observation. I hope a lot of this stuff is just a new generation of gear and software and gets worked out soon, but anecdotally I’ve been struggling to get consistent color workflows. Anybody else?

Follow up: 09-15-2006 Martin Baker from Digital Heaven posted a comment at Shane’s blog where he felt that Quicktime Conversion was going through RGB thus clipping subblack or superwhite levels. Which certainly makes sense. Most of the issues I’m chasing seem to have arisen when we went to an RGB codec via Kona 10Bit 4:4:4. Throw in a custom LUT and it’s hard to find your benchmark. I’m still not sure why we are getting different values when monitoring on individual workstations via Color Finesse vs Final Cut it seems like they should be reading the same RGB values. Throw in yet again Stu Maschwitz’ comment on the AE List about ICC implementations

> The biggest problem I’ve seen with ICC implementations is that they
> do precious little to guide one towards best practices.

I almost agree Tim — I think that’s the second biggest problem. The
biggest to me is the lack of a reference implementation. Meaning any
two software apps can produce any two versions of an ICC color
xform. Kinda makes the whole system seem bogus.

And you’ve got a big case of Yikes! going. If these guys can’t get it right, how am I supposed to?

UPDATE: 09-26-2006 Mike Curtis has an extensive post and comments followup on these issues. All signs seem to be pointing to a QuicktimeYUV/RGB conversion issue. I’ll try to followup with a post or point to a distillation of how to work with these issues once a consensus is available.

When we were last talking about reading timecode from FCP sourced Quicktime’s I followed a tangent into Spotlight and other OS X technologies. Well, recently I revisited this problem for another project I’m working on and I think I’ve got a working solution for reading embedded timecode in After Effects on OS X.

timecodereader is a command-line utility that takes a path from After Effects via system.callSystem() and returns either the embedded timecode or the string . Right now the included .JSX script just writes that information to the console, but there are a lot of cool uses for Quicktime timecode in productivity scripts.

How to Use timecodereader

Download the zip archive, and either compile a new binary from the included XCode project or move the already compiled binary from build/release to a local binary directory. Since this is a command line tool you will want to move it to a location in your Path statement so you can use system.callSystem(”timecodereader ” + String(thisItem.mainSource.file))” instead of system.callSystem(”/path/to/timecodereader ” + String(thisItem.mainSource.file)) .

Update 09-09-2006: Lloyd Alvarez points out at aenhancers that AE preferers ~/bin/timecodereader when we set up as below. I’ve adjusted the enclosed script.

In the default Bash shell

export PATH="$PATH:~/bin"
echo 'PATH="$PATH:~/bin"' >> .profile

will setup a path to bin in your User directory. Move the enclosed script to your Scripts folder, select a Quicktime file in the Project window and we’re done.

When I run the script, I get these results (The 1:00:00:00 timecode is due to the fact that I’m using writeln to write to the console so the previous lines scroll up until they are out of view).

I’ve tried timecodereader with quite a few Quicktime’s but obviously there might be a few that aren’t handled well, so consider this a beta release. The compiled binary is a universal binary so it should work on both PowerPC and Intel platforms. One gotcha, is that if you have a codec that is not compiled for the supported platform you might get a dyld returns 2 log message that might futz with your script. You can either work around it via string handling or remove the codec since it won’t work anyway. You’ll need to turn on Allow Scripts to Write Files and Access Network and turn off Enable JavaSript Debugger so we can eat the errors.

timecodereader requires After Effects 7, and OS X 10.4+

Updated file: 09.09.2006

Fixed a problem with external Hard Drives and Mounted Volumes. After Effects does not add /Volumes/ to drive paths like OS X, so /Volumes/MyHardDriveName is reported as /MyHardDriveNameI’m basically checking all mount points and adding /Volumes when appropriate, unfortunately the response is a bit slower if you have disk images or a lot of volumes mounted. If anybody knows anything faster send it my way.

Also. I’m using metadata checks for com.apple.quicktime-movie to check whether the file is a quicktime file. This only works on volumes that are indexed, so it might not work on mounted server volumes. I can add alternative QT detection if anybody finds it useful or necessary.

Zip Archive of Binary, Script and XCode Project

I was browsing around Apple’s Tech Note website when I ran into this technote. Apply the Broadcast Safe filter last in Final Cut Pro. The article discusses the importance of applying the Broadcast Safe last in a filter stack to insure that you really are limiting the colors in a sequence to broadcast safe colors.

The idea of a pipeline is one of the trickier ideas to master for new users of image processing applications. In image processing each new operation like a blur, composite operation or color adjustment is applied and then passed to the next operation. In the example that Apple gives above, a Proc Amp operation applied late in the pipeline could boost the color levels outside of a safe range even though Broadcast Safe filter was applied in the stack. I’ve often found when you’ve got a visually unexpected output, moving the filter arrangement will restore some sanity to the expected output.

Additionally, as applications like After Effects transition to 32 bit float environments you’ll see additions to pipelines like HDR Compander which can compress and decompress the pixel pipeline to work with 8 and 16bit effects.

It is important to get a strong conceptual understanding of core concepts like image processing pipelines. These ideas become even more important with programs moving to 32 bit float color and using pixel-level operations via the GPU like in Apple’s Core Image. Finally, I really recommend that you keep up with what Stu Maschwitz posts at Prolost. He writes about really hard tech like color space and image processing but with attention to form and artistry I can only aspire to.

I love using batch media programs like Cleaner and Compressor. They allow you to set up a workflow and accomplish a lot of work unattended. What I didn’t realize until just recently was that there was a great little mini-batch processing program built right into Final Cut Pro.
Read the rest of this entry »

A recent thread on aenhancers discussed reading source timecode from Quicktimes imported from Final Cut Pro captures in After Effects. Seems like a perfectly reasonable request, there could be a bunch of situations where this information would be useful in an After Effects project. Problem is, it doesn’t seem really easy to grab that data. When you view the timecode track in the imported quicktime, the timecode is in reference to the source Quicktime not the source tape from the capture.

Correction

Mark Burton posted on the thread mentioned above that actually the timecode is written to the Quicktime. He offered a link to Sebsky Tools as an example of an app using that info. I simply misread the Quicktime player info. There is still useful info from Final Cut we can access and I think the ideas in this article are interesting so I’m leaving it up with this correction. Thanks Mark.

At least until Apple decides to put that information into the captured Quicktime We’ll need to find a workaround way to read FCP info and that’s what I’m going to cover in this post.

Read the rest of this entry »

Update 04.17.2007

I’ve posted a small application and tutorial at Final Cut Pro to After Effects Scripting without the hassle for those that find the scripting presented here a little daunting. The following article covers the steps and ideas encapsulated in the application if you’re more interested in the process for learning to script After Effects and Final Cut Pro yourself.

I work a lot in Final Cut Pro and After Effects. They are both really strong programs and more than once I’ve had reason to move projects, art, text, etc. between the two programs.

Until recently, the only option you had to do this kind of interchange was a product called Automatic Duck (which I highly recommend btw) and in previous versions you still had to use Avid’s interchange format and jump through a few hoops to move between the programs.

Now with Final Cut XML and more robust scripting available in After Effects you can finally flow between the two programs with relative ease. In this first installment, I’ll outline a simple workflow for getting clips in a FCP sequence into a AE project. In future posts I’ll go over text elements, filters, generator items etc., but all of these follow the same general development cycle and will just be elaborations on a theme.

Read the rest of this entry »