Creative Workflow Hacks

Apple released Final Cut Pro 5.1.2 (pdf link) which addresses a lot of the problems that I’ve been running into lately. I want to take a bit of time to dig into the extensive bug fixes, and folks are already doing just that, but one change that I was not expecting really caught my eye. A new version of the Final Cut XML interchange format introduced some new elements.

The two that I find most interesting allow for management of metadata in QuickTime files and management of project components. I’ve only read the summaries and there could be some implementation gotchas, but what these appear to do is to open up a lot of the project management possibilites I’ve been hoping for in Final Cut Pro

If you develop a rich catalog of media and a strong database you should be able to build, replace and alter projects based on your workflow using elements like replaceiffound which replaces a target component if found. and addifnotfound which adds the imported component to the project provided the target component is not found. Combine that with adding and altering metadata in Quicktime files your workflow can really be enhanced. Obvious stuff like all files for a project as well as non-obvious, fuzzier ideas like all clips in a certain color range. Very cool. Time to dig in.

UPDATE: 09-27-2006 So, the glue that will hold this together is a set of 7 apple-events to use as an interchange between your custom app and FCP. Lots of possibilites. Download the sample app here.

SECOND UPDATE: 09-27-2006 Well, it looks like this gets us part of the way there. The apple-events seem to work nicely for sending to FCP, but it still doesn’t have an applescript dictionary which I think means we won’t be sending anything (like maybe the selected clips in a sequence) from FCP with this update. What that means is you’ll be able to do cool batch processing on the days work as a chron job and have lots of interesting data, but won’t be able to do things like space elements in a sequence unless I’m missing something. There are some private and undocumented data interchange protocols, but those require reverse engineering and are going to be out of the scope of your average power user or scripter.If anybody knows more please post a comment.

Have you had After Effects 7 hang on Initializing User Interface on start up? Like so…

I’ve seen a couple of instances of this problem and it seems to occur when a User Workspace becomes corrupted. The fix that’s worked for me is to replace the folders at

In OS X
YourLoginName/Library/Preferences/Adobe/After Effects/7.0/ModifiedWorkspaces/

and

YourLoginName/Library/Preferences/Adobe/After Effects/7.0/OriginalUserWorkspaces/

with backups. If you don’t have a recent backup, you can delete these folders and they’ll be recreated but you’ll lose your custom workspaces.

I’ve only had this crop up on OS X machines to this point, so I haven’t dug into the custom workspace definitions locations for Windows. If anybody has had to track this down for Windows could you leave a comment or send an email to dale(at remove-this)creative-workflow-hacks(dot)com and I’ll update this post.

UPDATE 11-18-2006 David Baertsch writes…

After reinstalling AE 7.0 for Windows, my system would hang on “Initializing User Workspace” I googled that
and got to your fix for Mac, that sent me in the direction to find the fix for Windows.

C:\Documents and Settings\UserName\Application Data\Adobe\AfterEffects\7.0\ModifiedWorkspaces

“UserName” is the name of the logged in individual, and the UserWorkspace1.xml files are the saved
workspaces… I deleted them and though my Workspaces were deleted, Aftereffects launched fine after
that. (There were several files UserWorkspaceX.xml)

Thanks for giving me the idea where to look. Hope this helps other users.

Interesting post from Shane Ross about some monitoring problems in FCP.

His Workaround

So we cancelled the output and tried to figure out what to do.

The engineer…my buddy…had a solution. And this is a nifty workaround (Workaround #214 for those keeping tally of needed FCP workarounds to get things working properly) for those of you taking notes. EXPORT VIA QUICKTIME CONVERSION. Make the settings match those of your project exactly. When you re-import that footage and drop it in the timeline above the originals, you will find that the conversion has chopped off the portions that are above 100IRE, and clamped (chooped, whatever) the portions that go below black. In other words, it does what Broadcast Safe should do, but doesn’t really do.

We’ve been struggling with inconsistent monitoring in our After Effects to Final Cut through AJA Kona products workflow. Particularly interesting has been the vast differences between the digital scopes in Color Finesse in After Effects and the digital scopes in Final Cut and haven’t been sure which was going awry. I’ve seen less of a problem in our monitoring using hardware scopes and Kona throughput. I’m not sure if I’d want to do a Quicktime conversion on every element that went to tape, but it is an interesting observation. I hope a lot of this stuff is just a new generation of gear and software and gets worked out soon, but anecdotally I’ve been struggling to get consistent color workflows. Anybody else?

Follow up: 09-15-2006 Martin Baker from Digital Heaven posted a comment at Shane’s blog where he felt that Quicktime Conversion was going through RGB thus clipping subblack or superwhite levels. Which certainly makes sense. Most of the issues I’m chasing seem to have arisen when we went to an RGB codec via Kona 10Bit 4:4:4. Throw in a custom LUT and it’s hard to find your benchmark. I’m still not sure why we are getting different values when monitoring on individual workstations via Color Finesse vs Final Cut it seems like they should be reading the same RGB values. Throw in yet again Stu Maschwitz’ comment on the AE List about ICC implementations

> The biggest problem I’ve seen with ICC implementations is that they
> do precious little to guide one towards best practices.

I almost agree Tim — I think that’s the second biggest problem. The
biggest to me is the lack of a reference implementation. Meaning any
two software apps can produce any two versions of an ICC color
xform. Kinda makes the whole system seem bogus.

And you’ve got a big case of Yikes! going. If these guys can’t get it right, how am I supposed to?

UPDATE: 09-26-2006 Mike Curtis has an extensive post and comments followup on these issues. All signs seem to be pointing to a QuicktimeYUV/RGB conversion issue. I’ll try to followup with a post or point to a distillation of how to work with these issues once a consensus is available.

When we were last talking about reading timecode from FCP sourced Quicktime’s I followed a tangent into Spotlight and other OS X technologies. Well, recently I revisited this problem for another project I’m working on and I think I’ve got a working solution for reading embedded timecode in After Effects on OS X.

timecodereader is a command-line utility that takes a path from After Effects via system.callSystem() and returns either the embedded timecode or the string . Right now the included .JSX script just writes that information to the console, but there are a lot of cool uses for Quicktime timecode in productivity scripts.

How to Use timecodereader

Download the zip archive, and either compile a new binary from the included XCode project or move the already compiled binary from build/release to a local binary directory. Since this is a command line tool you will want to move it to a location in your Path statement so you can use system.callSystem(”timecodereader ” + String(thisItem.mainSource.file))” instead of system.callSystem(”/path/to/timecodereader ” + String(thisItem.mainSource.file)) .

Update 09-09-2006: Lloyd Alvarez points out at aenhancers that AE preferers ~/bin/timecodereader when we set up as below. I’ve adjusted the enclosed script.

In the default Bash shell

export PATH="$PATH:~/bin"
echo 'PATH="$PATH:~/bin"' >> .profile

will setup a path to bin in your User directory. Move the enclosed script to your Scripts folder, select a Quicktime file in the Project window and we’re done.

When I run the script, I get these results (The 1:00:00:00 timecode is due to the fact that I’m using writeln to write to the console so the previous lines scroll up until they are out of view).

I’ve tried timecodereader with quite a few Quicktime’s but obviously there might be a few that aren’t handled well, so consider this a beta release. The compiled binary is a universal binary so it should work on both PowerPC and Intel platforms. One gotcha, is that if you have a codec that is not compiled for the supported platform you might get a dyld returns 2 log message that might futz with your script. You can either work around it via string handling or remove the codec since it won’t work anyway. You’ll need to turn on Allow Scripts to Write Files and Access Network and turn off Enable JavaSript Debugger so we can eat the errors.

timecodereader requires After Effects 7, and OS X 10.4+

Updated file: 09.09.2006

Fixed a problem with external Hard Drives and Mounted Volumes. After Effects does not add /Volumes/ to drive paths like OS X, so /Volumes/MyHardDriveName is reported as /MyHardDriveNameI’m basically checking all mount points and adding /Volumes when appropriate, unfortunately the response is a bit slower if you have disk images or a lot of volumes mounted. If anybody knows anything faster send it my way.

Also. I’m using metadata checks for com.apple.quicktime-movie to check whether the file is a quicktime file. This only works on volumes that are indexed, so it might not work on mounted server volumes. I can add alternative QT detection if anybody finds it useful or necessary.

Zip Archive of Binary, Script and XCode Project

Update 9/11/2006 6:20PMI said I’d update this post when updated drivers were available, and it looks like the 8/29/2006 drivers from AJA have addressed this issue. Download them here.

Update 8/19/2006 2:50PM OK. It appears that downgrading the codec to version 1.2 fixes the issue. Run the uninstaller, and grab the 1.2 codecs at Aja’s site. I’m going to leave this up for any future Google searches and I’ll update it when there is an upgraded codec. Thanks to Evan on this one.

I’m running into a problem with Kona 10bit RGB output with trillions+ that I’m hoping somebody might be able to help with since I’m on a really tight deadline. I’ve been happily producing 10 bit RGB quicktimes with trillions + option for weeks now. I bought several new machines that have introduced a new endian option in the compressor options like so…

This is what all of our older machines with the same version of OSX, Quicktime and Kona drivers look like…

All of the Quicktimes we produce from the computers with the endian option produce black frames no matter what combination we use when trillions+ is selected. I’m aware of endian issues with the new Intel Macs vs G5 Macs, but I can’t seem to produce a working combination and like I said all software is up to date on all machines. I’m guessing a firmware issue or something. I’m hoping somebody has a working workaround and can send a comment or email to dale(at)creative-workflow-hacks(dot)com. Normally, I’d just go through tech support options, but my deadline is pressing. Thanks folks.

After Effects 7 has debuted to decidedly mixed reviews at work. While most folks are excited about the new features, there is a general unease with the new interface. It’s as if we’re dealing with a new Adobe OS. I think as we get a few projects under our belt the unease will dissipate a bit especially since we have such a history with the application and it is strong in features and implementation. My personal opinion is that user feedback will moderate the interface in future versions and I want to point out one example of a platform specific UI cue that is an example of why you might want to not make an application too uni-platform.

A dirty document in OS X is a document with unsaved changes. There is a clear way to indicate this. Here are a few examples in some Adobe apps.

In Photoshop, a document before making a change

In Photoshop, after making a change

In Illustrator, a document before making a change

In Illustrator, after making a change

Notice the filled in bubble on the left? Now let’s do the same thing in After Effects.

In After Effects, a document before making a change

In After Effects, after, umm, making a change

In After Effects we lose an OS X interface cue. My guess, is this is an oversight from creating the interface with non-standard GUI widgets and that’ll get fixed in a later verson. But it is a great example of what happens when you try to go it alone in interface land, especially with Mac users who are a notoriously finicky lot. So, when I talk to Adobe developers, I’ll be sure to put in my voice for letting each platform retain its own individual personality.

btw…I am aware of the asterisk next to the file name cue, but this isn’t an either/or situation, so many interface cues are practically subliminal, and if a quick glance to the left corner doesn’t give me what I’m looking for, I’ll just think that it isn’t Mac like without really knowing why.

A common scripting requirement is to pull random values from within a defined range. For example, in the block dissolve transition the individual blocks of video are extracted from the video in a random order. Let’s revisit our float away video wall script and customize it to work in the same manner as a block dissolve but with a little more flair. We will scale the individual blocks up instead of just transitioning on and off.

Read the rest of this entry »

I’ve always been skittish around After Effects expressions, which when you think about it really doesn’t make much sense. I’m pretty good at programming and I get the concepts, so why the discomfort? Recently, I was adding some expressions to scripts and found myself really enjoying the process. I thought about it a bit, and it wasn’t the expressions I was skittish about, it was the entering the expressions that was making me uncomfortable. The one line data entry, the hierarchical function list, even the pickwhip with its cool little animation feels utterly confining and claustrophobic. What I was struggling with was a programming interface developed for use by designers.

And it’s not just After Effects. Flash has had that same kind of push and pull between developers and designers. From the hideous Normal/Expert mode to the pick your identity at startup evolution, every version seems to take another stab at the right mix. Even tools that have evolved into developer focused applications continue to show the roots of the struggle. I wince when I watch my Filemaker Pro using friends developing Repetitive Stress Injuries from the legacy programming environment, which amounts to a 400px x 200px bit of screen real estate, filled with buttons. Ok, I exaggerate, but still.

I’ve always been the type of guy who pinballs between the left and right sides of the brain. I’ve got a design background, but I like to program and end up working on the technical side of projects a fair bit. I think this is pretty common these days, Jen deHann coined the term deseloper (not much here right now) to describe that combination of designer and developer. What happens sometimes when I have my developer cap on in a design tools environment is a sort of cognitive dissonance between the ideas I want to express and the program interface.

I think this happens from a misguided understanding of what being a designer is all about. Design is not about being able to twiddle buttons and graphics, but about problem solving. Some of the most amazing problem solvers and process experts I know are designers. There has to be a better toolkit for letting designers express their ideas. There is a term used in computer science about a languages expressiveness or its ability to let the programmer express her ideas easily and without hinderance. I think there is a lot of room for design tools to develop that expressiveness outside of the gui toolkit. It’s obvious from the experimentation in interfaces within design applications that there is an embrace of the problem I’m trying to articulate.

My main hope is that there continues to be a flowering of cross discipline avenues of exploration. I hear similar complaints from my developer friends who feel like they are pushed away from creative solution making. I think there will always be folks who are more comfortable in a more analytical setting as well as others who embrace the more feeling parts of the Myers Briggs spectrum, but there’s also a fertile middle ground for us deselopers to explore.

One of the notable short comings of the workflow I outlined for converting Final Cut Pro elements into After Effects projects via scripting was the limited path support. That was mainly due to the fact that Final Cut Pro uses a fully qualified path url of the form file://path/to/file and After Effects has an Absolute URI that uses the tilde as a shortcut for the current user’s home directory. So, file://Volumes/MyHD/Users/Dale/Stuff becomes ~/Stuff. Trying to munge all of the contingencies in Javascript is not a pleasant task, and Cocoa has a nice set of methods for dealing with just this situation, including stringByAbbreviatingWithTildeInPath and stringByExpandingTildeInPath.

What I’ve done is written a small Cocoa foundation tool that works on the command line to move between the formats of interest. The tool takes the following switches.

pathconvert -u pathurl

Takes a pathurl of the form file://localhost/Volumes/Storage/Users/Dale/test used by Final Cut XML and converts it to an AbsoluteURI that After Effects will be happy with, including using stringByAbbreviatingWithTildeInPath to resolve directories in the User Home folder, such that the pathurl above becomes ~/test.

pathconvert -f fullpath

Takes a fullpath of the form /Volumes/MyHD/Users/Dale/Test and returns a short cut tilde path when appropriate. The string above would become ~/Test.

pathconvert -t abbreviatedPath

Takes a short cut path of the form ~/Test and returns a full path when appropriate. The string above would become /Volumes/MyHD/Users/Dale/Test.

This utility, combined with system.callSystem() in After Effects 7 will make our work with Final Cut Pro XML a lot more friendly in a scripting environment. Next up is an updated tutorial on working with Final Cut XML and After Effects scripting and more detailed instructions on how to install and use this utility. I wanted to post this now to get feedback from power users and developers on potential problems and gotchas. I’m not much of a c hacker, so I’m enclosing both the XCode project files and a compiled binary. Please send any improvements or ideas to me and I’ll share them here. As always, feedback and improvements to dale(at)creative-workflow-hacks(dot)com.

Compiled Binary and XCode Project files available here

final update July 15, 2006

Added a new Point of Interest expression instead of dealing with the Auto-Orient dialog as detailed in the comments.

second update July 14, 2006

Chris Prosser suggested a decent workaround of using app.executeCommand(app.findMenuCommandId(”Auto-Orient…”)) to auto-launch the Auto Orient dialog. Which in my opinion makes the script pretty usable, even with the slider range problem. Right now, I’m launching a dialog that reads “Auto Camera Rig” needs to have Auto-Orient set to “Off”, please set it in the following dialog and then immediately launching the Auto-Orient dialog. I’d be interested in feedback on whether I should leave the two dialog boxes for context or use a saveSetting() preference to give the user a Never Show Again option as well as Ok. Power users could always just comment out the alert line. Scripting UI’s are always difficult for me, I think I know how I’d handle it in a regular mac app, but I’m not so sure how to handle it here.

update July 14, 2006

Stu Maschwitz pointed out some deficiencies with the original version of this script and provided a sample project of a much better implementation of a rigged camera project. Unfortunately, there seem to be a couple of show stoppers for recreating his project via scripting and I’d love some feedback if anybody has worked through these issues, and if not I’ll send them off to aebugs@adobe.com. The first is the addCamera() method assumes a point of interest in its constructor. That seems to make the UI equivalent of layer:transform:Auto Orient:Off impossible. I’ve tried setting the property to Null, etc. with no success. Interestingly, there is a useful assistant to set the property. Secondly, I can’t seem to set the slider control range via scripting. I can set the value but not the range so this may be less of a show stopper than the first. I’ve uploaded the changes that reflect Stu’s sample project, so if you find running the script then turning Auto orient off (command-option 0, on the Mac..control-alt 0, windows? I don’t have a Windows box handy), and can live with the slider range limitation, then great. Send any fixes my way and I’ll incorporate them. I think I see why nobody took this one on .

I was doing a bit of research and ran across Kyle Sim’s request to automate his process of rigging an After Effects camera on aenhancers and the adobe forums. Seemed like a really useful script and as near as I can tell nobody has tackled it, so I decided to give it a shot.

Usage is really straight forward, copy the script to After Effects:Scripts and run it from the scripts menu with an active Comp selected. I’m not doing any layer name checking because I think it’s possible that you might want more than one camera and null, although we’d have to adjust the expressions accordingly.

When I run the script my effect window for the Mover Null looks like this and it does a great job of making the camera move really straight forward.

You’ll have to thank Kyle or whoever set up the original expressions for the idea, this just automates the creation of the rig.

Update 04.21.2007

Paul Tuersley kindly updated this script with some cool new features.

I've attached an update to your db_autoCameraRig script.
I've added a  check for AE8 which then automatically
turns of Auto-Orient.
I also  streamlined the bit that sets  the position / POI
values.

Here’s Paul’s new script, and I’ve left the original up as a legacy 1.0 version in case we get any problems out in the field.
JSX script With Paul’s Improvements
Original Source JSX script
Update 06.10.2007
Paul kindly sent me additonal updates to his modification of the script.
Paul writes…

I’ve tweaked how this version deals with the Auto-Orient issue, which
you may want to check out. In AE6.5 it puts up an alert to tell you
to turn Auto-Orient off, in AE7 it does the same and then opens the
Auto-Orient dialog and in AE8 it turns Auto-Orient off automatically.
I also added expressions to all the properties you shouldn’t alter,
effectively locking them.

Thanks Paul.

Paul’s lates modifications - JSX script in a zip archive