<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>Creative Workflow Hacks &#187; Beginner</title>
	<atom:link href="http://www.creative-workflow-hacks.com/category/beginner/feed/" rel="self" type="application/rss+xml" />
	<link>http://www.creative-workflow-hacks.com</link>
	<description>Sharing tips, scripts and hacks for your creative workflow.</description>
	<lastBuildDate>Mon, 10 May 2010 17:40:11 +0000</lastBuildDate>
	<generator>http://wordpress.org/?v=2.9.2</generator>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
			<item>
		<title>Scripting Basics: pulling random values from within a defined range in After Effects</title>
		<link>http://www.creative-workflow-hacks.com/2006/07/31/scripting-basics-pulling-random-values-from-within-a-defined-range/</link>
		<comments>http://www.creative-workflow-hacks.com/2006/07/31/scripting-basics-pulling-random-values-from-within-a-defined-range/#comments</comments>
		<pubDate>Mon, 31 Jul 2006 14:02:07 +0000</pubDate>
		<dc:creator>Dale</dc:creator>
				<category><![CDATA[After Effects ]]></category>
		<category><![CDATA[Beginner]]></category>
		<category><![CDATA[Scripting]]></category>

		<guid isPermaLink="false">http://www.creative-workflow-hacks.com/2006/07/31/scripting-basics-pulling-random-values-from-within-a-defined-range/</guid>
		<description><![CDATA[A common scripting requirement is to pull random values from within a defined range. For example, in the block dissolve transition the individual blocks of video are extracted from the video in a random order.  Let&#8217;s revisit our float away video wall script  and customize it to work in the same manner as [...]]]></description>
			<content:encoded><![CDATA[<p>A common scripting requirement is to pull random values from within a defined range. For example, in the <strong>block dissolve transition</strong> the individual blocks of video are extracted from the video in a random order.  Let&#8217;s <a href="http://www.creative-workflow-hacks.com/2006/07/09/how-to-make-a-float-away-video-wall-in-after-effects-via-scripting/">revisit our float away video wall script </a> and customize it to work in the same manner as a block dissolve but with a little more flair. We will scale the individual blocks up instead of just transitioning on and off.
</p>
<p><span id="more-40"></span></p>
<p>The key to getting random values within a defined range is to first gather our values of interest in one container and then extract them randomly from the container and delete the item after it is selected. In our <strong>block dissolve</strong>, we will scale the individual grid items based on a simple timeline keyframe, so our container can simply be an increment counter of the number of values in the grid. Like so,<br />
<code></p>
<pre>
var gridArray = new Array();
var counter = 0;

for(x = 0; x < gridWidth; x++){
      for(y= 0; y < gridHeight; y++){
          gridArray.push(++counter);
      }
}
</pre>
<p></code></p>
<p>
Later, when we need to randomize our values, we extract the numbers from the grid.
</p>
<p><code></p>
<pre>
var random = Math.floor(Math.random() * gridArray.length);
var startTime = gridArray.splice(random, 1) *  (win.speed.value * .10);
var endTime  = startTime +  (win.speed.value * .10);
scaleProp.setValueAtTime(startTime, [0,0]);
scaleProp.setValueAtTime(endTime, [100,100]);
</pre>
<p></code></p>
<p>
The <strong>splice Array method</strong> lets us extract the value from the array. The next time through we make the random values based on the <strong>length</strong> of the array, so that each time through the loop we are only pulling values for the remaining elements.
</p>
<p>When I run the script, I get results like this</p>
<div style="height: 270px">
<object width="360" height="260" codebase="http://www.apple.com/qtactivex/qtplugin.cab" classid="clsid:02BF25D5-8C17-4B23-BC80-D3488ABDDC6B"><param name="src" value="http://www.creative-workflow-hacks.com/blockDissolve/clickBlock.mov"  /><param name="target" value="myself"  /><param  name="href" value="http://www.creative-workflow-hacks.com/blockDissolve/scaledBlockDissolve.mov" /><param value="http://www.apple.com/quicktime/download/" name="pluginspage" /><embed width="360" height="260"  target = "myself" href="/blockDissolve/scaledBlockDissolve.mov" src="http://www.creative-workflow-hacks.com/blockDissolve/clickBlock.mov" pluginspage="http://www.apple.com/quicktime/download/indext.html"></embed></object>
</div>
<p>Because of the overhead of creating layers, this technique is especially useful in creating <strong>Gradient Wipes</strong> or <strong>Beauty/Matte</strong> passes. For example if we replace the above code with something like...
</p>
<p><code></p>
<pre>
var startTime = (x + y) *  (win.speed.value * .10);
var endTime = startTime +  (win.speed.value * .10);
scaleProp.setValueAtTime(startTime, [0,0]);
scaleProp.setValueAtTime(endTime, [100,100]);
</pre>
<p></code></p>
<p>We then get this.</p>
<div style="height: 270px">
<object width="360" height="260" codebase="http://www.apple.com/qtactivex/qtplugin.cab" classid="clsid:02BF25D5-8C17-4B23-BC80-D3488ABDDC6B"><param name="src" value="http://www.creative-workflow-hacks.com/blockDissolve/linearClick.mov"  /><param name="target" value="myself"  /><param  name="href" value="http://www.creative-workflow-hacks.com/blockDissolve/linearBlockDissolve.mov" /><param value="http://www.apple.com/quicktime/download/" name="pluginspage" /><embed width="360" height="260"  target = "myself" href="/blockDissolve/linearBlockDissolve.mov" src="http://www.creative-workflow-hacks.com/blockDissolve/linearClick.mov" pluginspage="http://www.apple.com/quicktime/download/indext.html"></embed></object>
</div>
<p>We can introduce some <strong>trig functions</strong> for some springy transitions, spirals etc. There are a lot of possibilities. Feel free to share some of your ideas and I'll put them up here.
</p>
<p><a href="http://www.creative-workflow-hacks.com/downloads/db_blockDissolveExperiments.jsx">Source JSX script</a></p>
]]></content:encoded>
			<wfw:commentRss>http://www.creative-workflow-hacks.com/2006/07/31/scripting-basics-pulling-random-values-from-within-a-defined-range/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
<enclosure url="http://www.creative-workflow-hacks.com/blockDissolve/clickBlock.mov" length="5789" type="video/quicktime" />
<enclosure url="http://www.creative-workflow-hacks.com/blockDissolve/scaledBlockDissolve.mov" length="44478" type="video/quicktime" />
<enclosure url="http://www.creative-workflow-hacks.com/blockDissolve/linearClick.mov" length="13010" type="video/quicktime" />
<enclosure url="http://www.creative-workflow-hacks.com/blockDissolve/linearBlockDissolve.mov" length="26579" type="video/quicktime" />
		</item>
		<item>
		<title>When you apply a filter can make a big difference</title>
		<link>http://www.creative-workflow-hacks.com/2006/06/05/when-you-apply-a-filter-can-make-a-big-difference/</link>
		<comments>http://www.creative-workflow-hacks.com/2006/06/05/when-you-apply-a-filter-can-make-a-big-difference/#comments</comments>
		<pubDate>Tue, 06 Jun 2006 02:52:48 +0000</pubDate>
		<dc:creator>Dale</dc:creator>
				<category><![CDATA[After Effects ]]></category>
		<category><![CDATA[Beginner]]></category>
		<category><![CDATA[Final Cut Pro]]></category>

		<guid isPermaLink="false">http://www.creative-workflow-hacks.com/2006/06/05/when-you-apply-a-filter-can-make-a-big-difference/</guid>
		<description><![CDATA[I was browsing around Apple&#8217;s Tech Note website when I ran into this technote. Apply the Broadcast Safe filter last in Final Cut Pro. The article discusses the importance of applying the Broadcast Safe last in a filter stack to insure that you really are limiting the colors in a sequence to broadcast safe colors.
The [...]]]></description>
			<content:encoded><![CDATA[<p>I was browsing around Apple&#8217;s Tech Note website when I ran into this technote. <a href="http://docs.info.apple.com/article.html?artnum=303801">Apply the Broadcast Safe filter last in Final Cut Pro</a>. The article discusses the importance of applying the <strong>Broadcast Safe</strong> last in a filter stack to insure that you really <strong>are</strong> limiting the colors in a sequence to broadcast safe colors.</p>
<p>The idea of a <strong>pipeline</strong> is one of the trickier ideas to master for new users of image processing applications. In <strong>image processing</strong> each new operation like a blur, composite operation or color adjustment is applied and then passed to the next operation. In the example that Apple gives above, a <strong>Proc Amp</strong> operation applied late in the pipeline could boost the color levels outside of a safe range even though <strong>Broadcast Safe</strong> filter was applied in the stack. I&#8217;ve often found when you&#8217;ve got a visually unexpected output, moving the filter arrangement will restore some sanity to the expected output. </p>
<p>Additionally, as applications like <strong>After Effects</strong> transition to 32 bit float environments you&#8217;ll see additions to pipelines like <strong>HDR Compander</strong> which can compress and decompress the pixel pipeline to work with 8 and 16bit effects.</p>
<p> It is important to get a strong conceptual understanding of core concepts like <strong>image processing pipelines</strong>. These ideas become even more important with programs moving to <a href="http://www.fxguide.com/article336.html">32 bit float</a> color and using pixel-level operations via the <a href ="http://en.wikipedia.org/wiki/Graphics_processing_unit">GPU</a> like in Apple&#8217;s <a href="http://developer.apple.com/macosx/coreimage.html">Core Image</a>. Finally, I really recommend that you keep up with what Stu Maschwitz posts at <a href="http://prolost.blogspot.com/">Prolost</a>. He writes about really hard tech like color space and image processing but with attention to form and artistry I can only aspire to.</p>
]]></content:encoded>
			<wfw:commentRss>http://www.creative-workflow-hacks.com/2006/06/05/when-you-apply-a-filter-can-make-a-big-difference/feed/</wfw:commentRss>
		<slash:comments>1</slash:comments>
		</item>
		<item>
		<title>Beginning javascript tutorial: Parsing a date from a sequenced jpeg filename</title>
		<link>http://www.creative-workflow-hacks.com/2006/05/14/beginning-javascript-tutorial-parsing-a-date-from-a-sequenced-filename/</link>
		<comments>http://www.creative-workflow-hacks.com/2006/05/14/beginning-javascript-tutorial-parsing-a-date-from-a-sequenced-filename/#comments</comments>
		<pubDate>Sun, 14 May 2006 19:03:34 +0000</pubDate>
		<dc:creator>Dale</dc:creator>
				<category><![CDATA[Beginner]]></category>
		<category><![CDATA[Scripting]]></category>

		<guid isPermaLink="false">http://www.creative-workflow-hacks.com/2006/05/14/beginning-scripting-tutorial-parsing-a-date-from-a-sequenced-filename/</guid>
		<description><![CDATA[The other day I ran across another post at aenhancers.com. (aenhancers is a really great AE scripting forum by the way). The poster, Dan wrote:

I have shot thousands of stills
that I am now wanting to make
a time lapse movie out of. Each
still is named with the date and time
it was taken. For example
0507270505011.jpg would have [...]]]></description>
			<content:encoded><![CDATA[<p>The other day I ran across another<a href="http://www.aenhancers.com/viewtopic.php?t=431"> post</a> at <a href="http://www.aenhancers.com">aenhancers.com</a>. (aenhancers is a really great AE scripting forum by the way). The poster, Dan wrote:<br />
<code></p>
<pre>I have shot thousands of stills
that I am now wanting to make
a time lapse movie out of. Each
still is named with the date and time
it was taken. For example
0507270505011.jpg would have been
taken yy/mm/dd/hh/mm/ss/(camera ID).
...
is there a way to parse the filename
to convert it into something more
legible? So 0507230004461.jpg would
translate into July 23, 2005 - 12:10 AM?...</pre>
<p></code><br />
The more I thought about it, this is exactly the kind of task I&#8217;m hoping to cover here at Creative Workflow Hacks. So in addition to posting a usable solution at aenhancers I decided to do a tutorial here with the problem solving steps involved. You know, teach a guy to fish and all.</p>
<p>I&#8217;m also going to try something new here. From the feedback I&#8217;m getting from users, the skill level and interest in DIY varies a lot. Some of you are hoping for beginners tutorials, some are looking for more advanced ideas and inspiration, and some of you are looking for solutions that are already thought through where you can download a bit of software, do your work and forget about it. So, I&#8217;m going to try labeling categories with beginner, intermediate, advanced and shrinkwrapped (although not that much software is actually shrinkwrapped anymore) depending on the skill level and the degree of involvement necessary to get the solution working. Send some feedback if you find it useful or not.</p>
<p>Back to our date parsing problem. Like almost all computer problems, the key here is to break down the big problem into a series of smaller problems.<span id="more-12"></span> Today we are going to write a <strong>function</strong> to do our bit of business and return the parsed date. A function allows us to organize these smaller tasks into one section of a program that does one <strong>function</strong>. Later, when we talk about <strong>Object Oriented Programming</strong> we will refer to a function as a <strong>method</strong>, but no matter which name is used they do similar things.</p>
<p>A <strong>function</strong> has a signature. It is all of the component parts of a function placed correctly so that the program <strong>interpreter</strong> will know what to do with the parts of the function. An <strong>interpreter</strong> is a program running behind the scenes that takes all the parts of the function and convert it it into instructions the computer and program will understand. If you type something the <strong>interpreter</strong> doesn&#8217;t understand you will get a <a title="syntax error" href="http://en.wikipedia.org/wiki/Syntax_error">syntax error</a> that indicates you made a mistake.</p>
<p>In javascript, our function signature looks something like this:</p>
<p><code></p>
<pre>
function doSomething(withThis){
         return thisResult;
}
</pre>
<p></code><br />
Javascript has it&#8217;s roots in a C style syntax. C is a programming language that has been around over 30 years, which is several centuries in programming dog years. Because it has been so popular for so long, a lot of programming languages (Java, C#, C++, Javascript, etc.) have adopted it&#8217;s syntax. Some of the notable features are the use of the <strong>( )</strong>, <strong>{ }</strong>, and <strong>;</strong> characters.</p>
<p>The ; character is used to define the end of a <strong>statement</strong>. A <strong>statement</strong> corresponds to a declarative sentence in english. You tell the <strong>interpreter</strong> something about the value of something or how something corresponds to something else.<br />
The  parentheses characters are used to group an <strong>evaluation</strong>. When we are testing for a value we use parentheses. This is called a <strong>conditional</strong>.<br />
The brace characters define a <strong>block</strong> of <strong>statements</strong>. It helps the <strong>interpreter</strong> understand the <strong>flow</strong> of your program. We&#8217;ll talk more about all of these ideas in just a bit.</p>
<p>So we&#8217;ll start with the <strong>function</strong> signature and intitial variables for our date parsing function.</p>
<p><code></p>
<pre>
var jpgName = '0507230004461.jpg';
var dateStringParsed;
dateStringParsed = parseDateString(jpgName);    

function parseDateString(jpgName){
   var evaluatedDateString = '';
   return evaluatedDateString;
}</pre>
<p></code><br />
First we set a <strong>variable</strong> for our jpeg name. A <strong>variable</strong> is a holder for our values. We can set values and manipulate values and keep them in a variable while we are doing so. Javascript is a typeless language. Meaning a variable can contain any kind of value. In this case it holds a <strong>string</strong> value of characters. Javascript has functions to convert between types like numbers and strings when we know we need a particular type of variable. We&#8217;ll go over one of these later.</p>
<p>We then set up a variable to hold our parsed date string. We <strong>declare</strong> that variable by putting the <strong>var</strong> keyword in front of it. Javascript is pretty loose about declaring variables. The <strong>var</strong> keyword helps us define the <strong>scope</strong> of the variable. We&#8217;ll cover <strong>scope</strong> in a later tutorial, but <strong>scope </strong> helps us organize whether we can use a variable anywhere from a <strong>global </strong>scope or only within a function in a <strong>local</strong> scope. Just file that away so you can say, &#8220;ahh, yes. I remember something about that&#8221;, later.</p>
<p>After we have a container variable, we <strong>call</strong> our <strong>dateParseString</strong> function so we can place the value calculated in the function in our variable. Notice how the function is on the right side of the variable? In javascript, we evaluate the right side of the <strong>statement</strong> and place that in the left side of the statement, separated by the <strong>=</strong> in this case. So, in english, we <strong>call</strong> the <strong>parseDateFunction</strong> and place the <strong>return</strong> value of the function in our <strong>variable</strong>. Notice the <strong>return</strong> keyword in our function? It indicates that we will return that value back to the function <strong>callee</strong> which in this case is our <strong>dateStringParsed</strong> variable. We also set up our <strong>dateString</strong> variable that we will calculate at this point.</p>
<p>We now have the function skeleton setup. Let&#8217;s do the calculating. We&#8217;ll start with gathering some initial values from the variable <strong>jpgName</strong><br />
<code> </p>
<pre>
var jpgName = '0507230004461.jpg';
var dateStringParsed;
dateStringParsed = parseDateString(jpgName);
function parseDateString(jpgName){
     var evaluatedDateString = '';
     var year =jpgName.substring(0,2);
     var month = jpgName.substring(2,4);
     var day = jpgName.substring(4,6);
     var hour = jpgName.substring(6,8);
     var minutes =jpgName.substring(8,10);
     var seconds = jpgName.substring(10,12);            

}</pre>
<p></code><br />
Dan told us that the jpg name followed a very specific pattern. He said:<strong> Each still is named with the date and time it was taken. For example 0507270505011.jpg would have been taken yy/mm/dd/hh/mm/ss/(camera ID)</strong>. This is the kind of task a script is really great with. We figure that every 2 characters based on location in the name will represent a certain value. We can than assign values to our variables based on this knowledge. . How do we get those values? We use a built in function that javascript assigns to every string. We are going to use the <strong>substring</strong> function. The <strong>substring</strong> function allows us to call that function from a <strong>string</strong> variable and return a string that starts at the first location called and ends at the last.<strong>var year = jpgName.substring(0,2);</strong> would then return <strong>&#8216;05&#8242;</strong>. Notice how the first called value is 0? A lot of beginners get tripped up with using 0 as the first value of a container. In general, javascript containers begin with 0 as the first value, so it is important to get your head around that. Programming is a lot like math, but if you remember a bit of algebra it can seem like what you remember as functions, etc. don&#8217;t work like you remember them. As you get familiar with programming it will seem less strange.</p>
<p>Let&#8217;s move on. We&#8217;ve assigned variables from our parsed jpeg and now we need to format them into a useful date. Here is the next part of our function:<br />
<code> </code></p>
<pre>var jpgName = '0507230004461.jpg';
var dateStringParsed;
dateStringParsed = parseDateString(jpgName);
alert(dateStringParsed);

function parseDateString(jpgName){
	var evaluatedDateString = '';
	var year =jpgName.substring(0,2);
	var month = jpgName.substring(2,4);
	var day = jpgName.substring(4,6);
	var hour = jpgName.substring(6,8);
	var minutes =jpgName.substring(8,10);
	var seconds = jpgName.substring(10,12);     

   switch (month)
   {
      case "01":
         month = 'January';
         break;
      case "02":
         month = 'February';
         break;
      case "03":
         month = 'March';
         break;
      case "04":
         month = 'April';
         break;
      case "05":
         month = 'May';
         break;
      case "06":
         month = 'June';
         break;
      case "07":
         month = 'July';
         break;
      case "08":
         month = 'August';
         break;
      case "09":
         month = 'September';
         break;
      case "10":
         month = 'October';
         break;
      case "11":
         month = 'November';
         break;
      case "12":
         month = 'December';
         break;

   }
 }</pre>
<p></code><br />
Here we use a <strong>switch</strong> statement to convert our numeric month to a month string. A <strong>switch</strong> statement is another version of an <strong>if..else</strong> conditional statement. <strong>if..else</strong> allows us to test for a condition and either match a checked value or continue on to the next statement. In some cases, a <strong>switch</strong> statment is easier to read. In checking for a month, the <strong>switch</strong> statement allows us to stack the month variables one on top of each other and at first glance you can tell what we're trying to do.</p>
<p>In programming, it is almost as important to be able to read the program easily as it is for the function to work correctly. This is so when another programmer reads your program it is clear what you were trying to accomplish. Even if you don't work with other programmers, if you try to read your own code a few months from now you'll wonder what the guy or gal who wrote this code was thinking about. It's a weird thing the first time you read your own code and wonder who wrote this ****. Another important component of understanding code is to use comments. Comments allow you to set up the scenario for your code. We'll go over comments in a future article.</p>
<p>There is one tricky part in our <strong>switch</strong> statement. Notice how we check for '01', '02'? If we test for 1, 2, etc. the test will fail. Why is that? Remember how we talked about types earlier. This is a case where we are checking for a <strong>string</strong> value and not an <strong>integer</strong>. Most of the time that flexibility is useful, but if you find yourself not getting the values you expected, one of the first things to check is some kind of type mismatch.</p>
<p>The next part of our function converts our two digit date into a full date string. Remember the Y2K bug? We recreate it right here in our function. In our case we'll assume the date is post 2000, but that is a big assumption in large bulky systems. Luckily for us, this isn't a large bulky system. Btw, the <strong>//assume 2000+ for year</strong> is an example of a comment, nice of us to tell them we are going to break their system isn't it?</p>
<p><code> </p>
<pre>
//assume 2000+ for year
year = "20" + year;
</pre>
<p></code><br />
The last part of our function is a really great example of using <strong> conditionals</strong>. We check and format our minutes variable. We also check and format to <strong>AM</strong> or <strong>PM</strong> based on what we know about the date. This is also a very simple example of an <a title="algorithm definition" href="http://en.wikipedia.org/wiki/Algorithm">algorithm</a>. If we were going to be fancy, it's also a  <a href="http://en.wikipedia.org/wiki/Heuristic_%28computer_science%29">heuristic algorithm</a>, because we take what we know about the problem and work out a system that returns the correct value without relying on a <a href="http://en.wikipedia.org/wiki/Mathematical_proof">mathematical proof</a> to generate that value. In scripting, we do this a lot. We find a way to get the values we are interested in without concerning our self too much about the theory behind it. Of course that opens ourself up into outlying cases that can produce bugs and it certainly is better to understand the whole problem if it is of a complex nature. In this case we'll just get our work done.</p>
<p>The <strong>minutes</strong> and <strong>AM/PM</strong> logic:</p>
<p><code> </p>
<pre>if(minutes > 0){
      minuteFormat = ":" + minutes;
   }else{
      minuteFormat = ":00";
   }   

   if(hour == 0){
      timeFormat = "12" + minuteFormat + " AM";
   }else if(hour < 12){

      timeFormat = hour + minuteFormat + " AM";

   }else if(hour > 12 &#038;&#038; hour != 24){
      hour = hour - 12;

      timeFormat = hour + minuteFormat + " PM";
   }else if(hour == 12){
      timeFormat = hour + minuteFormat + " PM";
   }else if(hour == 24){
      timeFormat = "12" + minuteFormat + " AM";
   }
</pre>
<p></code><br />
Notice these two conditional checks, <strong>if(minutes > 0){ </strong> and <strong>if(hour == 0){</strong>. These are examples of where javascript is doing it's own <a title="type conversion" href="http://en.wikipedia.org/wiki/Type_conversion">type coercion</a>. Type coercion is when the interpreter makes an assumption about the types necessary to perform a task. Since we are checking for greater than value for example, it makes the assumption that we want to compare the <strong>string</strong> as an <strong>integer</strong>. Most of the time this works, but if we wanted to explicitly change the value to a number, we could use a built in javascript function called <strong>parseInt()</strong> to give the interpreter the information that this was <strong>definitely</strong> an <strong>integer</strong><br />
This is also where things sort of look like the math you remember, "hey, I know <strong>></strong> means "greater than", but what's with the <strong>!=</strong> and <strong>==</strong>?". When we do a comparison using <strong>if..else</strong> we use <strong>comparison operators</strong> to do that comparison. The <strong>==</strong> characters indicate we are asking does the value of the left side <strong>equal</strong> the value of the right side. If we used <strong>!=</strong> we are asking the interpreter does the value on the left <strong>not equal</strong> the value on the right.<br />
When we use <strong>&#038;&#038;</strong> above, we are using a <strong>logical operator</strong>. When we write, <strong>else if(hour > 12 &#038;&#038; hour != 24){</strong> we are asking the interpreter to evaluate whether <strong>hour is greater than 12 AND hour is not equal to 24</strong>. The <strong>&#038;&#038;</strong> is a logical operator that means <strong>AND</strong>. Javascript also has the <strong>|| OR</strong>  logical operator and the <strong>! NOT</strong> logical operator.<br />
If I run our final function from within a browser, <strong>Flash</strong> (change the <strong>alert(dateStringParsed);</strong>to <strong>trace(dateStringParsed);</strong>), or the scripts menu in <strong>After Effects</strong>:</p>
<p><code></p>
<pre>var jpgName = '0507230004461.jpg';
var dateStringParsed;
dateStringParsed = parseDateString(jpgName);
alert(dateStringParsed);

function parseDateString(jpgName){
	var evaluatedDateString = '';
	var year =jpgName.substring(0,2);
	var month = jpgName.substring(2,4);
	var day = jpgName.substring(4,6);
	var hour = jpgName.substring(6,8);
	var minutes =jpgName.substring(8,10);
	var seconds = jpgName.substring(10,12);     

   switch (month)
   {
      case "01":
         month = 'January';
         break;
      case "02":
         month = 'February';
         break;
      case "03":
         month = 'March';
         break;
      case "04":
         month = 'April';
         break;
      case "05":
         month = 'May';
         break;
      case "06":
         month = 'June';
         break;
      case "07":
         month = 'July';
         break;
      case "08":
         month = 'August';
         break;
      case "09":
         month = 'September';
         break;
      case "10":
         month = 'October';
         break;
      case "11":
         month = 'November';
         break;
      case "12":
         month = 'December';
         break;

   }

   //assume 2000+ for year

   year = "20" + year;

   if(minutes > 0){
      minuteFormat = ":" + minutes;
   }else{
      minuteFormat = ":00";
   }   

   if(hour == 0){
      timeFormat = "12" + minuteFormat + " AM";
   }else if(hour < 12){

      timeFormat = hour + minuteFormat + " AM";

   }else if(hour > 12 &#038;&#038; hour != 24){
      hour = hour - 12;

      timeFormat = hour + minuteFormat + " PM";
   }else if(hour == 12){
      timeFormat = hour + minuteFormat + " PM";
   }else if(hour == 24){
      timeFormat = "12" + minuteFormat + " AM";
   }

   evaluatedDateString = month + " " + day + ", " + year + "  - " + timeFormat;

   return evaluatedDateString;
} </pre>
<p></code><br />
I get <strong>July 23, 2005 - 12:04 AM</strong>, which seems to pass the test of what we are trying to get our script to do.If this is an actual production script I would encourage you to try a bunch of different values and also to check and return for invalid values so that you don't break other parts of your system. I've got tutorials on user and unit testing as well as debugging coming up.</p>
<p>Don't be intimidated by scripting. Dive right in. The worst thing that can happen is you might be confused for a bit, but by being diligent and doing some tutorials you'll get useful stuff happening before too long. I especially wouldn't worry too much about the more heavy computer science ideas. While I don't question the importance of the theory behind the math and programming you might use, it tends to make us feel like we have to ask permission to try things out and that's simply not true.</p>
]]></content:encoded>
			<wfw:commentRss>http://www.creative-workflow-hacks.com/2006/05/14/beginning-javascript-tutorial-parsing-a-date-from-a-sequenced-filename/feed/</wfw:commentRss>
		<slash:comments>2</slash:comments>
		</item>
		<item>
		<title>Analyzing access logs to find patterns, or &#8220;Just how many bots are out there anyway?&#8221;: In which we give a gentle introduction to command-line tools.</title>
		<link>http://www.creative-workflow-hacks.com/2006/05/01/analyzing-access-logs-to-find-patterns-or-just-how-many-bots-are-out-there-anyway-in-which-we-give-a-gentle-introduction-to-command-line-tools/</link>
		<comments>http://www.creative-workflow-hacks.com/2006/05/01/analyzing-access-logs-to-find-patterns-or-just-how-many-bots-are-out-there-anyway-in-which-we-give-a-gentle-introduction-to-command-line-tools/#comments</comments>
		<pubDate>Mon, 01 May 2006 14:05:39 +0000</pubDate>
		<dc:creator>Dale</dc:creator>
				<category><![CDATA[Beginner]]></category>
		<category><![CDATA[Scripting]]></category>
		<category><![CDATA[System Administration]]></category>

		<guid isPermaLink="false">http://www.creative-workflow-hacks.com/2006/05/01/analyzing-access-logs-to-find-patterns-or-just-how-many-bots-are-out-there-anyway-in-which-we-give-a-gentle-introduction-to-command-line-tools/</guid>
		<description><![CDATA[You find the most interesting stuff in access logs. I spend a fair amount of time digging around in web analytic software. Looking at who is referring to us, analyzing traffic patterns, figuring out peak usage times&#8230;you get the picture. Sometimes though, you just need to get right down into the raw logs to figure [...]]]></description>
			<content:encoded><![CDATA[<p>You find the most interesting stuff in access logs. I spend a fair amount of time digging around in web analytic software. Looking at who is referring to us, analyzing traffic patterns, figuring out peak usage times&#8230;you get the picture. Sometimes though, you just need to get right down into the raw logs to figure out what&#8217;s going on, and that situation happened to me a while back.</p>
<p>Around the middle of last year a <a href="http://en.wikipedia.org/wiki/Exploit_%28computer_security%29">security exploit</a> was found in the <a href="http://news.netcraft.com/archives/2005/07/04/php_blogging_apps_vulnerable_to_xmlrpc_exploits.html">implementation of the XML-RPC protocol in PHP</a>. PHP is obviously a very popular web scripting language and dozens of content management systems, including the blogging software I use for this blog, utilize the language. Now an open security hole in such a popular language is just an invitation for the underground to attempt to exploit the situation, and try to exploit they did.</p>
<p>It started out as just a trickle, but before long it became obvious that the underground was attempting to exploit the security hole. How&#8217;d we know? By analyzing our logs.</p>
<p><span id="more-8"></span></p>
<p>An access log is generated by your web server, in our case Apache, and contains a lot of information. For example, here is a random line from this blog.<br />
<code></p>
<pre>
208.52.128.170 - - [28/Apr/2006:11:07:53 -0700] "GET /wp-content/themes/rdc/style.css HTTP/1.1" 200 7581 "http://www.creative-workflow-hacks.com/" "Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-us) AppleWebKit/418 (KHTML, like Gecko) Safari/417.9.2"
</pre>
<p></code></p>
<p> The first column is where the ip request comes from. The second is the date. The third is the url and the GET means it uses a GET style HTTP request. The fourth is the user agent, in this case Safari. So that&#8217;s all well and good, but what&#8217;s that have to do with the security exploit?</p>
<p>Well, by looking at your raw access logs, you start to notice patterns. Something like this is what we started to see in our access logs.</p>
<p><code></p>
<pre>
209.213.106.25 - - [26/Apr/2006:05:46:19 -0400] "GET /xmlrpc.php HTTP/1.0" 404 1453
</code>
</pre>
<p>and</p>
<p><code></p>
<pre>
209.213.106.25 - - [26/Apr/2006:05:46:20 -0400] "GET /drupal/xmlrpc.php HTTP/1.0" 404 1453
</code>
</pre>
<p></p>
<p>Drupal? We're not using Drupal. What was happening were attempts to exploit the security hole. And there were quite a few attempts and they kept coming.
</p>
<p>
So was this some lone hacker typing into her browser? Not likely. What was happening was bots. </p>
<p>
You've probably heard about spyware. In addition to sending you all kinds of weird popups and putting casino shortcuts on your desktop, malware like this can take over your computer and do things like send spam email, or in this case attempt to send security exploits to take over yet more computers and send spam email and then attempt...well, you get the picture.
</p>
<p>So by analyzing our logs we start to get a picture of what to watch out for and figure out what software *really* needs to be up to date. Better get over to <a href="www.wordpress.org">wordpress</a> and check for patches</p>
<p>What I want to show today is one little tool that I used to analyze our logs while this exploit was going on full tilt, although we still see a lot of requests for xmlrpc as the last laggards still haven't updated their software and the law of diminishing returns kicks in for the bots</p>
<p> We're going to use the command line today (it's really not that scary, you might even end up with 10 different shells open on your deskop one day like a real unix grey beard...don't laugh, it could happen).It goes without saying that the following will only work if your webserver is running apache and you have unix shell acess to the server your log files are kept. The first order of business is to find out where your log files are kept. A common location is <strong>/var/log/httpd</strong> or if your on a shared host, a <strong>logs</strong> directory in your <strong>home</strong> directory. Once you've figured out where your log file is located, let's <strong>cd</strong> to that directory.</p>
<p>Unix tools are designed to do one thing very well and work in conjunction with other small tools that do one thing well to accomplish the task at hand. Unix tools have a concept called <strong>standard input</strong> and <strong>standard ouput</strong> which allow each tool to do their bit of business and then pass the result on to the next tool. The command we are going to use today is:</p>
<p><code></p>
<pre>
cat access.log | grep xmlrpc |  awk '{print $1}' | sort | uniq | wc -l
</pre>
<p></code></p>
<p>
Scary huh? But don't freak out. We'll go through each part and you'll start to understand  that it really is easy and you might be able to do some useful work with this command line stuff any how.
</p>
<p>So let's start off with the first part. Commands start from the left like you're reading a sentence. <strong>cat access_log</strong> catalogs or writes out the acess_log to <strong>standard output</strong> which in this case is the terminal in which you typed the command. Your version of your <strong>access_log</strong> might be named something else like <strong>access.log</strong> or somesuch, but you should be able to figure it out. You could also use <strong>tail</strong> command if you had an access_log that is several days long and you only want to check out just the last bit of access.</p>
<p>
Use <strong> tail -n 100 </strong> to grab the last 100 entries. Which brings us to <strong>Unix command switches</strong>, the <strong> -n 100</strong> is a command switch. Most unix commands let you alter the base behavior of the command with a switch. How do you know which switches to use? You use the <strong>man</strong> command. Type <strong>man tail</strong> and see what you get.
</p>
<p>
A lot of the stuff in a <strong>man</strong> can seem hopelessly geeky at first, but with a little perserverance we can find what we need. So we've written out our <strong>access_log</strong> file. This is the raw data that comes from the web server. Take a look around, it's pretty interesting even before you've run analytics on it, but for most folks it's just too overwhelming to find anything useful. Let's take a look at the next part of our command.
</p>
<p><code></p>
<pre>
cat access_log | grep xmlrpc
</pre>
<p></code></p>
<p>
the <strong>|</strong> is the pipe indicator. We talked earlier about <strong>standard input</strong> and <strong>standard output</strong>. The <strong>|</strong> lets us <strong>pipe</strong> the output of <strong>cat</strong> into the input of <strong>grep</strong>. If we wanted to write out our acess_log to a file we would use the <strong>></strong> character.<br />
<code></p>
<pre>
tail -n 100 access_log > ourfile.txt
</pre>
<p></code><br />
would write a file called ourfile.txt with the last 100 lines of our access_log.
</p>
<p>
Back to our command. We pipe the output of our <strong>access_log</strong> to something called <strong>grep</strong>. What is <strong>grep</strong>? <a href="http://en.wikipedia.org/wiki/Grep">Grep</a> stands for "<strong>g</strong>eneralized <strong>r</strong>egular <strong>e</strong>xpression <strong>p</strong>rocessor". Grep allows us to search for patterns in our output called regular expressions. Think of a regular expression as a find and replace on steroids. Regular expressions can get really complicated, but in our command we're just looking for xmlrpc. It won't match on variations with upper case or with dashes, etc., but it's surprisingly effective here.
</p>
<p>Let's move on to<br />
<code></p>
<pre>
cat access_log | grep xmlrpc | awk '{print $1}'
</pre>
<p></code><br />
We now pipe any matches on xmlrpc to awk. <strong>awk</strong> is a general purpose language that is designed to process text data. I don't use <strong>awk</strong> a lot, but it is really handy to grab a specific column from text output. That's what we do here. <strong> awk'{print $1}'</strong> takes only the first column and prints that to standard output. So from the first three parts of our command, we've printed the ip address of every access on our web server that used the word xmlrpc. Pretty neat. Let's move on.
</p>
<p><code></p>
<pre>
cat access_log | grep xmlrpc | awk '{print $1}' | sort
</pre>
<p></code></p>
<p>
All of those ip addresses are all over the place. Let's sort them into something useful. We first sort them numerically with <strong>sort</strong>.We use the <strong>uniq</strong> command to grab only the unique ip's. These bots might hit us once or a bunch of times, but we are only interested in grabbing the unique ones.
</p>
<p><code></p>
<pre>
cat access_log | grep xmlrpc | awk '{print $1}' | sort | uniq
</pre>
<p></code></p>
<p>If we stop here we get a sorted, unique ip list of requests for xmlrpc. If we had a blackhole list we could add these ip's and not accept requests from these ip's in the future when the next exploit becomes available. We do need to be careful with that though because your most important customers might have a virus or spyware and not be able to access your site or your grep search string might find ip's that are accessing your site legitimately, use your judgement.
</p>
<p>
And finally</p>
<p><code></p>
<pre>
cat access_log | grep xmlrpc | awk '{print $1}' | uniq | sort | wc -l
</pre>
<p></code></p>
<p>let's use the word count command <strong>wc</strong> command with the <strong>-l</strong> switch to count the number of lines. 137. Wow. During the worst of our xmlrpc weirdness we were getting 137 unique ip's trying to exploit our web server.Definitely time to check for patches.
</p>
<p>
So that's it. This was definitely a long-winded post, but I hope you find it useful especially if your new to the command line. Like most things computer oriented, try to break down your big goal into smaller pieces until you work out the combinations that get you were you need to go.</p>
]]></content:encoded>
			<wfw:commentRss>http://www.creative-workflow-hacks.com/2006/05/01/analyzing-access-logs-to-find-patterns-or-just-how-many-bots-are-out-there-anyway-in-which-we-give-a-gentle-introduction-to-command-line-tools/feed/</wfw:commentRss>
		<slash:comments>2</slash:comments>
		</item>
		<item>
		<title>Diagnosing an OSX slowdown</title>
		<link>http://www.creative-workflow-hacks.com/2006/03/30/diagnosing-an-osx-slowdown/</link>
		<comments>http://www.creative-workflow-hacks.com/2006/03/30/diagnosing-an-osx-slowdown/#comments</comments>
		<pubDate>Thu, 30 Mar 2006 21:44:08 +0000</pubDate>
		<dc:creator>Dale</dc:creator>
				<category><![CDATA[Beginner]]></category>
		<category><![CDATA[OSX]]></category>
		<category><![CDATA[System Administration]]></category>

		<guid isPermaLink="false">http://www.creative-workflow-hacks.com/2006/03/30/debugging-an-osx-slowdown/</guid>
		<description><![CDATA[
Most of the hacks I post here are about getting things done. Today I&#8217;m going to write about avoiding obstacles that might stop you from getting things done. Namely, I&#8217;m going to case-study debugging a sluggish OSX machine.


THE SCENARIO


My boss approached me about a weird slowdown he was having where his machine was slowing to [...]]]></description>
			<content:encoded><![CDATA[<p>
Most of the hacks I post here are about getting things done. Today I&#8217;m going to write about avoiding obstacles that might stop you from getting things done. Namely, I&#8217;m going to case-study debugging a sluggish OSX machine.
</p>
<h3>
THE SCENARIO<br />
</h3>
<p>
My <a href="http://www.recharge06.com/speakers/doug-grimmett.php">boss</a> approached me about a weird slowdown he was having where his machine was slowing to a crawl. Trying to move the cursor was taking a lifetime to move a couple of inches. There wasn&#8217;t a beachball cursor indicating an application was sucking up resources and I was able to launch programs, just very S L O W L Y.
</p>
<p>
So, what did we do?
</p>
<p><span id="more-5"></span></p>
<p>
I&#8217;ve done this stuff for a while and you start to get a sense of what seems like the usual suspects when you&#8217;re debugging an OSX problem. I&#8217;m going to run through the process I use and tell you about what we found out.
</p>
<h3>
THE PROCESS<br />
</h3>
<p>
Step One.
</p>
<p>
We back up a <em>lot</em> of data at <a href="http://www.primalscreen.com">Primal Screen</a>. We do both SD and HD video and have about 10 Terabytes of online storage and 5 Terabytes for near-line archive storage, combine that with individual client machine backups, and we, unfortunately, end up backing up during business hours.  So when I hear about a mid-day slowdown, the first thing my &#8220;spidey-sense&#8221; goes off about is back-up. We use Retrospect from a central server so I did a quick check to see if it was running the client backup on Doug&#8217;s machine. Nope.
</p>
<p>
On to step two.
</p>
<p>
I also knew that Doug had recently installed Tiger(OSX 10.4) on his laptop. We are relatively late adopters on software for production machines and we&#8217;ve had enough problems with Tiger that it still isn&#8217;t universally adopted. One of the &#8220;features&#8221; we struggle with is <a href="http://www.apple.com/support/mac101/work/22/">Spotlight</a>. Spotlight is a great idea. It indexes your data and allows for quick context sensitive searches, but it&#8217;s definitely a version 1.0 technology. The interface is a bit wonky and you can sometimes run into the next problem I suspected, Spotlight indexing. Spotlight needs to &#8220;index&#8221; your computer to gather all of the information that it uses. Indexing means that spotlight goes through the folders on drives that are searched by Spotlight and gathers the metadata from your files so that you can search it. While this is happening your machine can slow down and become unresponsive. So that&#8217;s the next place we look.
</p>
<p>
If you open <strong>Applications/Utilities/Activity Monitor</strong> (or for the geeky who like the command line, via <strong>Applications/Utilities/terminal top -o cpu</strong>), the process <strong>mds</strong> is active when Spotlight is either indexing or searching. The <strong>mdimport</strong> process is also active during indexing. If we were having Spotlight problems, those processes would be active and using excess CPU cycles. Nope, in this case.
</p>
<p>
If this turns out to be your problem, a great resource is available at <a href="http://www.thexlab.com/faqs/stopspotlightindex.html">thexlab.com</a> on various ways of stopping and altering Spotlight indexing.
</p>
<p>
This is also the step where we would look for other programs taking too many CPU cycles. Sometimes you&#8217;ll have a spinning beachball cursor and not be sure of the source. Here&#8217;s where you&#8217;d check that.
</p>
<p>
On to step three.
</p>
<h3>
THE SOLUTION<br />
</h3>
<p>
We now take a stop at <strong>Applications/Utilities/Console</strong>. The console is where applications send data about their current state. You&#8217;ll find errors, status and crash information here. The GUI hides a lot of interesting information from you. Some of it is a bit geeky and extraneous but you can pick up a lot of info and understanding about your computer by checking out the console.
</p>
<p>
So, we open it and I notice these entries&#8230;.
</p>
<pre>
SoundGrinderCMPlugInFactory-I-Debug
AllocCMPlugInType-I-Debug
SoundGrinderCMPlugInQueryInterface-I-Debug
SoundGrinderCMPlugInAddRef-I-Debug
SoundGrinderCMPlugInRelease-I-Debug
SoundGrinderCMPlugInExamineContext-I-Debug
SoundGrinderCMPlugInPostMenuCleanup-I-Debug
</pre>
<p>
This looks promising&#8230;
</p>
<p>
hmmm, a CMPlugin is throwing debug info. A CM Plugin is a Contextual Menu plugin. Contextual Menus appear when you Control-click or Right-click (yeah, even macs work with two button mice these days) in a &#8220;context&#8221; that allows their use. Ok, but what is Sound Grinder?
</p>
<p>
Ahh, yes. <a href="http://www.monkey-tools.com/pages/products/sgmain.htm">Sound Grinder</a> is an audio conversion utility we were checking out to convert some audio clips. Doug had removed the program when he decided to use a different app. But why was I still getting debug info? Sure enough, in <strong>/Library/Contextual Menu Items/</strong> a <strong>SoundGrinderCMPlugIn.plugin</strong> file.
</p>
<p>
So, we&#8217;re definitely suspicious of this file. Let&#8217;s remove it. Another quick search for <a href="http://monkey-tools.com/phpBB/viewtopic.php?t=22">&#8220;uninstalling Sound Grinder&#8221;</a> revealed:
</p>
<pre>
To uninstall Sound Grinder you can simply drag the Sound Grinder folder to
the trash, and trash the plugin located at: <hard
drive>/Library/Contextual Menu Items/SoundGrinderCMPlugIn.plugin. If you
wish to delete the preferences, look in the preference folder for a file
called "com.monkeytools.soundgrinder.plist", and drag to the trash.
</pre>
<p></p>
<p>
We removed the plugin and that cleared up the problem. BTW, Sound Grinder seems like a very nice program and I&#8217;m not sure exactly why I was having the slowdown problems that we found related to the CMPlugin, although it probably didn&#8217;t help that we had yanked most of the rest of the program from the expected location.
</p>
<p>
I do think a nice universal OSX uninstall utility for programs that install things outside of the application bundle would be a good thing. I&#8217;ve got bits and pieces of no longer used detritus all over the place.
</p>
<h3>
REFERENCE<br />
</h3>
<p>
So, in this case we caught our problem in the triage stage. Sometimes, you don&#8217;t find your problem here though. Where to next? I&#8217;ve had great success with Jaguar, Panther and Tiger <a href="http://www.northernsoftworks.com/tigercachecleaner.html">Cache Cleaner</a> from Northern Softworks. It runs a bunch of standard maintenance and utility scripts that tend to ferret out corrupt caches and permissions problems. For disk problems, I recommend <a href="http://www.amazon.com/exec/obidos/redirect?link_code=ur2&#038;tag=creativeworkf-20&#038;camp=1789&#038;creative=9325&#038;path=tg%2Fdetail%2F-%2FB000095YXH%2Fsr%3D8-1%2Fqid%3D1143759946%2Fref%3Dpd_bbs_1%3F%255Fencoding%3DUTF8%26v%3Dglance">DiskWarrior</a><img src="http://www.assoc-amazon.com/e/ir?t=creativeworkf-20&#038;l=ur2&#038;o=1" width="1" height="1" border="0" alt="" style="border:none !important;margin:0px !important;" /> and avoid <a href="http://www.amazon.com/exec/obidos/redirect?link_code=ur2&#038;tag=creativeworkf-20&#038;camp=1789&#038;creative=9325&#038;path=tg%2Fdetail%2F-%2FB0001CJEHC%2Fsr%3D8-1%2Fqid%3D1143760009%2Fref%3Dpd_bbs_1%3F%255Fencoding%3DUTF8%26v%3Dglance">Norton</a><img src="http://www.assoc-amazon.com/e/ir?t=creativeworkf-20&#038;l=ur2&#038;o=1"width="1" height="1" border="0" alt="" style="border:none !important;margin:0px !important;" /> (mainly for it&#8217;s insistence in leaving bits and pieces of itself everywhere, go figure). I&#8217;ve also been greatly assisted by John Gruber and his <a href="http://daringfireball.net">Daring Fireball</a> site. His rundown of the <a href="http://daringfireball.net/2004/12/software_update">Software Update</a> process is a great piece of info and his <a href="http://daringfireball.net/2005/03/font_caches#fn3-2005-03-21"> font cache</a> problem exposé is at least vaguely the inspiration for this post. </p>
<p><p>
Hope you&#8217;ve find this helpful, and here&#8217;s hoping your avoiding these gotcha problems in your workflow.</p>
]]></content:encoded>
			<wfw:commentRss>http://www.creative-workflow-hacks.com/2006/03/30/diagnosing-an-osx-slowdown/feed/</wfw:commentRss>
		<slash:comments>3</slash:comments>
		</item>
	</channel>
</rss>
