Extracting and Organizing with Bash

Long week? Yeah, me too. I have my heavy metal Linux band in the motel room and no customers to attend to at the moment…let’s do some Bash scripting! Remember the “thumbnailing” script I did a few weeks ago? This is the script I use before doing thumbnails. It’s actually a bit more trippy and uses another script which I’ll cover in the next installment.

I begin by getting a temporary file setup:

#!/bin/bash
set -e
DateFile="/tmp/image-dates.$$"
[ -z "$DateFile" ] && echo "no datefile, bye." && exit 1
[ -f "$DateFile" ] && rm -f $DateFile
touch "$DateFile"

We have a guard: did we screw up our file name? That’s almost irrationally cautious, so you might feel justified in deleting that guard. The $$ symbol is a quick way to get your process ID (PID). You can use this for many things, but it is reasonably unique with low-frequency usage (that is, a few times a day).

You can only have 65535 process IDs, and there are 86400 seconds a day, so if you approach one process a second, you’ll get repeats. If we find an identical date file, let’s delete it anyhow and create a new one. (POP QUIZ: tell me a one-liner that does this.)

Turn that metal up! We’re about to “headbang” through a crazy lead solo of string manipulation:

ls *.jpg *.dng *.bmp *.tiff *.jpeg \
*.JPG *.DNG *.BMP *.TIFF *.JPEG 2>/dev/null \
| while read F
do
G=$( echo -n "$F" | perl -pe 'y/[A-Z]/[a-z]/' )
cmd="mv -v $F $G"
$cmd
done

Yeah! Let’s crush some upper-case file names! Why the 2>/dev/null though? Have you ever done an “ls * .txt” command just to get everything in your directory listed and followed by a warning: .txt: file not found ? Yeah. That error message, or those TIFF or JPEG files you probably won’t have will all give you that message. It’s a safe message to discard into /dev/null.

We’re taking each file name F, and piping it into perl and feeding it into the Sarlacc jaws of the y/// translator. This y/// operator is the perl equivalent to the ‘tr’ command: it translates one character pattern to another character pattern. We’re forcing all upper-case alphabetic characters into lower case characters. Why? Type in caps much? (The bassist does, the bastard.) Extra credit: give me the shell command that replaces that perl command.

Now G is the transformed name, and we just move the file from the original F value to the new G value. (POP QUIZ: Why, oh why would I assemble a string of this command? Could this be evidence of an erased echo statement?) Extra Credit: make this command safer for filenames with spaces in them.

Drum solo! We’re going to rattle out some dates with another wee utility called ImgDate.sh:

ls *.{jpg,dng,bmp,tiff,jpeg,gif,png} 2>/dev/null \
| while read F
do
echo -n "$F " >> $DateFile
~/bin/ImgDate.sh $F >> $DateFile
done

Remember that the >> operator appends to files and echo -n prints stuff without a newline. Pay attention to spaces. We’re immediately going to use this output in a loop:

cat $DateFile | sort | uniq \
| while read D
do
echo "D $D"
imgfile=`echo "$D" | awk '{print $1}'`
imgdir=`echo "$D" | awk '{print $NF}'`
if [ ! -d "$imgdir" ]
then
mkdir -v $imgdir
fi
mv -v $imgfile $imgdir
done

What, another external program…awk? Isn’t that the sound your drummer makes after the fifth shot of Jagermeister? Hell yeah. Awk loves to toss out column values, so $1 is anything in the first column. What did we put in the first column of DateFile? Oh yeah…the drummer, no, the image file name. The last column is always $NF. Pop Quiz: Why not $2 in the second column? Warning: Don’t ask where why the drummer is called NF, he’s shy. Extra Credit #2: what if the file name has spaces in it?

Now we need to clean the floor of our motel room: /tmp. Let’s wipe up $DateFile.

rm -f $DateFile

I’m going to drag the drummer off to the tour bus, but I’m going to leave you with one more puzzler: If I delete $DateFile at the end of this party, why do I bother deleting it at the start of the party as well? Think about it…

Jed Reynolds

Jed Reynolds has been known to void warranties and super glue his fingers together. When he is not doing photography or fixing his bike, he can be found being a grey beard programmer analyst for Candela Technologies. Start stalking him at https://about.me/jed_reynolds.