a little madness

A man needs a little madness, or else he never dares cut the rope and be free -Nikos Kazantzakis


Archive for the ‘Bash’ Category

Bash Tip: Reliable Clean Up With Trap

Wow, it has been over 6 years since my last bash tip. Bash scripts are still my go-to for automating the mundane and repetitive — it might not be beautiful but bash is feature rich and installed everywhere. Although a lot of my scripts are throw-away, others make it into the permanent tool kit. For scripts in the latter category, it’s worth the extra bit of effort to make them more robust.

A key part of a robust script is detecting and sensibly reporting errors. Related to this is proper clean up of work in progress when the script encounters a fatal error. Such clean up is often overlooked, as it is difficult to get right in a naive fashion. Thankfully, bash has a built-in mechanism that makes it simple: trap.

Setting a trap instructs bash to run a command when the shell process receives a specified signal. Further, bash defines pseudo-signals ERR and EXIT that can be used to trap any error or exit of the shell. I usually pair an exit trap with a clean up function to ensure my scripts don’t leave a mess. If the script runs successfully, I reset the trap (by specifying – as the command to run) and call the clean up function directly (if required). For example:

#! /usr/bin/env bash

set -e

# Defines a working area on the file system.

function cleanUp() {
    if [[ -d "$SCRATCH" ]]
        rm -r "$SCRATCH"

trap cleanUp EXIT
mkdir "$SCRATCH"

# Actual work here, all temp files created under $SCRATCH.

# We succeeded, reset trap and clean up normally.
trap - EXIT
exit 0

Note that the EXIT signal pairs well with set -e: if the script exits on error your EXIT trap is executed. You may also consider applying the trap to interrupts (signal INT) and kills (signal TERM) of your script. Multiple signals can be specified in one trap statement:

trap cleanUp EXIT INT TERM

If you’re used to programming with exceptions, you can think of a trap as a “finally” clause for your whole script. It’s certainly a whole lot easier and less error-prone that trying to figure out all the ways the script might exit and adding calls to clean up manually.

Want Continuous Integration without the Needless Duplication? Try pulse.

Bash Tip: Tracing Execution

When your bash script is going haywire, and you’re having trouble sorting out why, one of the simplest ways to see what is going on is to use:

set -x

This turns on the xtrace shell option, which prints the commands the shell is executing with expanded arguments, as they are executed. From the man page:

-x After expanding each simple command, display the
expanded value of PS4, followed by the command and its
expanded arguments.

This shows you exactly what is going on as your script executes. Take the following, contrived shell script:

#! /usr/bin/env bash

set -x

for i in $(seq 0 3)
if [[ $i -gt 1 ]]
echo $i

exit 0

When executed, the script will print the following:

$ ./example.sh
++ seq 0 3
+ [[ 0 -gt 1 ]]
+ [[ 1 -gt 1 ]]
+ [[ 2 -gt 1 ]]
+ echo 2
+ [[ 3 -gt 1 ]]
+ echo 3
+ exit 0

Note that the trace lines are prefixed with ‘+’ characters (in fact, whatever the value of $PS4 is), and regular output appears as normal. You can also see that the for loop and if conditions are displayed. This is all very handy for tracking down obscure problems.

One issue with tracing is the output can be very verbose, making the useful bits hard to find. If you have a rough idea of where the problem is, you can alleviate this problem by turning tracing on and off selectively around the interesting parts of the script (set +x turns it off). Happy scripting!

Automation crazy? Need a serious automated build server? Try pulse.

Bash Tip: Exit on Error

Back in my post Your Next Programming Language I mentioned I would post occassional tips about bash scripting. As soon as I started writing my next script, it occured to me: the first thing I always do when writing a new bash script is set the errexit option:

set -e

This option makes your script bail out when it detects an error (a command exiting with a non-zero exit code). Without this option the script will plough on, and mayhem often ensues. In all the noise generated it can be a pain to found the root cause of the problem. So I make it a rule to set this option and fail as early as possible.

Into continuous integration? Want to be? Try pulse.