set -ex - The most useful bash trick of the year

31 August 2014   3 comments   Linux

Powered by Fusion×

I just learned a really good bash trick which is something I've wanted to have but didn't really appreciate that it was possible so I never even searched for it.

set -ex

Ok, one thing at a time.

set -e

What this does, at the top of your bash script is that it exits as soon as any line in the bash script fails.
Suppose you have a script like this:

git pull origin master
find . | grep '\.pyc$' | xargs rm

If the first line fails you don't want the second line to execute and you don't want the third line to execute either. The naive solution is to "and" them:

git pull origin master && find . | grep '\.pyc$' | xargs rm && ./

but now it's just getting silly. (and is it even working?)

What set -e does is that it exits if any of the lines fail.

set -x

What this does is that it prints each command that is going to be executed with a little plus.
The output can look something like this:

+ rm -f pg_all.sql pg_all.sql.gz
+ pg_dumpall
+ apack pg_all.sql.gz pg_all.sql
++ date +%A
+ s3cmd put --reduced-redundancy pg_all.sql.gz s3://db-backups-peterbe/Sunday/
pg_all.sql.gz -> s3://db-backups-peterbe/Sunday/pg_all.sql.gz  [part 1 of 2, 15MB]
 15728640 of 15728640   100% in    0s    21.22 MB/s  done
pg_all.sql.gz -> s3://db-backups-peterbe/Sunday/pg_all.sql.gz  [part 2 of 2, 14MB]
 14729510 of 14729510   100% in    0s    21.50 MB/s  done
+ rm pg_all.sql pg_all.sql.gz

...when the script looks like this:

set -ex
rm -f pg_all.sql pg_all.sql.gz
pg_dumpall > pg_all.sql
apack pg_all.sql.gz pg_all.sql
s3cmd put --reduced-redundancy pg_all.sql.gz s3://db-backups-peterbe/`date +%A`/
rm pg_all.sql pg_all.sql.gz

And to combine these two gems you simply put set -ex at the top of your bash script.

Thanks @bramwelt for showing me this one.


Checkout out


Linux Ninja
useful for small scripts and debugging, but the proper way to ensure that a command only executes if the previous command succeeds is to use the && you seem to not like.

using your methodology, my script would simply die if any exit code were non-zero, but one should not rely on this. so, say I'm setting a variable to the output of a command. this would result in a return code of zero if the assignment succeeded, not just if the command's exit code were zero.

i recommend you investigate further into some bash best practices. set -e is good for debugging, but once a script is working as expected, you'll want to remove set -e in most cases and catch failures within the script to take appropriate action, not just always bail out.
The commentary on set -e behaviour is a somewhat lacking. To begin with, the concept of a failing line is nebulous. Lines don't fail but commands contained with them can. This distinction is important.

Regarding the sample script, the author implies that the goal is to exit if the git command fails, in which case the appropriate solution is this:

    git pull origin master || exit

Now, if git returns a non-zero exit status, the script will exit with the very same status. It's far more obvious what will happen from reading the script and there is no having to contend with the numerous side effects of enabling set -e (more on that below).

Next, the use of the && operator is mentioned as a possible solution:

    git pull origin master && find . | grep '\.pyc$' | xargs rm && ./

This approach has nothing in common with set -e because a failing command will not cause the script to exit. Further, the exit status of both find and grep will be disregarded, which is probably not what the author is expecting. In Bash, this deficiency can be addressed with set -o pipefail, in which case the pipeline returns the exit status of the rightmost command that failed, or zero if *all* commands succeeded. Still, in this case, a better approach would be to avoid using pipes at all:

    git pull origin master && find . -name '*.pyc' -delete && ./

For that matter, using find -delete is safer because it will cope with filenames that contain a newline character (as unlikely as that may be).

In summary, I recommend explicitly using the exit builtin if the failure of a given command should cause the script to exit. Not only does set -e result in action at a distance, but it also brings with it a surprising number of pitfalls that are only understood by experienced Bash practitioners. For more information on this topic, refer to
Peter Bengtsson
Thanks for your insightful response. Much appreciated.
Yeah, explicitness is a good thing and peppering your script with `...|| exit` gives it a chance to make some things exit on failure but allow others to fail and that's fine.

Your email will never ever be published

Related posts

Highlighted code syntax in Keynote 30 August 2014
An AngularJS directive with itself as the attribute 03 September 2014
Related by keywords:
To assert or assertEqual in Python unit testing 14 February 2009
Read in passwords with bash 25 March 2005
How I do deployments 16 December 2013
Catching a carriage return in bash 23 October 2006
ztar - my wrapper on tar -z 29 June 2005
Bash tip of the day: ff 25 March 2011
pwdf - a mix of ls and pwd 07 April 2008
File check before delete 07 March 2008
type - Writing shell scripts 28 April 2006