Most Linux system administrators write shell scripts from time to time. Often they are quick hacks to get a job done, but sometimes they linger. Yesterday’s quick hack becomes today’s “essential utility”. Of course, if we’d known that script was going to hang around, we’d have put a little more care into writing it.
This tip is really three in one: three things that are easy to include (or do) with every shell script you write. No matter whether you expect the script to be run once only or to be run multiple times every day, these three things take almost no time and will improve your scripts.
Stop On Error
When a shell script encounters an error, it will report it and carry on running. However, the system is now in an unexpected state in some way because of the failed command. Here’s a somewhat contrived example to illustrate that point. I’ve put a -v in the rm command so that when we’re testing we can see whether or not it executed; in reality, you’d probably not do that:
#!/bin/bash cp ImportantDataFile backup/$(date +'%Y-%m-%d') rm -v ImportantDataFile"
If we run that, we might find:
$ ./myscript cp: cannot create regular file 'backup/2018-05-04': No such file or directory removed 'ImportantDataFile'
We should have put some checks in to ensure the file was successfully copied, but the very least we could do it abort if an error occurs. Yes, there are much smarter ways of dealing with errors, but this is a quick hack, remember? We just want to be slightly safer. We can do that by setting -e:
#!/bin/bash set -e cp ImportantDataFile backup/$(date +'%Y-%m-%d') rm -v ImportantDataFile"
Now when we run that script and encounter an error, we immediately exit:
$ ./myscript cp: cannot create regular file 'backup/2018-05-04': No such file or directory $
Catch Unknown Variables
Here’s our next contrived example:
#!/bin/bash
FILE="ImportantDataFile"
if [[ -e $FILe ]]; then
cp -v $FILE backups/
echo "File copied"
fi
Notice that we’ve taken our finger off the shift key a little too soon in the if command (FILe rather than FILE). Again, I’ve put a -v in the cp command so that when we’re testing we can see whether or not it executed:
$ ./myscript $
No indication that anything is wrong, although because of the -v we know the file wasn’t copied. However, if we add a set -u to our script, it will treat unknown variables as errors:
#!/bin/bash set -u FILE="ImportantDataFile" if [[ -e $FILe ]]; then cp -v $FILE backups/ fi
And testing it now:
$ ./myscript ./myscript: line 7: FILe: unbound variable $
Use Both
Really, you will seldom want a script to continue after an error nor will you want to ignore an unbound variable. It takes almost no time to start shell scripts like this:
#!/bin/bash set -eu
You either love the EU or you hate it, but that makes this easy to remember.
Testing Shell Scripts
This suggestion does take a few seconds, but I’d argue they’re well-spent seconds. There’s a utility, shellcheck, which is available for most distros, and is even available online at https://www.shellcheck.net/. Here it is in action on the last version of myscript:
$ shellcheck myscript
In myscript line 7:
if [[ -e $FILe ]]; then
^-- SC2154: FILe is referenced but not assigned (did you mean 'FILE'?).
Pretty neat, isn’t it? Here’s another error that might have been hard to find:
$ shellcheck test.sh In test.sh line 3: X=$(date) ^-- SC1018: This is a unicode non-breaking space. Delete and retype it. ^-- SC1018: This is a unicode non-breaking space. Delete and retype it.
As well as finding errors such as above, it can also help improve your script writing:
$ shellcheck otherscript
In otherscript line 3:
if [ $# -ge 1 -a -f "$1" ]; then
^-- SC2166: Prefer [ p ] && [ q ] as [ p -a q ] is not well defined.
Conclusions
Quick ‘n’ dirty shell scripts aren’t going away any time soon, but if you follow the suggestions above they can remain quick but be a little less dirty.