Tips on good shell programming practices

What #! really does

Summary
This month Mo details the ins and outs of hash-bang (#!) and validating line commands. Read on and you'll agree: testing for arguments is simply "good programming practice." (1,400 words)

Once upon a time, Unix had only one shell, the Bourne shell, and when a script was written, the shell read the script and executed the commands. Then another shell appeared, and another. Each shell had its own syntax and some, like the C shell, were very different from the original. This meant that if a script took advantage of the features of one shell or another, it had to be run using that shell. Instead of typing:

doit

The user had to know to type:

/bin/ksh doit

or:

/bin/csh doit

To remedy this, a clever change was made to the Unix kernel -- now a script can be written beginning with a hash-bang (#!) combination on the first line, followed by a shell that executes the script. As an example, take a look at the following script, named doit:

#! /bin/ksh
#
# do some script here
#

In this example, the kernel reads in the script doit, sees the hash-bang, and continues reading the rest of the line, where it finds /bin/ksh. The kernel then starts the Korn shell with doit as an argument and feeds it the script, as if the following command had been issued:

/bin/ksh doit

When /bin/ksh begins reading in the script, it sees the hash-bang in the first line as a comment (because it starts with a hash) and ignores it. To be run, the full path to the shell is required, as the kernel does not search your PATH variable. The hash-bang handler in the kernel does more than just run an alternate shell; it actually takes the argument following the hash-bang and uses it as a command, then adds the name of the file as an argument to that command.

You could start a Perl script named doperl by using the hash-bang:

#! /bin/perl

# do some perl script here

If you begin by typing doperl, the kernel spots the hash-bang, extracts the /bin/perl command, then runs it as if you had typed:

/bin/perl doperl

There are two mechanisms in play that allow this to work. The first is the kernel interpretation of the hash-bang; the second is that Perl sees the first line as a comment and ignores it. This technique will not work for scripting languages that fail to treat lines starting with a hash as a comment; in those cases, it will most likely cause an error. You needn't limit your use of this method to running scripts either, although that is where it's most useful.

The following script, named helpme, types itself to the terminal when you enter the command helpme:

#! /bin/cat
vi     unix editor
man    manual pages
sh     Bourne Shell
ksh    Korn Shell
csh    C Shell
bash   Bourne Again Shell

This kernel trick will execute one argument after the name of the command. To hide the first line, change the file to use more by starting at line 2, but be sure to use the correct path:

#! /bin/more +2
vi     unix editor
man    manual pages
sh     Bourne Shell
ksh    Korn Shell
csh    C Shell
bash   Bourne Again Shell

Typing helpme as a command causes the kernel to convert this to:

/bin/more +2 helpme

Everything from line 2 onward is displayed:

helpme
vi     unix editor
man    manual pages
sh     Bourne Shell
ksh    Korn Shell
csh    C Shell
bash   Bourne Again Shell
etc.

You can also use this technique to create apparently useless scripts, such as a file that removes itself:

#! /bin/rm

If you named this file flagged, running it would cause the command to be issued as if you had typed:

/bin/rm flagged

You could use this in a script to indicate that you are running something, then execute the script to remove it:

#! /bin/ksh
# first refuse to run if the flagged file exists

if [-f flagged ]
then
    exit
fi

# create the flag file

echo "#! /bin/rm" >flagged
chmod a+x flagged

# do some logic here

# unflag the process by executing the flag file

flagged

Before you begin building long commands with this technique, keep in mind that systems often have an upper limit (typically 32 characters) on the length of the code in the #! line.

Testing command line arguments and usage
When you write a shell script, arguments are commonly needed for it to function properly. In order to ensure that those arguments make sense, it's often necessary to validate them.

Testing for enough arguments is the easiest method of validation. For example, if you've created a shell script that requires two file names to operate, test for at least two arguments on the command line. To do this in the Bourne and Korn shells, check the value of $# -- a variable that contains the count of arguments, other than the command itself. It is also good practice to include a message detailing the reasons why the command failed; this is usually created in a usage function.

The script twofiles below tests for two arguments on the command line:

#! /bin/ksh

# twofile script handles two files named on the command line

#  a usage function to display help for the hapless user

usage ()
{
     echo "twofiles"
     echo      "usage: twofiles file1 file2"
     echo      "Processes two files"
}

# test if we have two arguments on the command line
if [ $# != 2 ]
then
    usage
    exit
fi

# we are ok at this point so continue processing here

A safer practice is to validate as much as you can before running your execution. The following version of twofiles checks the argument count and tests both files. If file 1 doesn't exist (if [ 1 ! -f $1 ]) an error message is set up, a usage is displayed, and the program exits. The same is done for file 2:

#! /bin/ksh

# twofile script handles two files named on the command line

#  a usage function to display help for the hapless user

# plus an additional error message if it has been filled in

usage ()
{
     echo "twofiles"
     echo      "usage: twofiles file1 file2"
     echo      "Processes two files"
     echo " "
     echo $errmsg
}

# test if we have two arguments on the command line
if [ $# != 2 ]
then
    usage
    exit
fi

# test if file one exists and send an additional error message
# to usage if not found

if [ ! -f $1 ]
then
    errmsg=${1}":File Not Found"
    usage
    exit
fi

# same for file two

if [ ! -f $2 ]
then
    errmsg=${2}":File Not Found"
    usage
    exit
fi


# we are ok at this point so continue processing here

Note that in the Korn shell you can also use the double bracket test syntax, which is faster. The single bracket test actually calls a program named test to test the values, while the double bracket test is built into the Korn shell and does not have to call a separate program.

The double bracket test will not work in the Bourne shell:

if [[ $# != 2 ]]

or

if [[ ! -f $1 ]]

or
if [[ ! -f $2 ]]

This thorough validation can prevent later errors in the program logic when a file is suddenly found missing. Consider it good programming practice.

Contact us for a free consultation.