A Quick Intro to Linux Shell Scripting for Windows Developers

2016-11-18 - By Robert Elder

     This article is intended to provide an extremely brief and basic introduction to the Linux shell scripting.  Special emphasis will be placed on using shell scripts for the purpose of automated testing.  The intended audience is people who don't have much experience Linux command line before, but may be familiar with software development on Windows platforms.  An interested reader may also benefit from reading A Quick Intro to Linux Command Line for Windows Users.

What is A Shell Script?

     A 'shell script' is a computer program that contains commands that one could (with few exceptions) just as easily type into a command shell one-by one.

     A command shell is a command-line interface that allows you to tell the operating system kernel to do various useful things like launch a process, write to disk, or talk to a peripheral.  There are many common shells in Linux/Unix and each has their own flavour of syntax:

• bash
• dash
• sh
• zsh
• tsch

     This discussion will focus specifically on writing scripts for the bash shell.  Since what we'll be doing is very simple, the syntax is likely to be mostly identical in all other major shells that are used on Linux.  If you'd like more background on Linux commands you can check out an overview of basic Linux commands here.

Writing Your First Shell Script

     For starters, here are 4 basic Linux commands:

uname
date
whoami
free

     You can type these into a terminal one by one, or you can put them all into a file, one on each line as they are above and execute them all at once.

Making Your Script Executable

      Let's say you put your commands in a file called 'myscript'.  Before you run the script, check its permissions by running this command

ls -latr

in the directory where myscript is, you might see something like this:

total 24
drwxrwxrwt 12 root   root   20480 Nov 18 19:25 ..
-rw-rw-r--  1 robert robert     0 Nov 18 19:26 myscript
drwxrwxr-x  2 robert robert  4096 Nov 18 19:26 .

     Below is a highlighted view that identifies 3 different permission groupings for the myscript file:

-rw-rw-r--  1 robert robert     0 Nov 18 19:26 myscript

     In Linux, file permissions are separated into three groups as you can see above:  In this example they have been highlighted as user permissions green, group permissions red, and other permissions blue.  Each of user, group and other has a flag for Readable (r), Writable (w), and Executable (x).  If one of these permissions is turned off you will see '-' instead of r, w or x.

     If you're paying attention, you'll note that none of the user, group or other has the executable permission set on the myscript file.  We can give the user the Executable permission by running this command:

chmod u+x myscript

     And if we check the file permissions again, you'll see that the execute permission is now set for the user:

-rwxrw-r--  1 robert robert     0 Nov 18 19:26 myscript

     In the 'chmod' command that we issued above the 'u' stands for 'user' and the '+' stands for 'add these permissions', finally the 'x' stands for 'Executable'.  For me, the current user is 'robert', and that's also the user you see displayed in the example above.  For you, this name will be different.

     Now, we are finally able to run out script!  You can run myscript by running the following command:

./myscript

     When I run this, I get the following output:

Linux
Fri Nov 18 20:27:15 EST 2016
robert
              total        used        free      shared  buff/cache   available
Mem:       16334608     1870044    12434248      106212     2030316    13981212
Swap:      19530748           0    19530748

     If you get this instead:

bash: ./myscript: Permission denied

you have most likely not set the permissions correctly to execute the script.

Add A Shebang To Your Script

     Next, you should add a Shebang to your script:

#!/bin/bash

so you end up with

#!/bin/bash
uname
date
whoami
free

     The shebang will identify which shell should be used to run your script.  You should always include a shebang at the start of your scripts, but often the script will still be able to run without one and will simply use the default shell.

How To Capture Program Return Codes

     When you run a program (or a script, or a command) a return code will be available to indicate success or failure.  By convention, a return code of '0' means success, and any non-zero return code generally means an error.  In bash you can get the return code of the last command using:

$?

     You will even get a return code from running any shell command:

date
echo "The return code is "$?

Variables

     In bash you can create a variable and give it a value by doing the following:

my_variable=1234

     Note that variable assignments are WHITESPACE SENSITIVE:

#  This will work
my_variable=12
#  This NOT will work
my_variable =34
#  This NOT will work
my_variable = 56

     Also note that when you do a variable assignment, you leave off the '$' sign, but when you reference the variable you need to use it:

my_variable=1234
echo "Here is the value of my_variable: "$my_variable

     There are cases where you may want to surround the variable name with '{', '}' characters in order to avoid ambiguity.

my_variable=1234
echo "Here is the value of my_variable: "__${my_variable}__

     You can also place the variable inside of double quotes and it will still be interpolated:

my_variable=1234
echo "Here is the value of my_variable: ${my_variable}"

Using If Statements

     Let's combine what we've just learned and do something useful with an if statement:

#!/bin/bash

#  Try to make a folder called 'foo'
mkdir foo
#  Grab the return code to see if it worked
RETURN_CODE=$?

if [ ${RETURN_CODE} -eq 0 ]; then
	echo "The return code was 0."
else
	echo "The return code was non-zero."
fi

     The above script will try to create the directory 'foo', and then detect whether the command ran successfully or not.  If we run this script 2 times in a row, the 'mkdir' command will fail the second time because the directory 'foo' will already exist:

./myscript
The return code was 0.

./myscript
mkdir: cannot create directory ‘foo’: File exists
The return code was non-zero.

Pipes and Output Re-direction

     By default most of the output of a command or program goes to what's called 'stdout' which just gets displayed on the terminal you're currently using.  You can take output on 'stdout' and put it in a file by using the '>' character:

ls > files_list

     In the above examples, the file 'files_list' will now contain the output of the 'ls' command.

     There is also something called 'stderr' which is often used to display error messages.  You can redirect stderr to file by using '2>' instead:

ls non_existing_file 2> error_message

     Now the file 'error_message' will contain the resulting error message from trying to list details on a file that does not exist.  If the file did exist, there would be nothing in the 'error_message' file.

     Another way you can redirect input/output it to put the output of one command directly into the input of another command:

ls | sort

     The above will list all files in the current directory, and then send them into another program called 'sort' that will sort them.

Evaluate A Subshell

     One very useful thing you can do in bash is evaluate a bash command (or program or script) and assign the output of that to a variable:

THE_DATE=$(date)
echo "${THE_DATE}"

     The '...' below can be pretty much any shell command that you could directly into the shell:

$(...)

     For example you can even use an entire shell pipe which can be assigned to a variable:

THE_RESULT=$(head /dev/urandom | xxd | grep 1 | sort -n)

Sending Parameters to A Script

     If you want to run a script from your shell and pass in parameters, like this:

./myscript 1 hello "hi there"

     The above example sent 3 parameters.  Parameters are separated by whitespace, expect for whitespace in between quotes.  You can access the parameters in your script by typing ${<parameter number>}.  For example, if my script contained the following and was run with the previous command:

echo "First Parameter: ${1}"
echo "Second Parameter: ${2}"
echo "Third Parameter: ${3}"
echo "Total number of parameters is: ${#}"

     The output would be

First Parameter: 1
Second Parameter: hello
Third Parameter: hi there
Total number of parameters is: 3

Environment Variables

     In every shell there are a number of environment variables that keep relevant information such as the default locations to look for executables, sessions ids and a number of other things.  To find out what your current environment variables are, you can use this command:

env

     The output of this command is rather verbose, so I won't paste it all here, but if I run

env |  grep "^PATH"

     I get the following:

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin

Conclusion

     In this article we've seen some basic ideas that can get you started in writing shell scripts on Unix.  To re-cap these features are: shebangs, file permissions, return codes, variables, if statements, pipes and I/O redirection, sub-shell evaluation, sending parameters, and environment variables.  This should be enough to get you started with automated testing by allowing you to automatically run programs and check return codes.  There are a lot more details to each of the features we've seen so far but in the interest of keep this article short I've left most of them out.