Linux shell

Intro

On UNIX systems, from which Linux was derived, the program used to interpret and manage commands was referred to as the shell (in Windows the interpreter used to run commands in DOS or in the CMD command interface). No matter which Linux distribution you are using, you can always count on one thing being available to you: the shell. As you grow into Linux you will surely need to use the shell at some point to track down a problem or administer some features. It provides a way to:

• create executable script files
  
• run programs
  
• work with file systems
  
• compile computer code
  
• operate a system, and
  
• manage the computer
  

Although the shell is less intuitive than common graphic user interfaces (GUIs), most Linux experts consider the shell(CLI) to be much more powerful than GUIs. The Linux shell illustrated here is called the bash shell, which stands for Bourne Again Shell. The name is derived from the fact that bash is compatible with the first UNIX shell: the Bourne shell (named after its creator, and represented by the sh command). While bash is included with most distributions, and considered a standard, other shells are available. Other popular shells include:

• the C shell (csh), which is popular among BSD UNIX users (Linux also has a tcsh shell [a C shell look-alike], and the
  
• Korn shell (ksh), which is popular among UNIX System V users (Linux also has a zsh most closely resembles ksh but includes many enhancements)
• ash shell (another Bourne shell look-alike)

Anyway it's not common that the Linux distribution you are using has more than one shell installed by default and available for your use.

To save on typing, you can find, edit, and repeat commands from your shell history. Programs consisting of commands that are stored and run from a file are referred to as shell scripts.

Most Linux system administrators use shell scripts to automate tasks such as backing up data, monitoring log files, or checking system health.

Starting a Shell

Three of the most common ways are:

• The shell prompt If your Linux system has no graphical user interface (or one that isn’t working at the moment or one that's a pure server), you will most likely see a shell prompt after you log in. Typing commands from the shell will probably be your primary means of using the Linux system. The default prompt for a regular user is simply a dollar sign: $ The default prompt for the root user —many administrative tools require root permission to be able to run them— is a pound sign (also called a hash mark): # Commonly the $ and # prompts are preceded by your username, system name, and current directory name. Anyway you can change the prompt to display any characters you like(see later). The prompt is followed by the command that you type (and then you press Enter or Return, depending on your keyboard).
  
  
• Terminal window With the desktop GUI running, you can open a terminal emulator program (known as a Terminal window) to start a shell. Most Linux distributions make it easy for you to get to a shell from the GUI. Here are two common ways to launch a Terminal window from a Linux desktop:
  • Right-click the desktop. In the context menu that appears, look for Shells, New Terminal, Terminal Window, Xterm, or some similar item and select it.
  • Click on the panel menu. Many Linux desktops include a panel at the bottom of the screen from which you can launch applications. For example, in systems that use the GNOME desktop, you can select Applications➪Accessories➪Terminal to open a Terminal window. For Mandriva, select System➪Terminals.
    
  In all cases, you should just be able to type a command as you would from a shell with no GUI. Different terminal emulators are available with Linux. One of the following is likely to be the default used with your Linux system:
  
  • xterm—A common terminal emulator for the X Window System. (In fact, I’ve never seen an X Window System for a major Linux distribution that didn’t include xterm.) Although it doesn’t provide menus or many special features, it is available with most Linux distributions that support a GUI.
  • gnome-terminal—The default Terminal emulator window that comes with GNOME. It consumes more system resources than xterm does, and it has useful menus for cutting and pasting, opening new Terminal tabs or windows, and setting terminal profiles.
  • konsole—The konsole terminal emulator that comes with the KDE desktop environment. With konsole, you can display multi-language text encoding and text in different colors.

    The differences in running commands within a Terminal window have more to do with the shell you are running than the type of Terminal window you are using. Differences in Terminal windows have more to do with the features each supports—for example, how much output is saved that can be scrolled back to, whether you can change font types and sizes, and whether the Terminal window supports features such as transparency.

• virtual terminal Most Linux systems that include a desktop interface start multiple virtual terminals running on the computer. Virtual terminals are a way to have multiple shell sessions open at once outside of the graphical interface you are using. You can switch between virtual terminals much the same way that you would switch between workspaces on a GUI. Press Ctrl+Alt+F1 (or F2, F3, F4, and so on up to F6 on Ubuntu, Fedora and other Linux systems) to display one of six virtual terminals. The next virtual workspace after the virtual terminals is where the GUI is, so if there are six virtual terminals, you can return to the GUI (if one is running) by pressing Ctrl+Alt+F7. (For a system with four virtual terminals, you return to the GUI by pressing Ctrl+Alt+F5)

Using the Shell Prompt

To find out what your current login shell is, type:

$ echo $SHELL
/bin/bash

If there are many other shells, you can activate a different one by simply typing the new shell’s command (ksh, tcsh, csh, sh, bash, and so forth) from the current shell. For example, to change temporarily to the C shell, type the following command:

$ csh

Most full Linux systems include all of the shells described in this section. However, some smaller Linux distributions may include only one or two shells. The best way to find out if a particular shell is available is to type the command and see if the shell starts. if you get a "not found" or a blank line as the response message, your system may not have it; there's a chance that you might have some version of bash installed on the system in a place (directory) that is not normally accessible to you.

You might want to choose a different shell to use because:

• You are used to using UNIX System V systems (often ksh by default) or Sun Microsystems and other Berkeley UNIX–based distributions (frequently csh by default), and you are more comfortable using default shells from those environments.
• You want to run shell scripts that were created for a particular shell environment, and you need to run the shell for which they were made so you can test or use those scripts.
• You might simply prefer features in one shell over those in another. For example, you may prefer ksh over bash because you doesn’t like the way aliases are always set up with bash.
  

Although most Linux users have a preference for one shell or another, when you know how to use one shell, you can quickly learn any of the others by occasionally referring to the shell’s man page (for example, type man bash). Most people use bash just because they don’t have a particular reason for using a different shell.


Using bash (earlier known as sh) Shells
The name bash is an acronym for Bourne Again Shell, acknowledging the roots of bash coming from the Bourne shell (sh command) created by Steve Bourne at AT&T Bell Labs. Brian Fox of the Free Software Foundation created bash, under the auspices of the GNU Project. Development was later taken over by Chet Ramey at Case Western Reserve University.

Bash includes features originally developed for sh and ksh shells in early UNIX systems, as well as some csh features. Expect bash to be the default shell in whatever Linux system you are using, with the exception of some specialized Linux systems (such as those run on embedded devices or run from a floppy disk, some bootable Linux distributions --which use the ash shell instead) that may require a smaller shell that needs less memory and entails fewer features. The bash shell is worth knowing not only because it is the default in most installations, but because it is the one tested in most Linux certification exams.

Bash can be run in various compatibility modes so that it behaves like different shells. It can be run to behave as a Bourne shell (bash +B) or as a POSIX-compliant shell (type bash --posix), for example, enabling it to read configuration files that are specific to those shells and run initialization shell scripts written directly for those shells, with a greater chance of success.

Using tcsh (earlier csh) Shells
The tcsh shell is the open source version of the C shell (csh). The csh shell was created by Bill Joy and used with most Berkeley UNIX systems (such as those produced by Sun Microsystems) as the default shell. Features from the TENEX and TOPS-20 operating systems (used on PDP-11s in the 1970s) that are included in this shell are responsible for the T in tcsh. Many features of the original csh shell, such as command-line editing and its history mechanism, are included in tcsh as well as in other shells. While you can run both csh and tcsh on most Linux systems, both commands actually point to the same executable file. In other words, starting csh actually runs the tcsh shell in csh compatibility mode.


Using ash
The ash shell is a lightweight version of the Berkeley UNIX sh shell. It doesn’t include many of the sh shell’s basic features, and is missing such features as command histories. Kenneth Almquist created the ash shell. The ash shell is a good shell for embedded systems that have fewer system resources available. The ash shell is about one-seventh the size of bash (about 100K versus 712K for bash). Because of cheaper memory prices these days, however, many embedded and small bootable Linux systems have enough space to include the full bash shell.


Using ksh
The ksh shell was created by David Korn at AT&T Bell Labs and is the successor to the sh shell. It became the default and most commonly used shell with UNIX System V systems. The open source version of ksh was originally available in many rpm-based systems (such as Fedora and Red Hat Enterprise Linux) as part of the pdksh package. Now, however, David Korn has released the original ksh shell as open source, so you can look for it as part of a ksh software package in most Linux systems.


Using zsh
The zsh shell is another clone of the sh shell. It is POSIX-compliant (as is bash), but includes some different features, such as spell checking and a different approach to command editing. The first Mac OS X systems used zsh as the default shell, although now bash is used by default.

Exploring the Shell

If you don’t like your default shell, simply type the name of the shell you want to try out temporarily. To change your shell permanently, use the usermod command. For example, to change to the csh shell for the user named chris, type the following as root user from a shell:

# usermod -s /bin/csh chris

Checking Your Login Session
When you log in to a Linux system, Linux views you as having a particular identity, which includes your:

• username,
  
• group name,
  
• user ID (uid) and
  
• group ID (gid)

Linux also keeps track of your login session. It knows:

• when you logged in,
  
• how long you have been idle, and
  
• where you logged in from.

To find out information about your identity type:

$ id
uid=1000(harrykar) gid=1000(harrykar) groups=4(adm),20(dialout),24(cdrom),46(plugdev),106(lpadmin),121(admin),122(sambashare),1000(harrykar)

In this example:

• the username is harrykar, which is represented by the numeric user ID (uid) 1000 (Based on the distribution you are using, the uid numbering may be in the hundrends.).
• The primary group for harrykar is called harrykar, which has a group ID (gid) of 1000. The user harrykar also belongs to other groups called adm (gid 4), dialout(gid 20), lpadmin (gid 106) etc. These names and numbers represent the permissions that harrykar has to access computer resources.
  

To see information about your current login session: In the following example:

• -u option add information about idle time and the process ID (pid)
• -H asks that a header be printed

$ who -uH
NAME   LINE     TIME            IDLE      PID          COMMENT
chris  tty1   Jan 13 20:57       .        2013

The output from this who command shows that

• the user chris is logged in on tty1 (which is the monitor connected to the computer), and
  
• his login session began at 20:57 on January 13.
  
• The IDLE time shows how long the shell has been open without any command being typed (the dot indicates that it is currently active).
  
• PID shows the process ID of the user’s login shell.
  
• COMMENT would show the name of the remote computer the user had logged in from, if that user had logged in from another computer on the network, or the name of the local X display if you were using a Terminal window (such as :0.0).
  

Checking Directories and Permissions
Associated with each shell is a location in the Linux file system known as the current or working directory.

Each user has a directory that is identified as the user’s home directory. When you first log in to Linux, you begin with your home directory as the current directory. Your shell uses the current directory as the point of reference. Simply provide a filename when you save a file, and it is placed in the current directory. Alternatively, you can identify a file by its relation to the current directory (relative path), or you can ignore the current directory and identify a file by the full directory hierarchy that locates it (absolute path). To find out what your current directory (the present working directory) is type:

$ pwd
or
$ echo $HOME
or
$ echo ~

To get back to your home directory:

$ cd

To list the contents of your home directory, either

• type the full path to your home directory, or
  
• use the ls command without a directory name.
  
• Using the -a option to ls enables you to view the hidden files (known as dot files because they start with that character) as well as all other files.
• With the -l option, you can see a long, detailed list of information on each file. (You can put multiple single-letter options together after a single dash—for example, -la.)
  

$ harrykar@harrysas:~$ ls -la
total 840
drwxr-xr-x 92 harrykar harrykar   4096 2009-09-07 16:34 .
drwxr-xr-x  5 root     root       4096 2009-08-19 11:58 ..
drwx------  4 harrykar harrykar   4096 2009-08-26 19:36 .adobe
-rw-r--r--  1 harrykar harrykar  50064 2009-08-24 04:37 AGConfig240809.kry
drwxr-xr-x 15 harrykar harrykar   4096 2009-08-28 17:39 armyops
drwx------  4 harrykar harrykar   4096 2009-08-28 02:58 .armyops250
-rw-------  1 harrykar harrykar   5011 2009-09-04 21:42 .bash_history
-rw-r--r--  1 harrykar harrykar    220 2009-08-06 23:34 .bash_logout
-rw-r--r--  1 harrykar harrykar   3115 2009-08-06 23:34 .bashrc
drwxr-xr-x  2 harrykar harrykar   4096 2009-09-05 11:46 .bazaar
drwx------  7 harrykar harrykar   4096 2009-09-08 01:11 .beagle
etc...

^           ^      ^        ^       ^        ^              ^
col 1      col 2  col 3   col 4    col 5    col 6        col 7

• The total line shows the total amount of disk space in KB used by the files in the list (840 kB in this example).
  
• Directories such as the current directory (.) and the parent directory (..)—the directory above the current directory—are noted as directories by the letter d at the beginning of each entry (each directory begins with a d, and each regular file begins with a -). The file and directory names are shown in column 7. In this example, a dot (.) represents /home/harrykar and two dots (..) represents /home—the parent directory of /harrykar. Most of the files in this example are dot (.) files that are used to store GUI properties (.kde, .gnome directorys) or shell properties (.bash files).
  
• Column 3 shows the file or directory(all in Linux are considering files) owner. The /home directory is owned and grouped by root, and everything else is owned by the regular user harrykar, who belongs to the harrykar group (groups are listed in column 4).
• In addition to the d or -, column 1 on each line contains the permissions set for that file or directory.
• Other information in the listing includes the number of links to the item (column 2),
  
• the size in bytes of each file (column 5). The number of characters shown for a directory (4096 bytes in these examples) reflects the size of the file containing information about the directory. While this number can grow to more than 4096 bytes for a directory that contains a lot of files, this number doesn’t reflect the size of files contained in that directory.
  
• the date and time each file was most recently modified (column 6). The format of the time and date column can vary. Instead of displaying May 12, the display can be 2008-05-12 depending upon the distribution.

Checking System Activity
In addition to being a multiuser operating system, Linux is a multitasking system. Multitasking
means that many programs can be running at the same time. An instance of a running program is referred to as a process. Linux provides tools for:

• listing running processes,
  
• monitoring system usage, and
• stopping (or killing) processes when necessary.
  

The most common utility for checking running processes is the ps command. Use it to see:

• which programs are running,
  
• the resources they are using, and
  
• who is running them.
  

Here’s an example:

$ ps au
USER  PID  %CPU  %MEM  VSZ   RSS   TTY    STAT START TIME   COMMAND
root 2146   0.0  0.8   1908  1100  ttyp0   S   14:50 0:00   login --jake
jake 2147   0.0  0.7   1836  1020  ttyp0   S   14:50 0:00   -bash
jake 2310   0.0  0.7   2592   912  ttyp0   R   18:22 0:00    ps au

In this example:

• the -a option asks to show processes of all users(multiuser OS) who are associated with your current terminal, and
  
• the -u option asks that usernames be shown, as well as other information such as the time the process started and memory and CPU usage.
  

The concept of a terminal comes from the old days, when people worked exclusively from character terminals, so a terminal typically represented a single person at a single screen. Now you can have many “terminals” on one screen by opening multiple Terminal windows. On this shell session, there isn’t much happening.

• The first process shows that the user named jake logged in to the login process (which is controlled by the root user).
  
• The next process shows that jake is using a bash shell and has just run the ps au command. The terminal device ttyp0 is being used for the login session.
  
• The STAT column represents the state of the process, with R indicating a currently running process and S representing a sleeping process. Several other values can appear under the STAT column. For example, a plus sign (+) indicates that the process is associated with the foreground operations.
  
• The USER column shows the name of the user who started the process. Each process is represented by a unique ID number referred to as a process ID (PID). (You can use the PID if you ever need to kill a runaway process.)
  
• The %CPU and %MEM columns show the percentages of the processor and random access memory, respectively, that the process is consuming.
  
• VSZ (virtual set size) shows the size of the image process (in kilobytes), and
  
• RSS (resident set size) shows the size of the program in memory
  
• START shows the time the process began running, and
  
• TIME shows the cumulative system time used. (Many commands consume very little CPU time, as reflected by 0:00 for processes that haven’t even used a whole second of CPU time.)
  

Many processes running on a computer are not associated with a terminal. A normal Linux system has many processes running in the background. Background system processes perform such tasks as logging system activity or listening for data coming in from the network. They are often started when Linux boots up and run continuously until it shuts down. To page through all the processes running on your Linux system, add the pipe (|) and the less command to ps aux, like this:

$ ps aux | less

A pipe (above the backslash character on the keyboard) enables you to direct the output of one
command to be the input of the next command. In this example, the output of the ps command (a list of processes) is directed to the less command, which lets you page through that information. Use the spacebar to page through, and type q to end the list. You can also use the arrow keys to move one line at a time through the output.


Exiting the Shell
To exit the shell when you are done, type exit or press Ctrl+D. You’ll find many commands in the /bin and /usr/bin directories, use ls to see a directory’s command list. Then use the man command (for example, man hostname) to see what each command does. Administrative commands are also in /sbin or /usr/sbin directory.

Getting Help Using the Shell
When you first start using the shell, it can be intimidating. All you see is a prompt. How do you know which commands are available, which options they use, or how to use advanced features? Fortunately, lots of help is available. Here are some places you can look to supplement what you learn in this chapter:

• Check the PATH—Type echo $PATH. You see a list of the directories containing commands that are immediately accessible to you. Listing the contents of those directories displays most standard Linux commands.
• Use the help command—Some commands are built into the shell, so they do not appear in a directory. The help command lists those commands and shows options available with each of them. (Type help | less to page through the list.) For help with a particular built-in command, type help >, replacing command with the name that interests you. The help command works with the bash shell only.
• Use --help with the command—Many commands include a --help(or -h) option that you can use to get information about how the command is used. For example, type date --help | less. The output shows not only options, but also time formats you can use with the date command.
• Use the man command—To learn more about a particular command, type man >. (Replace command with the command name you want.) A description of the command and its options appears on the screen.
• Use the info command—The info is another tool for displaying information about commands from the shell. The info command can move among a hierarchy of nodes (like hyperlinks) to find information about commands and other items. Not all commands have information available in the info database, but sometimes more information can be found there than on a man page.

Using the Shell in Linux

When you type a command in a shell, you can include other characters that change or add to how
the command works. In addition to the command itself, these are some of the other items that you can type on a shell command line:

• Options—Most commands have one or more options you can add to change their behavior. Options typically consist of a single letter, preceded by a dash. You can also often combine several options after a single dash. For example, the command ls -la lists the contents of the current directory. The -l asks for a detailed (long) list of information, and the -a asks that files beginning with a dot (.) also be listed. When a single option consists of a word, it is usually preceded by a double dash (--). For example, to use the help option on many commands, you enter --help on the command line. You can use the --help option with most commands to see the options and arguments that they support—for example, hostname --help.
  

• Arguments—Many commands also accept arguments after certain options are entered or at the end of the entire command line. An argument is an extra piece of information, such as a filename, that can be used by the command. For example, cat /etc/passwd displays the contents of the /etc/passwd file on your screen. In this case, /etc/passwd is the argument.
  

• Environment variables—The shell itself stores information that may be useful to the user’s shell session in what are called environment variables. Examples of environment variables include $SHELL (which identifies the shell you are using), $PS1 (which defines your shell prompt), and $MAIL (which identifies the location of your mailbox). You can check your environment variables at any time. Type declare to list the current environment variables( because there are always multiple ways to do anything in Linux, you can also type env to get a succinct list of the current environment variables and their values.). Or you can type echo $VALUE, where VALUE is replaced by the name of a particular environment variable you want to list.
  

• Metacharacters—These are characters that have special meaning to the shell. To name a few :can be used to direct the output of a command to a file(>), pipe the output to another command(|), and run a command in the background (&) .

To save you some typing, there are

• shell features that store commands you want to reuse,
• recallprevious commands, and
  
• edit commands.
  
• You can create aliases that enable you to type a short command to run a longer one.
  

The shell stores previously entered commands in a history list, which you can display and from which you can recall commands. Unless you specifically change to another shell, the bash shell is the one you use with most Linux systems. The bash shell contains most of the powerful features available in other shells. You can learn more about the bash shell by typing man bash.


Locating Commands
If you know the directory that contains the command you want to run, one way to run it is to type the full (absolute) path to that command. For example, you run the date command from the /bin directory by typing:

$ /bin/date

Of course, this can be inconvenient, especially if the command resides in a directory with a long path name. The better way is to have commands stored in well-known directories, and then add those directories to your shell’s PATH environment variable. The path consists of a list of directories that are checked sequentially for the commands you enter. To see your current path, type the following:

harrykar@harrysas:~$ echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/home/harrykar/bin

The results show the default path for a regular Linux user. Directories in the path list are separated by colons. Most user commands that come with Linux are stored in the /bin, /usr/bin, or /usr/local/bin directory.

Although many graphical commands (that are used with GUIs) are contained in /usr/bin, there are some special X commands that are in /usr/bin/X11 and /usr/X11R6/bin directories.

The last directory shown is the bin directory is the user’s home directory. If you want to add your own commands or shell scripts, place them in the bin directory in your home directory (such as /home/chris/bin for the user named chris). This directory is automatically added to your path in some Linux systems, although you may need to create that directory or add it to your PATH on other Linux systems. So as long as you add the command to your bin with execute permission, you can immediately begin using the command by simply typing the command name at your shell prompt. Unlike some other operating systems, Linux does not, by default, check the current directory for an executable before searching the path. It immediately begins searching the path, and executables in the current directory are run only if they are in the PATH variable or you give their absolute address. If you are the root user, directories containing administrative commands are typically in your path. These directories include /sbin and /usr/sbin. (You may need to start your shell with a -l or -login option to have /sbin and /usr/sbin added to your PATH.)

The path directory order is important. Directories are checked from left to right. So, in this example, if there is a command called foo located in both the /usr/local/sbin and /usr/local/bin directories, the one in /usr/local/sbin is executed. To have the other foo command run, you either type the full path to the command or change your PATH variable. Not all the commands that you run are located in directories in your PATH variable. Some commands are built into the shell. Other commands can be overridden by creating aliases that define any commands and options that you want the command to run. There are also ways of defining a function that consists of a stored series of commands. Here is the order in which the shell checks for the commands you type:

1. Aliases—Names set by the alias command that represent a particular command and a set of options. (Type alias to see what aliases are set.)

   harrykar@harrysas:~$ alias
   alias ls='ls --color=auto'
   

   Often, aliases enable you to define a short name for a long, complicated command.
2. Shell reserved word—Words reserved by the shell for special use. Many of these are words that you would use in programming-type functions, such as do, while, case, and else.
3. Function—A set of commands that are executed together within the current shell.
   
4. Built-in command—A command built into the shell. As a result, there is no representation of the command in the file system. Some of the most common commands you will use are shell built-in commands, such as cd (to change directories), echo (to echo text to the screen), exit (to exit from a shell), fg (to bring a command running in the background to the foreground), history (to see a list of commands that were previously run), pwd (to list the present working directory), set (to set shell options), and type (to
   show the location of a command).
5. File system command—A command that is stored in and executed from the computer’s file system. (These are the commands that are indicated by the value of the PATH variable.)
   

To find out where a particular command is taken from, you can use the type command (If you are using a shell other than bash, use the which command instead.) For example, to find out where the bash shell command is located, type the following:

harrykar@harrysas:~$ type bash
bash is /bin/bash

or

harrykar@harrysas:~$ which bash
/bin/bash

If a command resides in several locations, you can add the -a option to have all the known locations of the command printed. Sometimes you run a command and receive an error message that the command was not found or that permission to run the command was denied. In the first case, check that you spelled the command correctly and that it is located in your PATH variable. In the second case, the command may be in the PATH variable, but may not be executable.

Rerunning Commands
After typing a long or complex command line, it’s annoying to learn that you mistyped something. Fortunately, some shell features let you:

• recall previous command lines,
  
• edit those lines, or
  
• complete a partially typed command line.
  

The shell history is a list of the commands that you have entered before. Using the history command in a bash shell, you can:

1. view your previous commands.
   
2. Then, using various shell features, you can recall individual command lines from that list and
   
3. change them however you please.
   

Command-Line Editing
If you type something wrong on a command line, the bash shell ensures that you don’t have to
delete the entire line and start over. Likewise, you can recall a previous command line and change the elements to make a new command. By default, the bash shell uses command- line editing that is based on the emacs text editor. If you are familiar with emacs, you probably already know most of the keystrokes described here. If you prefer the vi command for editing shell command lines, you can easily make that happen. Add the line:

set -o vi

to the .bashrc file in your home directory. The next time you open a shell, you can use vi commands to edit your command lines.

To do the editing, you can use a combination of control keys, meta keys, and arrow keys. For
example, Ctrl+F means to hold the Ctrl key and type f. Alt+F means to hold the Alt key and type f. (Instead of the Alt key, your keyboard may use a Meta key or the Esc key. On a Windows keyboard, you can use the Windows key.) To try out a bit of command-line editing, type the following:

$ ls /usr/bin | sort -f | less

This command lists the contents of the /usr/bin directory, sorts the contents in alphabetical order (regardless of case), and pipes the output to less. The less command displays the first page of output, after which you can go through the rest of the output a line (press Enter) or a page (press the spacebar) at a time (type q when you are done). Now, suppose you want to change /usr/bin to /bin. You can use the following steps to change the command:

1. Press the up arrow to recall the line.
2. Press Ctrl+A. This moves the cursor to the beginning of the command line.
3. Press Ctrl+F or the right arrow (→) key. Repeat this command a few times to position the cursor under the first slash (/).
4. Press Ctrl+D. Type this command four times to delete /usr from the line.
5. Press Enter. This executes the command line.
   

As you edit a command line, at any point you can type regular characters to add those characters
to the command line. The characters appear at the location of your cursor. You can use right (→)
and left (←) arrow keys to move the cursor from one end to the other on the command line. You
can also press the up (↑) and down (↓) arrow keys to step through previous commands in the history list to select a command line for editing. Table 2-1 lists the keystrokes that you can use to move around the command line.

Command-Line Completion
To save you a few keystrokes, the bash shell offers several different ways of completing partially
typed values. To attempt to complete a value, type the first few characters, and then press Tab.
Here are some of the values you can type partially:

• Environment variable—If the text you type begins with a dollar sign ($), the shell completes the text with an environment variable from the current shell.
• Username—If the text you type begins with a tilde (~), the shell completes the text witha username.
• Command, alias, or function—If the text you type begins with regular characters, the shell tries to complete the text with a command, alias, or function name.
• Hostname—If the text you type begins with an at (@) sign, the shell completes the text with a hostname taken from the /etc/hosts file.

  To add hostnames from an additional file, you can set the HOSTFILE variable to the name of that file. The file must be in the same format as /etc/hosts.

Here are a few examples of command completion. (When you see , it means to press the Tab key on your keyboard.) Type the following:

$ echo $OS
$ cd ~ro
$ fing

The first example causes $OS to expand to the $OSTYPE variable. In the next example, ~ro
expands to the root user’s home directory (~root/). Next, fing expands to the finger command.
Of course, there will be times when there are several possible completions for the string of characters you have entered. In that case, you can check the possible ways text can be expanded by pressing Esc+? (or by pressing Tab twice) at the point where you want to do completion. This
shows the result you would get if you checked for possible completions on $P.

$ echo $P
$PATH $PPID $PS1 $PS2 $PS4 $PWD
$ echo $P

In this case, there are six possible variables that begin with $P. After possibilities are displayed, the original command line returns, ready for you to complete it as you choose.

Command-Line Recall
After you type a command line, that entire command line is saved in your shell’s history list. The
list is stored in a history file, from which any command can be recalled to run again. After it is
recalled, you can modify the command line, as described earlier. To view your history list, use the history command. Type the command without options or followed by a number to list that many of the most recent commands. For example:

harrykar@harrysas:~$ history 5
162  echo $P8
163  history 2

A number precedes each command line in the list. You can recall one of those commands using an
exclamation point (!). Keep in mind that when using an exclamation point, the command runs blind, without presenting an opportunity to confirm the command you’re referencing. There are
several ways to run a command immediately from this list, including:

• !n—Run command number. Replace the n with the number of the command line, and that line is run.
• !!—Run previous command. Runs the previous command line.
• !?string?—Run command containing string. This runs the most recent command that
  contains a particular string of characters.

Instead of just running a history command line immediately, you can recall a particular line and
edit it. You can use the following keys or key combinations to do that, as shown in Table 2-4. Another way to work with your history list is to use the fc command. Type fc followed by a history line number, and that command line is opened in a text editor. Make the changes that you
want. When you exit the editor, the command runs. You can also give a range of line numbers (for example, fc 100 105). All the commands open in your text editor, and then run one after the other when you exit the editor. The history list is stored in the .bash_history file in your home directory. Up to 1,000 history commands are stored for you by default.

Some people disable the history feature for the root user by setting the HISTFILE to /dev/null or simply leaving HISTSIZE blank. This prevents information about the root user’s activities from potentially being exploited. If you are an administrative user with root privileges, you may want to consider emptying your file upon exiting as well, for the same reasons.

Connecting and Expanding Commands
A truly powerful feature of the shell is the capability to redirect the input and output of commands to and from other commands and files. To allow commands to be strung together, the shell uses metacharacters. As noted earlier, a metacharacter is a typed character that has special meaning to the shell for connecting commands or requesting expansion.

Piping Commands
The pipe (|) metacharacter connects the output from one command to the input of another command. This lets you have one command work on some data, and then have the next command deal with the results. Here is an example of a command line that includes pipes:

$ cat /etc/password | sort | less

This command lists the contents of the /etc/password file and pipes the output to the sort
command. The sort command takes the usernames that begin each line of the /etc/password
file, sorts them alphabetically, and pipes the output to the less command (to page through the
output).

Pipes are an excellent illustration of how UNIX, the predecessor of Linux, was created as an operating system made up of building blocks. A standard practice in UNIX was to connect utilities in different ways to get different jobs done. For example, before the days of graphical word processors, users created plain-text files that included macros to indicate formatting. To see how the document really appeared, they would use a command such as the following:

$ gunzip < /usr/share/man/man1/grep.1.gz | nroff -c -man | less

In this example, the contents of the grep man page (grep.1.gz) are directed to the gunzip command to be unzipped. The output from gunzip is piped to the nroff command to format the man page using the manual macro (-man). The output is piped to the less command to display the output. Because the file being displayed is in plain text, you could have substituted any number of options to work with the text before displaying it. You could sort the contents, change or delete some of the content, or bring in text from other documents. The key is that, instead of all those features being in one program, you get results from piping and redirecting input and output between multiple commands. Sequential Commands Sometimes you may want a sequence of commands to run, with one command completing before the next command begins. You can do this by typing several commands on the same command line and separating them with semicolons (;):

$ date ; troff -me verylargedocument | lpr ; date 

In this example, I was formatting a huge document and wanted to know how long it would take. The first command (date) showed the date and time before the formatting started. The troff command formatted the document and then piped the output to the printer. When the formatting was done, the date and time was printed again (so I knew how long the troff command took to complete). Another useful command to add to the end of a long command line is the mail command. You could add mail -s “Finished the long command” chris@example.com to the end of a command line. Then, for example, a mail message is sent to the user you choose after the command completes. Background Commands Some commands can take a while to complete. Sometimes you may not want to tie up your shell waiting for a command to finish. In those cases, you can have the commands run in the background by using the ampersand (&). Text formatting commands (such as nroff and troff, described earlier) are examples of commands that are often run in the background to format a large document. You also might want to create your own shell scripts that run in the background to check continuously for certain events to occur, such as the hard disk filling up or particular users logging in. Here is an example of a command being run in the background:

$ troff -me verylargedocument | lpr &

Expanding Commands With command substitution, you can have the output of a command interpreted by the shell instead of by the command itself. In this way, you can have the standard output of a command become an argument for another command. The two forms of command substitution are

• $(command) and
• `command` (backticks, not single quotes).

The command in this case can include options, metacharacters, and arguments. Here is an example of using command substitution:

$ vi $(find /home | grep xyzzy)

In this example, the command substitution is done before the vi command is run.

• First, the find command starts at the /home directory and prints out all files and directories below that point in the file system.
  
• The output is piped to the grep command, which filters out all files except for those that include the string xyzzy in the filename.
  
• Finally, the vi command opens all filenames for editing (one at a time) that include xyzzy.
  

This particular example is useful if you want to edit a file for which you know the name but not the location. As long as the string is uncommon, you can find and open every instance of a filename existing beneath a point you choose in the file system. (In other words, don’t use grep a from the root file system or you’ll match and try to edit several thousand files.) Expanding Arithmetic Expressions There may be times when you want to pass arithmetic results to a command. There are two forms you can use to expand an arithmetic expression and pass it to the shell:

• $[expression] and
• $(expression)

Here is an example:

$ echo “I am $[2008 - 1957] years old.”
I am 51 years old.

The shell interprets the arithmetic expression first (2008 - 1957), and then passes that information to the echo command. The echo command displays the text, with the results of the arithmetic (51) inserted. Here’s an example of the other form:

$ echo “There are $(ls | wc -w) files in this directory.”
There are 14 files in this directory.

This lists the contents of the current directory (ls) and runs the word count command to count the number of files found (wc -w). The resulting number (14 in this case) is echoed back with the rest of the sentence shown. Expanding Environment Variables Environment variables that store information within the shell can be expanded using the dollar sign ($) metacharacter. When you expand an environment variable on a command line, the value of the variable is printed instead of the variable name itself, as follows:

harrykar@harrysas:~$ ls -l $BASH
-rwxr-xr-x 1 root root 833592 2009-03-02 15:23 /bin/bash

Using $BASH as an argument to ls -l causes a long listing of the bash command to be printed. The following section discusses shell environment variables.

Creating Your Shell Environment

You can tune your shell to help you work more efficiently. Your prompt can provide pertinent information each time you press Enter. You can set aliases to save your keystrokes and permanently set environment variables to suit your needs. To make each change occur when you start a shell, add this information to your shell configuration files. Configuring Your Shell Several configuration files support how your shell behaves. Some of the files are executed for every user and every shell, while others are specific to the user who creates the configuration file. Table 2-5 shows the files that are of interest to anyone using the bash shell in Linux. To change the /etc/profile or /etc/bashrc files, you must be the root user. Users can change the information in the $HOME/.bash_profile, $HOME/.bashrc, and $HOME/.bash_logout files in their own home directories. The following sections provide ideas about items to add to your shell configuration files. In most cases, you add these values to the .bashrc file in your home directory. However, if you administer a system, you may want to set some of these values as defaults for all of your Linux system’s users. Setting Your Prompt Your prompt consists of a set of characters that appear each time the shell is ready to accept a command. The PS1 environment variable sets what the prompt contains and is what you interact with most of the time. If your shell requires additional input, it uses the values of PS2, PS3, and PS4. When your Linux system is installed, often a prompt is set to contain more than just a dollar sign or pound sign. For example, in Ubuntu, Fedora or Red Hat Enterprise Linux, your prompt is set to include the following information:

• your username,
  
• your hostname, and the
  
• base name of your current working directory.
  

That information is sometimes surrounded by brackets and followed by a dollar sign (for regular users) or a pound sign (for the root user). Here is an example of that prompt: [chris@myhost bin]$ or chris@myhost: bin$ . If you change directories, the bin name would change to the name of the new directory. Likewise, if you were to log in as a different user or to a different host, that information would change. You can use several special characters (indicated by adding a backslash to a variety of letters) to include different information in your prompt. These can include your terminal number, the date, and the time, as well as other pieces of information. Table 2-6 provides some examples (you can find more on the bash man page). If you are setting your prompt temporarily by typing at the shell, you should put the value of PS1 in quotes. For example, you could type export PS1=”[\t \w]\$ “ to see a prompt that looks like this: [20:26:32 /var/spool]$. To make a change to your prompt permanent, add the value of PS1 to your .bashrc file in your home directory (assuming that you are using the bash shell). There may already be a PS1 value in that file that you can modify. Refer to the Bash Prompt HOWTO for information on changing colors, commands, and other features of your bash shell prompt. Adding Environment Variables You may consider adding a few environment variables to your .bashrc file. These can help make working with the shell more efficient and effective:

• TMOUT—Sets how long the shell can be inactive before bash automatically exits. The value is the number of seconds for which the shell has not received input. This can be a nice security feature, in case you leave your desk while you are still logged in to Linux. So as not to be logged off while you are working, you may want to set the value to something like TMOUT=1800 (to allow 30 minutes of idle time). You can use any terminal session to close the current shell after a set number of seconds—for example, TMOUT=30.
  

• PATH—As described earlier, the PATH variable sets the directories that are searched for commands you use. If you often use directories of commands that are not in your PATH, you can permanently add them. To do this, add a PATH variable to your .bashrc file. The example below first reads all the current path directories into the new PATH ($PATH), adds the /getstuff/bin directory, and then exports the new PATH. Some people add the current directory to their PATH by adding a directory identified simply as a dot (.), as follows:

  PATH=.:$PATH ; export PATH

  This enables you always to run commands in your current directory before evaluating any other command in the path (which people may be used to if they have used DOS). However, the security risk with this procedure is that you could be in a directory that contains a command that you don’t intend to run from that directory. For example, a malicious person could put an ls command in a directory
  that, instead of listing the content of your directory, does something devious. Because of this, the practice of adding the dot to your path is highly discouraged:
  

  PATH=$PATH:/getstuff/bin ; export PATH

• WHATEVER—You can create your own environment variables to provide shortcuts in your work. Choose any name that is not being used and assign a useful value to it. For example, if you do a lot of work with files in the /work/time/files/info/memos directory, you could set the following variable:

  M=/work/time/files/info/memos ; export M

  You could make that your current directory by typing cd $M. You could run a program from that directory called hotdog by typing $M/hotdog. You could edit a file from there called bun by typing vi $M/bun.

Adding Aliases Setting aliases can save you even more typing than setting environment variables. With aliases, you can have a string of characters execute an entire command line. You can add and list aliases with the alias command. Here are some examples of using alias from a bash shell:

alias p=’pwd ; ls -CF’
alias rm=’rm -i’

In the first example, the letter p is assigned to run the command pwd, and then to run ls -CF to print the current working directory and list its contents in column form. The second runs the rm command with the -i option each time you simply type rm. (This is an alias that is often set automatically for the root user, so that instead of just removing files, you are prompted for each individual file removal. This prevents you from automatically removing all the files in a directory by mistakenly typing something such as rm *.) While you are in the shell, you can check which aliases are set by typing the alias command. If you want to remove an alias, type unalias. Remember that if the alias is set in a configuration file, it will be set again when you open another shell.) Using Shell Environment Variables Every active shell stores pieces of information that it needs to use in what are called environment variables. An environment variable can store things such as

• locations of configuration files,
  
• mailboxes,
  
• path directories.
  

They can also store values for

• your shell prompts,
  
• the size of your history list, and
  
• type of operating system.

To see the environment variables currently assigned to your shell, type the declare command. (It will probably fill more than one screen, so type declare | more. The declare command also shows functions as well as environment variables.) You can refer to the value of any of those variables by preceding it with a dollar sign ($) and placing it anywhere on a command line. For example:

$ echo $USER
chris

This command prints the value of the USER variable, which holds your username (chris). Substitute any other value for USER to print its value instead. Common Shell Environment Variables When you start a shell (by logging in or opening a Terminal window), a lot of environment variables are already set. Table 2-7 shows some variables that are either set when you use a bash shell or that can be set by you to use with different features. Setting Your Own Environment Variables Environment variables can provide a handy way to store bits of information that you use often from the shell. You can create any variables that you want (avoiding those that are already in use) so that you can read in the values of those variables as you use the shell. (The bash man page lists variables already in use.) To set an environment variable temporarily, you can simply type a variable name and assign it to a value. Here’s an example:

$ AB=/usr/dog/contagious/ringbearer/grind ; export AB

This example causes a long directory path to be assigned to the AB variable. The export AB command says to export the value to the shell so that it can be propagated to other shells you may open. With AB set, you go to the directory by typing the following:

$ cd $AB

The problem with setting environment variables in this way is that as soon as you exit the shell in which you set the variable, the setting is lost. To set variables permanently, add variable settings to a bash configuration file, as described later in this section.

Another option to add the settings to the bash configuration file is to create an executable script file that contains these settings. This is useful when you don’t use the settings all the time, but need to use them occasionally. They are there only for the life of the session after the script file has run.

If you want to have other text right up against the output from an environment variable, you can surround the variable in braces. This protects the variable name from being misunderstood. For example, if you want to add a command name to the AB variable shown earlier, you can type the following:

$ echo ${AB}/adventure
/usr/dog/contagious/ringbearer/grind/adventure

Remember that you must export the variable so that it can be picked up by other shell commands. You must add the export line to a shell configuration file for it to take effect the next time you log in. The export command is fairly flexible. Instead of running the export command after you set the variable, you can do it all in one step, as follows:

$ export XYZ=/home/xyz/bin

You can override the value of any environment variable. This can be temporary, by simply typing the new value, or you can add the new export line to your $HOME/.bashrc file. One useful variable to update is PATH:

$ export PATH=$PATH:/home/xyz/bin

In this example, the /home/xyz/bin directory is added to the PATH, a useful technique if you want to run a bunch of commands from a directory that is not normally in your PATH, without typing the full or relative path each time. If you decide that you no longer want a variable to be set, you can use the unset command to erase its value. For example, you can type unset XYZ, which causes XYZ to have no value set. (Remember to remove the export from the $HOME/.bashrc file—if you added it there—or it will return the next time you open a shell.)

Resources

Linux Bible 2008 Edition: Boot Up to Ubuntu, Fedora, KNOPPIX, Debian, openSUSE, and 11 Other Distributions  
by Christopher Negus
Wiley Publishing ISBN: 978-0-470-23019-0  

A Practical Guide to Linux Commands, Editors, and Shell Programming 
By Mark G. Sobell 
Prentice Hall PTR ISBN: 0-13-147823-0

Learning the bash Shell, 3rd Edition
By Cameron Newham
O'Reilly ISBN: 0-596-00965-8

Howto's