Total Pageviews

Search: This Blog, Linked From Here, The Web, My fav sites, My Blogroll

23 June 2010

FreeBSD --- Shell


Using the Shell

There are several different shells to choose from on BSD systems.
  • By default, the root user is assigned to use the C shell (csh) and a regular user is assigned the Bourne shell (sh). 
  • With Linux systems, most users use what is called the Bourne Again Shell (bash). 
  • There are other shells available as well (such as zsh, ksh, or tcsh).
Here i will help you use BSD shells, in general, and the bash shell, in particular.

Terminal Windows and Shell Access
The most common way to access a shell from a BSD graphical interface is
using a Terminal window. From a graphical interface, you can often access
virtual terminals to get to a shell. With no graphical interface, and a text-based login, you are typically dropped directly to a shell(ttyv0) after login.

Using Terminal Windows
To open a Terminal window
  • from GNOME, select Accessories ➪ Terminal. This opens a gnome-terminal window, displaying a bash shell prompt.
  • From an XFce desktop, select the Terminal icon from the XFce panel. 
  • From KDE, select the Terminal Program icon to start a konsole terminal window.
if just installed  echo $SHELL command output the current shell as the bash shell (/usr/local/bin/bash);
  1. the current user is the desktop user who launched the window (chris)
  2. the current directory is that user’s home directory (/home/chris). The user name (i.e. chris) and hostname (i.e localhost) appear in the title bar.
The gnome-terminal window not only lets you access a shell, it also has controls for managing your shells. For example, click File ➪ Open Tab to open another shell on a different tab, click File ➪ Open Terminal to open a new Terminal window, or select Terminal ➪ Set Title to set a new title in the title bar.
    You can also use control key sequences to work with a Terminal window.
  • Open a shell on a new tab by typing Shift+Ctrl+t
  • open a new Terminal window with Shift+Ctrl+n
  • close a tab with Shift+Ctrl+w
  • close a Terminal window with Shift+Ctrl+q
  • Highlight text and copy it with Shift+Ctrl+c, then paste it in the same or different window with Shift+Ctrl+v or by clicking the center button on your mouse.
  • Other key sequences for controlling Terminal windows include pressing F11 to show the window in full screen mode. 
  • Type Ctrl+Shift++ to zoom in (make text larger) or Ctrl+- (that’s Ctrl and a minus sign) to zoom out (make text smaller).   
  • Switch among tabs using Ctrl+PageUp and Ctrl+PageDown (previous and next tab), or use Alt+1, Alt+2, Alt+3 and so on to go to tab one, two, or three (and so on). 
  • Type Ctrl+D to exit the shell, which closes the current tab or entire Terminal window (if it’s the last tab).
The gnome-terminal window also supports profiles (select Edit ➪ Current Profile). Some profile settings are cosmetic (allow bold text, cursor blinks, terminal bell, colors, images, and transparency). Other settings are functional. For example, by default, the terminal saves 500 scrollback lines (318 kilobytes). Some people like to be able to scroll back further and are willing to give up more memory to allow that.
 If you launch gnome-terminal manually, you can add options. Here are some examples:
# gnome-terminal -x alsamixer  Start terminal with alsamixer displayed
# gnome-terminal --tab --tab --tab  Start a terminal with three open tabs
# gnome-terminal --geometry 80x20  Start terminal 80 characters by 20 lines
# gnome-terminal --zoom=2  Start terminal with larger font

Using Virtual Terminals (ttyvx)
When FreeBSD boots in multi-user mode, eight virtual consoles (represented by a
console message window and virtual terminals ttyv0 through ttyv7) are created with text-based logins.
If an X Window System desktop is running, X is probably running in virtual console 8.
Virtual console 1 is where console messages are displayed. If X isn’t running, chances are you’re looking at virtual console 2 --Press Alt+F9 to return to the X GUI--. From X, you can switch to another virtual console with Ctrl+Alt+F1, Ctrl+Alt+F2, and so on up to F8. From a text virtual console, you can switch using Alt+F1, Alt+F2 and so on.
    Each console allows you to log in using different(or the same) user accounts. Switching to look at another console doesn’t affect running processes in any of them. When you switch to virtual terminal one through six, you see a login
prompt similar to the following:
FreeBSD/i386 (localhost) (ttyv1)

Separate getty processes manage each virtual terminal. If you are in a Terminal window, type this command to see what getty processes look like before you log in to any virtual terminals:
# ps awx | grep -v grep | grep getty
780  v1  Is+  0:00.01 /usr/libexec/getty Pc ttyv1
781  v2  Is+  0:00.01 /usr/libexec/getty Pc ttyv2
782  v3  Is+  0:00.01 /usr/libexec/getty Pc ttyv3
783  v4  Is+  0:00.01 /usr/libexec/getty Pc ttyv4
784  v5  Is+  0:00.01 /usr/libexec/getty Pc ttyv5
785  v6  Is+  0:00.01 /usr/libexec/getty Pc ttyv6
786  v7  Is+  0:00.01 /usr/libexec/getty Pc ttyv7

After you log in on the first console, getty handles your login, and then fires up an
sh shell:
# ps awx | grep -v grep | grep v0
780  v0  Is  0:00.04  login  [pam] (login)
2300   v0  S  0:00.02 -sh (sh)

Virtual consoles are configured in the /etc/ttys file. You can have fewer or more
virtual terminals by adding or deleting getty lines from that file. Type the following to see which ttys are configured:
# grep ttyv /etc/ttys
ttyv0   “/usr/libexec/getty Pc”  cons25 on secure

After you open a shell (whether from a text-based login or Terminal window), the shell environment is set up based on the user who started the shell. Each user’s default shell is assigned based on the user’s entry in the /etc/passwd file.
    By default, the root user’s password is set to csh (/bin/csh) and regular users are set to sh (/bin/sh).
To use the bash shell, you can set the user’s shell to /usr/local/bin/bash(i.e trough sysinstall). 
Examples in this section use the bash shell.

Bash shell settings for all users’ shells are located in /etc/profile. User-specific shell settings are determined by commands executed from several dot files in the user’s home directory (if they exist):
  • .bash_profile, 
  • .bash_login, and 
  • .profile. 
When a shell is closed, any commands in the user’s ~/.bash_logout file are executed. Changing settings in these files permanently changes the user’s shell settings but does not affect shells that are already running. (Other shells use different configuration files). There are a variety of ways in which you can list and change your shell environment. One of the biggest ways is to change which user you are; in particular, to become the super user.

Using Bash History
The Bourne Again Shell (bash) is used by many UNIX-like systems. Built into bash, as with other shells, is a history feature that lets you review, change, and reuse commands that you have run in the past.
    When bash starts, it reads the ~/.bash_history file and loads it into memory. This file is set by the value of $HISTFILE.
During a bash session, commands are added to history in memory. When bash exits, history in memory is written back to the .bash_history file. 
The number of commands held in history during a bash session is set by $HISTSIZE, while the number commands actually stored in the history file is set by $HISTFILESIZE:
/home/fcaen/.bash_history 500 500

To list the entire history, type history. To list a previous number of history commands, follow history with a number. This lists the previous five commands in your history:
$ history 5
975 mkdir extras
976 mv *doc extras/
977 ls -CF
978 vi house.txt
979 history

To move among the commands in your history, use the up arrow and down arrow. When a command is displayed, you can use the keyboard to edit the current command like any other command: left arrow, right arrow, Delete, Backspace, and so on. Here are some other ways to recall and run commands from your bash history:
$ !!  Run the previous command$ !997  Run command number 997 from history
ls -CF  
$ !997 *doc  Append *doc to command 997 from history
ls -CF *doc  
$  !?CF?  Run previous command line containing the CF string
ls -CF *doc
$ !ls  Run the previous ls command
ls -CF *doc
$  !ls:s/CF/l  Run previous ls command, replacing CF with l
ls -l *doc

Another way to edit the command history is using the fc command. With fc, you open the chosen command from history using the vi editor. The edited command runs when you exit the editor. Change to a different editor by setting the FCEDIT variable (for example, FCEDIT=gedit) or on the fc command line. For example:
$ fc 978   Edit command number 978, then run it
$ fc   Edit the  previous command, then run it
$ fc -e /usr/local/bin/nano 989    Use nano to edit command 989

(Use pkg_add -r to install the text editor you want, such as nano, if it is not already installed).

Use  to search for a string in history. For example, typing Ctrl+r followed by the string ss resulted in the following:
# "Ctrl+r"
(reverse-i-search)`ss’: sudo /usr/bin/less /var/log/messages

Press Ctrl+r repeatedly to search backward through your history list for other occurrences of the ss string.
NOTE By default, bash command history editing uses emacs-style commands. If you prefer the vi editor, you can use vi-style editing of your history by using the set command to set your editor to vi. To do that, type the following: set -o vi.

Using Command Line Completion
You can use the Tab key to complete different types of information on the command line. Here are some examples where you type a partial name, followed by the Tab key, to have bash try to complete the information you want on your command line:
$ tracer  Command completion: Completes to traceroute command$ cd /home/ch  File completion: Completes to /home/chris directory
$ cd ~jo  User homedir completion: Completes to /home/john
$ echo $PA  Env variable completion: Completes to $PATH
$ ping @    Host completion: Show hosts from /etc/hosts
@localhost @zooey

Redirecting stdin and stdout
Typing a command in a shell makes it run interactively. The resulting process has two output streams: stdout for normal command output and stderr for error output. In the following example, when /tmpp isn’t found, an error message goes to stderr but output from listing /tmp (which is found) goes to stdout:
$ ls /tmp /tmpp
ls: /tmpp: No such file or directory
gconfd-fcaen  keyring-b41WuB keyring-ItEWbz mapping-fcaen orbit-fcaen 

By default, all output is directed to the screen. Use the greater-than sign (>) to direct output to a file. More specifically, you can direct the standard output stream (using >) or standard error stream (using 2>) to a file. Here are examples:
$ ls /tmp /tmmp > output.txt
ls: /tmpp: No such file or directory

$ ls /tmp /tmmp 2> errors.txt
gconfd-fcaen keyring-b41WuB keyring-ItEWbz mapping-fcaen orbit-fcaen

$ ls /tmp /tmmp 2> errors.txt > output.txt
$ ls /tmp /tmmp > everything.txt 2>&1

  1. In the first example, stdout is redirected to the file output.txt, while stderr is still directed to the screen. 
  2. In the second example, stderr (stream 2) is directed to errors.txt while stdout goes to the screen. 
  3. In the third example, the first two examples are combined. The last example directs both streams to the everything.txt file. 
To append to a file instead of overwriting it, use two greater-than signs:
$ ls /tmp >> output.txt

If you don’t ever want to see an output stream, you can simply direct the output stream to a special bit bucket file (/dev/null):
# ls /tmp 2> /dev/null

TIP Another time you may want to redirect stderr is when you run jobs with crontab. You could redirect stderr to a mail message that goes to the crontab’s owner. That way, any error messages can be sent to the person running the job.
Just as you can direct standard output from a command, you can also direct standard input to a command. For example, the following command e-mails the /etc/hosts file to the user named chris on the local system:
$ mail chris < /etc/hosts

Using pipes, you can redirect output from one process to another process rather than just files. Here is an example where the output of the ls command is piped to the sort command to have the output sorted:
# ls /tmp | sort

In the next example, a pipe and redirection are combined (the stdout of the ls command is sorted and stderr is dumped to the bit bucket):
# ls /tmp/ /tmmp 2> /dev/null | sort

 Pipes can be used for tons of things:
#pkg_info | grep -i gnome | wc -l
#ps auwx | grep firefox
#ps  auwx | less
#whereis -m pkg_add | awk ‘{print $2}’

  1. The first command line in the preceding code lists all installed packages, grabs those packages that have gnome in them (regardless of case), and does a count of how many lines are left (effectively counting packages with gnome in the name). 
  2. The second command line displays Firefox processes taken from the long process list (assuming the Firefox web browser is running). 
  3. The third command line lets you page through the process list. 
  4. The last line displays the word pkg_add: followed by the path to the
    pkg_add man page, and then displays only the path to the man page (the second element on the line).
Using backticks, you can execute one section of a command line first and feed the output of that command to the rest of the command line. Here are examples:
# pkg_info -W `which mkisofs`
/usr/local/bin/mkisofs was installed by package cdrtools-2.01_6

# ls -l `which traceroute`
-r-sr-xr-x 1 root wheel 19836 Jan 15 18:33 /usr/sbin/traceroute

  1. The first command line in the preceding example finds the full path of the mkisofs command and finds the package that contains that command. 
  2. The second command line finds the full path to traceroute and does a long list (ls -l) of that command.
A more advanced and powerful way to take the output of one command and pass it as a parameter to another is with the xargs command. For example, after installing the cdrtools package, I ran the following command:
$ find /usr/local/bin | grep iso | xargs ls

To display the command xargs is going to run, use the -t option as follows:
# cd /usr/local/bin
# echo iso* | xargs -t ls
ls iso-info iso-read isodebug isodump isoinfo isovfy
iso-info isodebug isoinfo iso-read isodump isovfy

In the above example, the entire output of echo is passed to pkg_info. Using the -t
option to xargs, a verbose output of the command line appears before the command is executed. Now have xargs pass each output string from ls as input to individual pkg_add commands. Use the -I option to define {} as the placeholder for the string:
# ls /usr/local/bin/mk* | xargs -t -I {} pkg_info -W {}
pkg_info  -W /usr/local/bin/mkbundle
/usr/local/bin/mkbundle was installed  by package mono-
pkg_info -W /usr/local/bin/mkisofs
/usr/local/bin/mkisofs  was installed by package cdrtools-2.01_6

As you can see from the output, separate pkg_info commands are run for each
option passed by ls.

Using aliases
Use the alias command to set and list aliases. Some aliases are already set in each user’s ~/.cshrc file, if a user uses the csh shell. Here’s how to list the aliases that are currently set for that shell:
# alias
h   (history 25)
j    (jobs -l)
la  (ls -a)
lf   (ls -FA)
ll   (ls -lA)
Aliases can also be set simply as a way of adding options to the default behavior of a command (such as alias mv mv -i, so that the user is always prompted before
moving a file).
    To define other aliases for the csh shell, use the alias command, followed by
the alias name and command to run. For example:
$ alias fi “find . | grep $*”

Using the alias just shown, typing fi string causes the current directory and its
subdirectories to be searched for any file names containing the string you entered.
You can define your own aliases for the current bash session as follows:
# alias la=’ls -la’

Add that line to your ~/.bashrc file for the definition to occur for each new bash session. Remove an alias from the current bash session using the unalias command, as follows:
# unalias la  Unalias the previously aliased la command
# unalias -a  Unalias all aliased commands

Tailing Files
To watch the contents of a plain text file grow over time, you can use the tail command. For example, you can watch as messages are added to the /var/log/cron file as follows:
# tail -f /var/log/cron

Press Ctrl+C to exit the tail command.

Acquiring Super-User Power
When you open a shell, you are able to run commands and access files and directories based on your user/group ID and the permissions set for those components. Many system features are restricted to the root user, also referred to as the super user.
    Because BSD distributions have a reputation for extraordinary security, access to the root user account is discouraged except in very specific situations. By default, root access is only allowed from terminals listed in /etc/ttys as secure. That includes only virtual terminals accessible from the console. So if you need to use the root account, the only way to do that, as FreeBSD is delivered, is to log in directly as root from the computer’s console terminal. To be able to become root user, while logged in as another user, you need to change some security settings and use either the su or sudoers command as described below.

Using the su Command
Non-root users are prevented from becoming root users based on settings in the
/etc/pam.d/su file. Those settings only allow the root user or members of the wheel group access to the root user account via the su command. If you change the /etc/pam.d/su file to allow a user to use the su command, you can run the
su (super user) command to become the root user. However, simply using su, as in the following code, doesn’t give you a login shell with the root user’s environment:
$ su
# echo $MAIL

After running su, the user still has fcaen’s MAIL folder. To enable the root user’s environment, use the su command with the dash option (-), as follows:
# exit
$ su -
Password: *****
# echo $MAIL

In most cases, use su -, unless you have a very specific reason not to. If no user is specified, su defaults to the root user. However, su can also be used to become other users:
$ su - cnegus

The su command can also be used to execute a single command as a particular user provided that the user has been given permission to use su at all:
$ su -c whoami
Password: ******
# su -c ‘less  /var/log/messages’

Although in the second example you are logged in as a regular user, when you run whoami with su -c, it shows that you are the root user. In the directly preceding example, the quotes are required around the less command line to identify /var/log/messages as an option to less. As seen above, whoami can be useful to determine which user you’re currently running a command as:
$ whoami

Delegating Power with sudo
The sudo command allows very granular delegation of power to users other than the root user.
The sudo facility is a great tool, when you have multiple users, for granting specific escalated privileges and logging everything the users do with those privileges.
Unless otherwise specified, sudo runs as root. To use the sudo command, you must install the sudo package.
    The sudo command is configured in /usr/local/etc/sudoers.

WARNING! Never edit this file with your normal text editor. Instead, always use the visudo command.
If you look at the sudoers file that shipped with your distribution, you’ll see different empty sections delimited by comments and one active statement:
root  ALL=(ALL) ALL
This means that the user root is allowed on any hosts to run any command as any user.
    Now add the following line setting the first field to a user account on your system:
fcaen ALL= /usr/bin/less /var/log/messages
Now fcaen (or whichever user you’ve added) can do the following:
$ sudo /usr/bin/less /var/log/messages

After fcaen types his own password, he can page through the /var/log/messages
A timestamp is set at that time as well. For the next five minutes (by default), that user can type the command line above and have it work without being prompted for the password.
Every use of sudo gets logged in /var/log/messages:
Feb 24 21:58:57 localhost sudo: fcaen : user NOT in sudoers ;  TTY=ttyv3 ;
PWD=/home/fcaen ; USER=root ; COMMAND=/usr/bin/less  /var/log/messages

Next add this line to /etc/sudoers:
fcaen server1=(chris) /bin/ls /home/chris

Now fcaen can do the following:
$ sudo -u chris /bin/ls /home/chris

The sudo command just shown runs as chris and will work only on the host server1.
In some organizations, the sudoers file is centrally managed and deployed to all the hosts, so it can be useful to specify sudo permissions on specific hosts.
Sudo also allows the definition of aliases, or predefined groups of users, commands, and hosts. Check the sudoers man page and the /usr/local/etc/sudoers file on your BSD system for examples of those features.

Using Environment Variables
Small chunks of information that are useful to your shell environment are stored in what are referred to as environment variables. By convention, environment
variable names are all uppercase (although that convention is not enforced). If you use the bash shell, some environment variables might be set for you from various bash start scripts:
  • /etc/profile and 
  • ~/.bash_profile (if it exists).
To display all of the environment variables, in alphabetical order, that are already set for your shell, type the following:
$ set | less

The output just shown contains only a few examples of the environment variables you will see. You can also set, or reset, any variables yourself. For example, to assign the value 123 to the variable ABC (then display the contents of ABC), type the following:
$ ABC=123
$ echo $ABC

The variable ABC exists only in the shell it was created in. If you launch a command from that shell (ls, cat, firefox, and so on), that new process will not see the variable. Start a new bash process and test this:
$ bash
$ echo $ABC

You can make variables part of the environment and inheritable by children processes by exporting them:
$ export ABC=123
$ c
$ echo $ABC

Also, you can concatenate a string to an existing variable:
# export PATH=$PATH:/home/fcaen

To list your bash’s environment variables use:
# env

When you go to create your own environment variables, avoid using names that are already commonly used by the system for environment variables. See Figures  above for a list of shell environment variables.

Creating Simple Shell Scripts
Shell scripts are good for automating repetitive shell tasks.
Bash and other shells include the basic constructs found in various programming languages, such as loops, tests, case statements, and so on. The main difference is that there is only one type of variable: strings.

Editing and Running a Script
Shell scripts are simple text files. You can create them using your favorite text editor (such as vi). To run, the shell script file must be executable. For example, if you created a shell script with a file name of, you could make it executable as follows:
$ chmod u+x

Or, instead of making it executable, you could precede the script with the bash command to run it (bash
    Also, the first line of your bash scripts should always be the following:
As with any command, besides being executable the shell script you create must also either be in your PATH or be identified by its full or relative path when you run it. In other words, if you just try to run your script, you may get the following result:
bash: command not found

In this example, the directory containing is not included in your PATH.
To correct this problem, you can
  • edit your path, 
  • copy the script to a directory in your PATH, or 
  • enter the full or relative path to your script as shown here:

$mkdir ~/bin ; cp ~/bin/ ; PATH=$PATH:~/bin
$cp /usr/local/bin

Avoid putting a dot (.) into the PATH to indicate that commands can be run from the current directory. This is a technique that could result in commands that have the same file name as important, well-known commands (such as ls or cat). The proper file could end up being overridden, if a command of the same name exists in the current directory.

Adding Content to Your Script
Although a shell script can be a simple sequence of commands, shell scripts can also be used as you would any programming language. For example, a script can produce different results based on giving it different input. here i describe how to use compound commands, such as if/then statements, case statements, and for/while loops in your shell scripts.
    The following example code assigns the string abc to the variable MYSTRING. It then tests the input to see if it equals abc and acts based on the outcome of the test. The test is the section that takes place between the brackets ( [ ] ):
if [ $MYSTRING = abc ] ; then
echo “The  variable is abc”

To negate the test, use != instead of = as shown in the following:
if [ $MYSTRING != abc ] ; then
echo “$MYSTRING is not abc”;

The following are examples of testing for numbers:
if [ $MYNUMBER -eq 1] ; then echo “MYNUMBER equals 1”; fi
if [ $MYNUMBER -lt 2] ; then echo “MYNUMBER <2”; fi
if [ $MYNUMBER -le 1] ; then echo “MYNUMBER <=1”; fi
if [ $MYNUMBER -gt 0] ; then echo “MYNUMBER >0”; fi
if [ $MYNUMBER -ge 1] ; then echo “MYNUMBER >=1”; fi

Let’s look at some tests on file names. In this example, you can check whether a file exists (-e), whether it’s a regular file (-f), or whether it is a directory (-d). These checks are done with if/then statements. If there is no match, then the else statement is used to produce the result.
if [ -e $filename ] ; then echo “$filename exists”; fi
if [ -f “$filename” ] ; then echo “$filename is a regular file”
  elif [ -d “$filename” ] ; then echo “$filename is a directory”
      echo “I have no idea what $filename is”

Table 3-1 shows examples of tests you can perform on files, strings, and variables.

Another frequently used construct is the case command. Using the case statement, you can test for different cases and take an action based on the result. Similar to a switch statement in programming languages, case statements can take the place of several nested if statements.
case “$VAR” in
      { action1 };;
      { action2 };;
      { default action } ;;

You can find examples of case usage in the system start-up scripts found in the
/etc/rc.d/ directory. Each initscript takes actions based on what parameter was passed to it and the selection is done via a large case construct.
    The bash shell also offers standard loop constructs, illustrated by a few examples that follow.
In the first example, all the values of the NUMBER variable (0 through 9) appear on the for line:
for NUMBER in 0 1 2 3 4 5 6 7 8 9
   echo The number is $NUMBER

In the following examples, the output from the ls command (a list of files) provides the variables that the for statement acts on:
for FILE in `/bin/ls`; do echo $FILE; done
Instead of feeding the whole list of values to a for statement, you can increment a value and continue through a while loop until a condition is met. In the following example, VAR begins as 0 and the while loop continues to increment until the value of VAR becomes 3:
while [ $VAR -lt 3 ];
    echo $VAR

Another way to get the same result as the while statement just shown is to use the
until statement, as shown in the following example:
until [ $VAR -eq 3 ]; do echo $VAR; VAR=$[$VAR+1]; done



  • Bash Guide for Beginners: If you are just starting with shell programming. Use that guide, along with reference material such as the bash man page, to step through many examples of shell scripting techniques.
      •  BSD UNIX TOOLBOX 1000+ Commands for FreeBSD, OpenBSD, and NetBSD®Power Users (2008 by Wiley Publishing) by Christopher Negus,   François Caen ISBN: 978-0-470-37603-4


        1. I like the short intro to the shell, but I am a little sad that you introduce xargs and the for loop as the most common use for these are better solved with GNU Parallel

          Here are a few examples:

          ls /usr/local/bin/mk* | parallel -v pkg_info -W

          seq 0 9 | parallel -k echo The number is {}

          ls | parallel echo {}

          They are easy to read and will often make better use of multi-core CPUs.

          I invite you to watch the intro video to GNU Parallel:

        2. @tange Thanks for your comment. The post is an excerpt from a book just now. Later i revision it.