------------------------------------------------------------------------------- Hints and Tips for general shell script programing ------------------------------------------------------------------------------- Where is this script located WARNING: this will fail if the user is playing with $0 For example using a symbolic or hard link with a unexpected name. # Simplest... # PROGNAME=`type $0 | awk '{print $3}'` # search for executable on path PROGNAME=`basename $PROGNAME` # base name of program ----- # Advanced... # Script name, in what directory, and in what directory is user # running the script from. # # Discover where the shell script resides ORIGDIR=`pwd` # original directory (builtin) PROGNAME=`type $0 | awk '{print $3}'` # search for executable on path PROGDIR=`dirname $PROGNAME` # extract directory of program PROGNAME=`basename $PROGNAME` # base name of program cd $PROGDIR # go to that directory PROGDIR=`pwd` # remove any symbolic link parts cd $ORIGDIR # return to original directory (opt) Results... $ORIGDIR -- where the users was when called $PROGDIR -- script directory location (and now current directory) $PROGNAME -- This scripts executable name A T/CSH version of the above is also available (Email me - anthony) ------------------------------------------------------------------------------- Shell Script Option Handling PROGNAME=`basename $0` # Or the above script locator Usage() { echo >&2 "$PROGNAME:" "$@" echo >&2 "Usage: $PROGNAME [options] [file...]" exit 10 } #!/bin/sh # # script [options] args... # # The frist 'Usage()' code that follows these comments locates the script on # disk, and then reads and output these comments as the documentation for this # script. That is the script commands and usage documention are in the same # file, making it self documenting, via options. # ### # # programmers only docs, whcih are not output by Usage() # PROGNAME=`type $0 | awk '{print $3}'` # search for executable on path PROGDIR=`dirname $PROGNAME` # extract directory of program PROGNAME=`basename $PROGNAME` # base name of program Usage() { # output the script comments as docs echo >&2 "$PROGNAME:" "$@" sed >&2 -n '/^###/q; /^#/!q; s/^#//; s/^ //; 3s/^/Usage: /; 2,$ p' \ "$PROGDIR/$PROGNAME" exit 10; } # # Generalized Otion handling. # while [ $# -gt 0 ]; do case "$1" in # Standard help option. --help|--doc*) Usage ;; # Simple flag option -d) DEBUG=true ;; # Word flag option -debug) DEBUG=true ;; # Simple option and argument EG: -n name -n) shift; name="$1" ;; # Option and argument joined EG: -Jname -J) name=expr "$1" : '-.\(.*\)'` || Usage "Option \"$1\" missing required value" ;; # Joined OR unjoined Argument EG: -Nname or -N name -N*) Name=`expr "$1" : '-.\(..*\)'` || { shift; Name="$1"; } ;; # As last, but if -b0 is an posibility then you need to use this instead. # Otherwise it will think the value is unjoined when it isn't Arrrgghhhh... -b*) bits=`expr "$1" : '-.\(..*\)'` [ "$bits" ] || { shift; bits="$1"; } ;; # Numbers with type checks EG: -{width}x{height} -[0-9]*x[0-9]*) w=`expr "$1" : '-\([0-9]*\)x'` || Usage "Bed Geometry" h=`expr "$1" : '-[0-9]*x\([0-9]*\)$'` || Usage "Bad Geometry" [ "$width" -eq 0 -o "$height" -eq 0 ] && Usage "Zero Geometry" geometry="${w}x${h}" ;; # Number with type check EG: -{size} -[0-9]*) size=`expr "$1" : '-\([0-9]*\)$'` || Usage [ "$size" -eq 0 ] && Usage Width=$size; Height=$size ;; # Generalised Argument Save EG: -G value => $opt_G -*) var=`expr "$1" : '-\(.\).*'` case "$var" in [a-zA-Z]) arg=`expr "$1" : '-.\(..*\)'` || { shift; arg="$1"; } eval "opt_$var"="\"\$arg\"" ;; *) Usage "Bad Non-Letter option \"$1\"" ;; esac ;; -) break ;; # STDIN, end of user options --) shift; break ;; # end of user options -*) Usage "Unknown option \"$1\"" ;; *) break ;; # end of user options esac shift # next option done # Handle normal arguments now... Files=${1+"$@"} # or [ $# -gt 0 ] && Usage "To many Arguments" ------------------------------------------------------------------------------- Variable testing under shell (rules of thumb) Boolean Variable tests... true="true"; false="" [ "$var" ] && echo var is true [ -z "$var" ] && echo var is false Test of variables containing ANY string (see PROBLEM CASES below) option=-xyzzy [ "X$option" != X ] && echo option is defined [ "X$a" = "X$b" ] && echo "$a equals $b" Rules of thumb... * Always quote all variables being tested * Only use the boolean type test (above) when all posible values are defineately known. * Prepend X (or other alphanumberic) to unknown strings being compared * Don't use ! if you can avoid it. PROBLEM CASES... These cases cause the "[...]" tests to fail badly! [ $var ] but var="a = b" [ "$var" ] but var=-t (actually any option starting with '-') [ "$a" = "$b" ] but a='(' and b=')' This is why you must use the string test above (with 'X' prefixes). NOTE test ! ... differs from UNIX to UNIX Under bash it is the NOT of the next 3 arguments following Under solaris "sh" it seems to handled with precedence. EG: test "" -a ! "" is false as you would expect BUT test ! "" -a "" is true for BASH and false for Solaris SH ------------------------------------------------------------------------------- Argument Sorting By Shells All Shells normally sort the output of any commands with meta characters. Also "ls" will sort its arguments again internally, while "echo" will not. EG compare the output of these commands ls -d b* a* echo b* a* echo [ba]* The first sorts the output so a's are before b's, the second does not. However the shell sorts the arguments of the third so a's are again first. However the "{}" meta characters do NOT sort the results... echo {b,a}* will output b's first then a's. This can be useful when the order of the arguments is important. This has been tested and works as described in csh, tcsh, bash, and zsh ------------------------------------------------------------------------------- Argument handling... If you care for the possibility that there aren't any arguments. "$@" -> "" (one empty argument) ${1+"$@"} -> nothing at all This is not a problem with newer modern shells, only very old Bourne shells. WARNING: Command line shell argument start at 0! sh -c 'echo $*' 1 2 3 4 5 or sh -c 'echo $*' `seq 5` Will print 2 3 4 5 Note that the first arguement is missing. That is because it was assigned to argument $0 and not $1 or the $@ and $* variables! To use all the arguments on the shell command line, use either sh -c 'echo $*' junk `seq 5` or sh -c 'echo $0 $*' `seq 5` the later is not recomended as if no arguments are provided $0 defaults to "sh" instead of the empty string! This is a postix compliant feature, $0 being the program name when a shell script is called. the "-c" command line script just layers on top of this. this is the case with solaris bourne sh, bash, ksh, and zsh. csh and tcsh shell does not have an equivelence. See "find" and its "-exec" option, (see below), for a useful example of this. I also use it in csh aliases to provide a default argument. EG: alias xvd 'sh -c '\''cd ${1:-.}; xv -vsmap &'\'' junk !:* &' ------------------------------------------------------------------------------- Auto change shells on brain dead systems (Ultrix) #!/bin/sh - # # Check the type of shell that is running (For Ultrix) [ "X$1" != 'X-sh5' -a -f /bin/sh5 ] && exec /bin/sh5 -$- "$0" -sh5 "$@" [ "X$1" = 'X-sh5' ] && shift # ------------------------------------------------------------------------------- Getting environment from csh env - DISPLAY=$DISPLAY HOME=$HOME TERM=dumb \ csh -cf 'set prompt="> "; source .cshrc; source .login; echo "#------ENVIRONMENT------" env' | sed -n '/#------ENVIRONMENT------/,$p' ------------------------------------------------------------------------------- Inserting a AWK or PERL script inside a SHELL script... Example... nawk '# output to mailx commands! /^[^ ]/ { recipent=$0; } /^ / { print $0 > "|mailx -s \"This is the test\" " recipent; } /^$/ { close( "|mailx -s \"This is the test\" " recipent ); ' list.txt Note the whole script is inside single quotes on the nawk command line! ASIDE: old versions of awk must have something on the first line thus the addition of the # comment to keep it happy! Perl needs no such comment but does require a -e option to execute a command line argument. To insert a external shell variable into the script you need to close the single quotes, output variable and re-open the single quotes. Also the variable sould be in double quotes so as to prvent any insertion of space characters Example inserting a $prefix shell variable into a awk string. ... { print "'"$prefix"'" $0; } ... Also to insert a single quote into the script you have to also exit the wrapping single quotes and supply it outside those quotes ... { print "I just can'\''t do that!"; } ... CAUTION: Watch for single quotes inside any COMMENTS which is in the script! Comments are within the single quotes so are also scanned for thos quotes. CSH SCRIPTS: If you must write a csh script you will need to escape the new line at the end of every line, even though single quotes are being used, which requires to backslashes.. Also watch out for history escapes `!' which work inside single quotes! ------------------------------------------------------------------------------- Is COMMAND available There is two techniques available to test if a command is available. The first is to use the `status' return of the "type" or "which" command of the shell you are using. The Second is to examine the output of that command itself. Using status return. This is a simple method of testing the existance of a command but DOES NOT WORK ON ALL SYSTEMS! The problem is that old shells (For Example: SunOS bourne-sh) always returns a true status weather the command is present or not! Bourne Shell if type COMMAND 2>&1; then # COMMAND is available fi C-Shell Warning: The "which" command in C shell is not a builtin but a external script which sources your .cshrc (YUCK). As such the bourne shell alias is prefered which will only search your current command PATH. if ( ! $?tcsh ) then alias which 'sh -c "type \!:1 2>&1"' endif if ( -x "`which COMMAND >/dev/null`" ) then # COMMAND Available endif TC-Shell Tcsh 6.06 also does not return the correct status in its which command. Use it like the csh which above. WARNING: this method will also test positive for :- subroutines, bash aliases, and probably other non-command definitions. Examine output The other am more reliable method is to examine the output of the "type" or "which" command to looks for the string "not found" (See below). This is more complex than the above status check but should work on ALL unix machines regardless of age. Bourne Shell cmd_found() { case "`type $1 2>&1`" in *'not found'*) return 1 ;; esac; return 0 } ... if cmd_found COMMAND; then # COMMAND is available fi C-Shell & Tcsh (See notes below) if ( ! $?tcsh ) then alias which 'sh -c "type \!:1 2>&1"' endif ... if ( "`which less`" !~ *'not found'* ) then # COMMAND Available endif NOTES for "which/type" commands :- The above methods look for the specific string "not found" This is important as the sh type command and tcsh which command produce different output, and this may also vary from bourne shell to bourne shell or other shell types. Csh -- "which" is an unreliable shell script! fudge it into a shell script "type" command. See the "Which Problem" below. Tcsh > which less /opt/bin/less > which junk junk: Command not found. > which which which: shell built-in command. > alias a alias > which a a: aliased to alias Solaris Bourne shell > type less less is /opt/bin/less > type junk junk not found > type type type is a shell builtin > func(){ echo ha; } > type func func is a function func(){ echo ha } Solaris Ksh As per Sh, but the actual function definition is NOT listed Bash > type less less is /opt/bin/less > type junk bash: type: junk: not found > type type type is a shell builtin > func(){ echo ha; } > type func func is a function func () { echo ha } NOTE: bash also has a type -t which responds with a single word "file", "alias", "function", "builtin", "keyword", or nothing if command does not exist. A -p will print the disk file name, or nothing. A -a prints all the places that have that name. From the results above, only the appearence of "not found" in the false statement is consistant. But only when the result is not a bourne shell function, which presumably replaces the real-command of that name. The Expanded Bourne shell form, without using the "cmd_found" function is as follows, But is is a low simpler and easier to read if you use the funtion. If command present if expr match "`type COMMAND 2>&1`" '.*not found' == 0 >/dev/null; then # COMMAND is available fi and its inverse (not present) if expr match "`type COMMAND 2>&1`" '.*not found' >/dev/null; then # Comand is NOT present fi finally only using built-in commands... case "`type COMMAND`" in *'not found'*) # Command not found ;; *) # Command found ;; esac Functional forms cmd_found() { expr match "`type $1 2>&1`" '.*not found' == 0 >/dev/null } OR cmd_found() { case "`type $1 2>&1`" in *'not found'*) return 1 ;; esac; return 0 } ------------------------------------------------------------------------------- Which problem! The which command is a csh script that specifically reads the .cshrc file to find out about aliases. To avoid having .cshrc do lots of odd things to your script, use the following form. set program = `/bin/env HOME= /usr/ucb/which $program` This is NOT a problem in Tcsh, where "which" is a built-in. The best solution it to replace "which" with the bourne shell "type" command in C shells but leave it alone for TC shells. if ( ! $?tcsh ) then alias which 'sh -c "type \!:1 2>&1" | sed "s/.* is //"' endif set program = `which $program` ------------------------------------------------------------------------------- builtin cat command This cat only uses the shell builtins! As such can be used on a machine which has no access to shared libraries and nothing but the statically linked bourne sh can run. shcat() { while test $# -ge 1; do while read i; do echo "$i" done < $1 shift done } Of course the real cat command is only as big as it is, to protect the user from himself and to provide a huge number of options. PS: If the ls command is also not available then you can use echo * to do a directory listing using only builtins. Though this will not tell you what files are executables or sub-directories. ------------------------------------------------------------------------------- Note `cmd` does save the newline characters within the output ::::> m=`mount`; ::::> echo $m / /usr /home ::::> echo "$m" / /usr /home In other words outside quotes newlines in the input are treated purely as white space between arguments and thus ignored. Inside quotes newlines are retained as newlines. ------------------------------------------------------------------------------- One line if-then-else shell command cmd1 && cmd2 || cmd3 This hoever will execute cmd3 if cmd2 fails! If cmd2 never fails -- fine ------------------------------------------------------------------------------- The useless use of 'cat'! You often see in shell scripts... cat input_file | some_command The cat is usless as it is exactly the same as some_command < input_file without needing to fork the extra "cat" process or creating a pipeline. However it is sometimes usefull to do anyway to make the code more readable. Particularly in long pipelines. ------------------------------------------------------------------------------- Cat with here file and pipe to another command (Shell Syntax Example) cat <; chomp @a; print_cols \@a' Otherwise manually handle columns in perl seq -f %03g 100 |\ perl -e ' @a=<>; chomp @a; \ my ( $c, $i ) = (10,10); \ foreach ( @a ) { \ print("\n"),$i=$c unless $i; \ printf " %4s", $_; $i--; \ } print "\n"; ' ------------------------------------------------------------------------------- Suppressing the shell background fork message (*csh) The trick is to redirect the standard error output of the _shell_ itself Korn (with loss of standard error) ( command & ) 2>/dev/null csh/tcsh (this appears to work) ( command & ) bourne shell does not give a message about background forks ever. ------------------------------------------------------------------------------- Auto Background a shell script #!/bin/csh -f if ( $1:q != '...' ) then ( $0 '...' $*:q & ) exit 0 endif shift ...rest of script to run in background... OR #!/bin/sh foreground stuff ( background stuff ) & exit ------------------------------------------------------------------------------- Sort a shell array =======8<-------- i=0 for n ; do   a[$((i++))]=$n ; done # sort array for (( i=0; $i<(${#a[@]}-1); i++ )) ; do   for (( j=i+1; $j<(${#a[@]}); j++ )) ;   do     if [[ ${a[$j]} < ${a[$i]} ]] ;     then       t=${a[$j]};       a[$j]=${a[$i]};       a[$i]=$t;     fi   done done # launch xmms with sorted list xmms -e "${a[@]}" =======8<-------- ------------------------------------------------------------------------------- Command timeout Run a command but kill it if it runs for too long. This prevents scripts hanging on a command, especially network related commands, like nslookup. Simple C solution... Look at ~/store/c/programs/timeout.c which will timeout a command simply and easilly. A lot better than the complex solutions below. ---- Attempt 1 ---- Runs the command in the background and waits for it to complete. A sleep command is also run to provide a timeout. Works but full timeout period is always waited before exiting TIMEOUT=60 # timelimit for command ( command_which_can_hang & sleep $TIMEOUT; kill $! 2>/dev/null ) This does not work. We will wait on the sleep for the full TIMEOUT period regardless of how fast the command completes. If the command is known to never finish then the above will be fine! ---- Attempt 2 ----- Works for modern day shells... do_quota() { # lookup the users disk quota but with a timeout quota -v "$@" & # now ensure that the above command does not run too long cmd_pid=$! ( sleep $QUOTA_TIMEOUT echo >&2 "Quota Timeout" kill -9 $cmd_pid 2>/dev/null ) & kill_pid=$! wait $cmd_pid kill $kill_pid 2>/dev/null } The problem here is that this works fine but the sleep will still continue for the full timeout period. Basically the parent has no simple way of determining the PID of the sleep, to abort it, if it was not needed. This is not a problem if the sleep itself is not too long, so it doesn't stick around for hour. The sub-shell however is killed, so the kill command is not run if the command completes before the timeout Unfortunatally BASH generates a "terminate" message on standard out when the command is killed due to timeout :-( Full Example try with/without a timeout argument of 15... =======8<-------- #!/bin/sh # # Countdown [abort_after] # COUNTDOWN_FROM=10 # count down from this value COUNTDOWN_ABORT=${1:-4} # abort countdown after this long countdown() { i=$COUNTDOWN_FROM echo "Countdown continuing at T minus $i" while [ $i -gt 0 ]; do sleep 1 i=`expr $i - 1` [ $i -eq 3 ] && echo "Main engine start" \ || echo "--> $i" done echo "We have lift off!!!" exit 0; } countdown & # the command that we wish to timeout cmd_pid=$! ( sleep $COUNTDOWN_ABORT; kill $cmd_pid echo "countdown aborted"; ) & kill_pid=$! wait $cmd_pid echo "Launch status: $?" kill $kill_pid 2>/dev/null #/bin/ps # uncomment to show if sleep is still running =======8<-------- ---- Attempt 3 ---- Reports a `Terminate' error to stdout if command timesout. TIMEOUT=60 # timelimit for command command_which_can_hang & cmd_pid=$! sleep $TIMEOUT | ( read nothing # wait on sleep to finish kill $cmd_pid 2>/dev/null; # otherwise abort the command! ) & sleep_pid=$! wait $cmd_pid kill -ALRM $sleep_pid 2>/dev/null # kill sleep if still running The reason this works is that we are specifically killing the sleep (we hope). This is because $! returns the pid of the first command in a command pipeline, so we fake such a pipeline. WARNING: Some shells may have $! set to the last command in pipeline! which make this equivelent to "Attempt 2" though will also work as expected. Using an ALRM signal to kill sleep is especially good as it means sleep exits normally and cleanly, stopping a shell like "bash" from producing "Terminated" messages all over the place. In the "kill command" sub-shell, the "read" is needed to ensure it waits for non-existant output from the sleep command. It will automatically exit when the sleep exits without producing any output. A "kill -0 $pid" could be used to check if the pid given is still running or not, and in a full working version is probably a good idea. I suppose we could also backgound the sleep, and have a background kill process wait specifically on the background sleep process but this seems to complex to me.... It also does NOT work as you wither have to put a a wait for the sleep the sub-shell (or a looped pid poll) which did NOT launch the sleep and thus does not know its pid, or the background sleep in the sub-shell, in which case the main script did not launch it so can't abort it. Very difficult to sort that out. Better to use a command pipeline. I supose you could use a `SIGCHLD trap' but that is a completely different method. If anyone have created such a trap in shell, please email me your code, so I can include it here (with your name of course). Full example.. TIMEOUT=20 # Limit name server lookup to 20 seconds # ... do_nslookup() { # Do a FAST nslookup - even if primary namserver is down # Background the required command - note the process id # any output to be printed to standard argument and calling function nslookup 2>/dev/null <<-EOF | sed -n '/arpa/s/.*name \= //p' & set type=PTR $1 EOF cmd_pid=$! # # Wait for it (sleep pipeline) - note sleep's process id sleep $TIMEOUT | ( read nothing # wait on sleep to finish kill -0 $cmd_pid 2>/dev/null || exit; # cmd finished? - exit sub-shell kill $cmd_pid 2>/dev/null; echo >&2 "WARNING: nslookup failed to return in $TIMEOUT seconds!" ) & sleep_pid=$! # # wait for nslookup to finish or be killed wait $cmd_pid kill -0 $sleep_pid 2>/dev/null || return; # sleep finished? kill $sleep_pid 2>/dev/null } # ... Reverse_IP_Arg="4.3.2.1.in-addr.arpa" FQDN_hostname=`do_nslookup $Reverse_IP_Arg` ------------------------------------------------------------------------------- Loop until parent process dies Background tasks have an annoying habit of continuing to run AFTER you have logged out. This following example program looks up the launching parent process (typically your login shell) and only loops if the parent process is still alive. WARNING: The PS command varies from UNIX system to UNIX system so you will have to tweek the arguments to the `ps' command to make this script work on your UNIX system =======8<-------- #!/bin/sh # # Loop until parent dies # sleep_time=300 # time between background checks # Pick the appropriate ps options for your UNIX system # Uncomment ONE of the following lines #job_opt=xj; sep=""; ppid=1; # SunOS job_opt=xl; sep=" "; ppid=4; # Solaris #job_opt=xl; sep=" "; ppid=4; # IBM UNIX (aix) #job_opt=xl; sep=" "; ppid=4; # SGI UNIX (irix) # Discover the parents process ID set - `ps $job_opt$sep$$ | tail -n+2` "1" eval parent=\$$ppid # While parent is still alive # The kill command "-0" option checks to see if process is alive! # It does NOT actually kill the process (EG: test process) while kill -0 $parent 2>/dev/null; do # ... # Do the background job here # ... sleep $sleep_time done # Parent process has died so we also better die. exit 0 =======8<-------- Also see the script ~anthony/bin/scripts/hostspace as an example of a shell script for multiple hosts ------------------------------------------------------------------------------- Convert ls permissions to octal format ... anyone? ... ------------------------------------------------------------------------------- Time in Seconds, NOW! (for later comparision) EG: getting the number of seconds since epoch (midnight, 1/1/1970, GMT) This is for the purposes of timing various command and comparing the start and end times. The problem is as there is no standard way for geting the system clock under normal plain shell scripts. Under linux (Gnu-date) you can date +%s With perl you can perl -e 'print time(), "\n"' This seems to work very well on all system I know of, and can be used for sequencing events. Do not rely on it for time deltas (how long) as it does not handle years (%j is day within the year) date +'%j * 86400 + %H * 3600 + %M * 60 + %S' | bc Convert Time since epoch to Date (for more see the perl/general.hints page) perl -e 'require "ctime.pl"; print &ctime(1000000000); ------------------------------------------------------------------------------- Get the date of yesterday (without a lot of fuss) Getting yesterdays day is usually best done in either C or perl. #!/usr/bin/perl require "ctime.pl"; print &ctime($^T - 60*60*24); By changing your timezone you can fake the getting of yesterdays date, But ONLY if you live in Australia where it is already tomorrow!. It is not perfect (2 hours short but close enough in most cases) env TZ=GMT-12 date ------------------------------------------------------------------------------- Four Digit Year in shell (for logfiles) For Newer UNIX Machines, IEEE date format date=`date +'%Y-%m-%d %R:%S'` For Older UNIX machines (SunOS)... While the newest UNIX's allow a %Y for a four digit year older machines like SunOS does NOT. This means the year will need to be extracted from the date output (or auturnative format code). year=`date | sed 's/.*\([0-9][0-9][0-9][0-9]\).*/\1/'` date=`date $year-%m-%d %H:%M:%S" If you have the MH mail system install you can convert the date to rfc822 date format, use the "dp" program. This has its own mh output format codes. /usr/lib/mh/dp "`date`" or /usr/lib/mh/dp -format '%(year{text})' "`date`" or /usr/lib/mh/dp \ -format '%04(year{text})-%02(mon{text})-%02(mday{text})' \ "`date`" ------------------------------------------------------------------------------- Setting a timed alarm in shell This can be done (as a background task) exactly how is an another matter As an educated guess probably something like.. # Trap the USR1 signal trap "do timeout commands" 16 ( sleep $timeout; kill -16 $$; ) & See also "Command timeout" above which was a later addition to this file. ------------------------------------------------------------------------------- Am I a Non-Interactive Shell bourne sh: if [ -z "$PS1" ]; then # script/remote execution (non-interactive) csh: if ( ! $?prompt ) then # script/remote execution (non-interactive) In the bourne shell a better way is to test the shell options ``$-'' for the interactive flag directly. case $- in *i*) ;; # do things for interactive shell *) ;; # do things for non-interactive shell esac ------------------------------------------------------------------------------- Merge multiple blank lines into one line. This is not easy, as we want to preserve a blank line, removing extras. Two methods, print paragraphs, or delete extra blanks. awk '{ printf "%s ", $0 } NF == 0 { print "\n" }' filename cat -s # will do this (if available) perl -ne 'if (/\S/) { print; $i=0 } else {print unless $i; $i=1; }' perl -ne 'print if /\S/../^\s*$/' sed '/./,/^$/!d' # NOTE: (t)csh users must backslash the ! sed '/[^ ]/,/^[ ]*$/!d' # For blank lines with spaces and tabs # Line joining in the vim editor! as a macro :map QE :$s/$/\\rZ/:g/^[ ]*$/,/[^ ]/-jGdd or :%s/\n\s*\n\(\s*\n\)*/\r\r/ ------------------------------------------------------------------------------- LIST functions (Also usable for PATH-like environment variables) list="list_variable" # a:b:c:d element="element_of_list" # e sep="list_seperator" # : Append to list (even if empty!) list="${list:+$list$sep}$element" Split up list function split_path () { ( IFS="$sep" set -- $list for f in "$@"; do echo -n "${f:-.} " done; echo ) } Count of elements count=`IFS="$sep"; set - $list; echo $#` get I'th element (shell must understand shift argument) element=`IFS="$sep"; set - $list; shift $I; echo $1` get first element element=`IFS="$sep"; set - $list; echo $1` delete first element list=`echo "$list" | sed "/$sep/!d; /:/s/^[^$sep]*$sep//;"` delete specific element over the whole list list=`echo "$list" | sed "s|${element}${sep}||g; s|${sep}${element}\$||;"` ------------------------------------------------------------------------------- Checking for a numerical value case "$var" in '' | *[!0-9]*) echo "non-numeric" ;; *) echo "numeric" ;; esac # --- Nick Holloway (alfie@dcs.warwick.ac.uk) OR echo "$arg" | egrep '^[0-9]+$' >/dev/null || echo "not-numeric" Get value from option (NOT zero) expr "$var" : '\([0-9]*\)$' || echo "not a non-zero-numeric value" ------------------------------------------------------------------------------- random number generation in a shell nawk: set range = ???? set random = `nawk 'BEGIN { srand(); printf "%d", rand()*'$range' }' /dev/null` date: set range = ???? # file length -- `wc -l <$file` set date = `date +%j%H%M%S` set random = `expr $date \% $range` (k/z/ba)sh $RANDOM Posible improvment... !/bin/bash MX="0123456789" NL="5" # size of the random number wanted while [ ${n:=1} -le $NL ] do NUM="$NUM${MX:$(($RANDOM%${#MX})):1}" let n+=1 done echo "$NUM" Programed (csh) (could be via expr too) set multiplier = 25173 set modulus = 65536 set increment = 13849 set seedfile = $HOME/.rnd # seed file to use if ( ! -f $seedfile ) then echo '17' > $seedfile endif @ number = ( `cat $seedfile` * $multiplier + $increment ) % $modulus echo $number > $seedfile @ number = $number % $range echo $number ------------------------------------------------------------------------------- increment a character in shell char=`echo $char | tr ' -~' '\!-~'` or (sutable for hex chars) str1="ABCDEFGHIJKLMNOPQRSTUVWXYZ" str2="BCDEFGHIJKLMNOPQRSTUVWXYZA" pos=`expr index $str1 $char` char=`expr substr $str2 $pos 1` echo $char In perl the "increment alpha string" makes this easy char=abzzz char=`perl -e '$n="'"$char"'"; print ++$n'` echo $char acaaa ------------------------------------------------------------------------------- Protecting shell scripts from ^Z This signal can't normally be stoped in a shell, the trick is to change key generating the signal (Don't forget to return it to normal). stty susp undef -- if available stty susp '^-' -- maybe system dependant ------------------------------------------------------------------------------- Curses in shell script To use termcap entries in a shell script use the `tput' command EXAMPLES /usr/5bin/tput bold # bold (extra half brigtht mode) /usr/5bin/tput bink # bink mode (if available) /usr/5bin/tput rev # reverse video /usr/5bin/tput smul # underline mode /usr/5bin/tput smso # standout (usually reverse) /usr/5bin/tput rmso # end standout (return to normal) /usr/5bin/tput clear # clear screen /usr/5bin/tput cup 5 23 # cursor to move to row 5 column 23 See terminfo(5) for more info. ------------------------------------------------------------------------------- Split pipe into two separate commands (executable tee) awk '{ print | "'cmd1'" ; print }' | cmd 2 ------------------------------------------------------------------------------- Capitalize the first word. The initial solutions in the news group cam out to more than 10 lines of code! # Ken Manheimer (expr-tr-expr) NOTE: fails if word="match" Word=`expr "$word" : "\(.\).*" | tr a-z A-Z``expr "$word" : ".\(.*\)"` # Paul Falstad (cut-tr-sed) Word=`echo $word|tr a-z A-Z|cut -c1``echo $word|sed s/.//` ==> # Logan Shaw (cut-tr-cut) Word=`echo "$word" | cut -c1 | tr [a-z] [A-Z]``echo "$word" | cut -c2-` # Harald Eikrem (sed only) Word=`echo $word | sed -e ' h; 'y/abcdefghijklmnopqrstuvwxyz/ABCDEFGHIJKLMNOPQRSTUVWXYZ/; G; 's/\(.\).*\n./\1/; ' ` # Tom Christiansen (perl) Word=`echo $word | perl -pe 's/.*/\u$&/'` # Jean-Michel Chenais # (korn shell built-ins only) typeset -u W;typeset -l w;W=${word%${word#?}};w=$W;Word=$W${word#${w}} # perl of course makes this easy perl -e 'print "\u'"$word"'"' ------------------------------------------------------------------------------- Multi-line Paragraph handline Most text files including this one consists of paragraphs of multiple lines seperated by a blank line. Convert all paragraphs into a single line (blank line seperated) sed '/^$/d; :loop y/\n/ /; N; /\n$/! b loop; s/ */ /g; s/^ //; s/ $//' Also remove blank lines between paragraph lines sed '/^$/d; :loop N; s/\n$//; T loop; y/\n/ /; s/ */ /g; s/^ //; s/ $//' WARNING: better space handling is probably needed, especially for a last paragraph that has no final blank line after it. ------------------------------------------------------------------------------- Find a Process of a particular name --- see csh alias pf Usign grep ps auxgww | grep "$NAME" | grep -v grep | cut -c1-15,36-99 Merging the two greps.. ps uxw | grep "[s]sh-agent" | awk '{print $2}' The "[s]sh-agent" will not match the grep process itself! EG: it will not match the "[s]sh-agent" string in the grep process Using awk... ps auxgww | awk "/$NAME/ && \! /(awk)/" | cut -c1-15,36-99 or for a exact process name ps auxgww | awk '/(^| |\(|\/)$NAME( |\)|$)/' | cut -c1-15,36-99 or alturnativeally which matches under a lot of conditions... EG: matches :01 NAME arg :01 some/path/NAME :01 NAME: ps auxgww | \ awk '/:[0-9][0-9] (|[^ ]*\/)$NAME($| |:)/' | cut -c1-15,36-99 Perl version (also matches on username) ps auxgww | \ perl -nle 'print if $. == 1 \ || /^\s*\!:1\s/o \ || /:\d\d (|\[ *|[^ ]*\/)\!:1($|[]: ])/o; ' As you can see things can get complex very quickly ------------------------------------------------------------------------------- Context Grep Or how to display lines before/after search pattern. GNU grep, has context built in to it, check out the -A, -B, and -C options. otherwise grep -v pattern file | diff -c3 - file | grep '^. ' | colrm 1 2 or grep -n $1 $2 | awk -F: '{ print $1 }' | while read linenum do awk 'NR>target-5 && NR&/dev/null /dev/null 2>&1 <&1 &' Either Shell or other shell rsh machine -n '/bin/sh -c "exec command >/dev/null 2>&1 <&1" &' -------------------------------------------------------------------------------