About Me

The best place to start learning Oracle, Linux/Unix scripting, and Oracle administration skills. Currently, as Oracle DBA with 11g/10g/9i OCP certified, I want to help you with my background and knowledge. Hopefully, this info would help you as a job aid.
Showing posts with label Unix Linux commands scripting. Show all posts
Showing posts with label Unix Linux commands scripting. Show all posts

Friday, December 16, 2011

Sar command in Linux, retrieving cpu, memory information

'sar' command in linux is a command to return CPU usage and memory usage information.
You can issue repeated option to keep it running returning the stats with specified times.
Ex: >sar 5 150
this will return the result every 5 seconds for 150 times.
some example out put is below:
21:00:56          CPU     %user     %nice   %system   %iowait    %steal     %idle
21:01:11          all         2.21        0.00      0.85          6.13          0.00         90.81
21:01:26          all         3.71        0.00      1.35          3.03          0.00         91.91
21:01:41          all         0.93        0.00      0.68          4.33          0.00         94.05
21:01:56          all         0.88        0.00      0.77          4.41          0.00         93.94
21:02:11          all         0.83        0.00      0.63          4.15          0.00         94.39
Average:          all       15.29        0.00      4.77        32.46          0.00         47.48
Here is what you can do to extract specific time interval from sar data file:
Sar puts its data in daily logs in /var/log/sa. So for instance, if you want to extract from /var/log/sa/sa16 file, use the –f option and then –s for start and –e for end in hh:mm:ss format:

> sar -f sa16 -s 19:00:01  -e 20:27:01

It will display to standard out and the last line is always the Average. If you need to span across two files, you can either append one to the other making a new file (watch for any extraneous lines for averages), or you can get data from each file and average them together. Hope this helps.

Saturday, November 5, 2011

KSH - Korn Shell Tutorial

KSH - Korn Shell Tutorial
  • Matching Patterns
  • Conditional Statements
  • Test Objects (Files, Directories, etc.)
  • Format of flow control functions
  • Positional Parameter
  • Redirections
  • Other functionalities
  • Examples
  • Regular Expression
  • Array


Matching Patterns
pattern:    example:      matches:                    not matched:
------------------------------------------------------------------
*           boo*          boot,boo,booth

?           boo?          boot                        booth
[...]       [aeiou]*      ark                         bark
[!...]      boo[!st]      boor                        boot
*(cc|cc)    boo*(ze|r)    boo,boor,booze,boozer       boot
+(cc|cc)    boo+(ze|r)    boor,booze,boozer           boo
?(cc|cc)    boo?(ze|r)    boo,boor,booze              boozer
@(cc|cc)    boo@(ze|r)    booze,booth                 boo
!(cc|cc)    boo!(ze|r)    booth,boo,boot              booze,boor
{c,c,c}     a{b,c,d}e     abe,ace,ade                 axe



Conditional Statements
format                           "true" if:
---------------------------------------------------
(( _num1_ == _num2_ ))           numbers equal
(( _num1_ != _num2_ ))           numbers not equal
(( _num1_ < _num2_ ))            num1 < num2
(( _num1_ > _num2_ ))            num1 > num2
(( _num1_ <= _num2_ ))           num1 <= num2
(( _num1_ >= _num2_ ))           num1 >= num2

[[ _str1_ == _str2_ ]]           strings equal
[[ _str1_ != _str2_ ]]           strings not equal
[[ _str1_ < _str2_ ]]            str1 precedes str2
[[ _str1_ > _str2_ ]]            str1 follow str2
[[ _str1_ = _pattern_ ]]         str1 = pattern
[[ _str1_ != _pattern_ ]]        str1 != pattern
[[ -z _str_ ]]                   str is null
[[ -n _str_ ]]                   str is not null

[ x=y -o k=j ]                   or in expression
[ x=y -a k=j ]                   and in expression



Test Objects (Files, Directories, etc.)
test "true" if:                 ksh
-----------------------------------
object exist                    -a
readable                        -r
writable                        -w
executable                      -x
non-zero length                 -s
zero length


directory                       -d
plain file                      -f
symbolic link                   -h
named pipe                      -p
block special file              -b
character special file          -c
soft link                       -L
socket                          -S

owned by me                     -O
owned by my group              not

"sticky" bit set                -k
set-group-ID bit set            -g
set-user-id bit set             -u

opened on a terminal           not



Format of flow control functions
"if-then"               if _expr_ then
                            _cmd(s)_
                        elif _expr_
                            _cmd(s)_
                        else
                            _cmd(s)_
                        fi

"case"                  case _word_ in
                            _pattern1_)   _cmd(s)_
                            _pattern2_)   _cmd(s)_
                            *)            break ;;
                        esac

"while"                 while _expr_ do
                            _cmd(s)_
                        done

"for"                   for _variable_ in _list_
                            _cmd(s)_
                        done

"until"                 until _expr_
                        do
                            _cmd(s)_
                        done



POSITIONAL PARAMETER
program, function or shell                         $0
argument 1 through 9                               $1 .. $9
nth argument                                       ${n}
number of positional parameters                    $#
every positional parameter                         $@, $*
decimal value returned by last executed cmd        $?
pid of shell                                       $$
pid of last backgrounded command                   $!



REDIRECTIONS
0             stdin
1             stdout
2             stderr

<&-           close stdin
>&-           close stdout
<>filename    open filename for read-write
2>&1          open 2 for write and dup as 1

Examples:
  cmd 2>/dev/null
  cmd >/dev/null 2>&1
  exec 1<&-           # close descriptor 1
  exec 2<&-           # close descriptor 2
  exec 1< /dev/null   # open descriptor 1
  exec 2< /dev/null   # open descriptor 2


 

OTHER FUNCTIONALITIES
cmd1 || cmd2    exec cmd2 if cmd1 fail
cmd1 && cmd2    exec cmd2 if cmd1 is OK

V1=${V2:=V3}    Set V1 with the value of V2 if this is set else set the
                variable V1 with value of V3 (V3 could be a number).
                sh replacement:  if [ $V2 ] ; then
                                                V1=$V2
                                 else
                                                V1=$V3
                Example: DisplaySize=${LINES:24} ; Command=${Command:"cat"}


${V1:?word}     if V1 set  & V1!=null   ret $V1 else print word and exit
                  : ${V1:?"variable V1 not set on null"}
${V1:=word}     if V1 !set | V1==null   set V1=$word
${V1:-word}     if V1 set  & V1!=null   ret $V1 else ret word
${V1:+word}     if V1 set  & V1!=null   ret word else ret nothing
${V1##patt}
${V1#patt}      if patt are found at the begin of V1 return V1 whitout the patt
                else return V1
                V1="lspwd" ; ${V1#"ls"}  # exec pwd
${V1%%patt}
${V1%patt}      if patt are found at the end of V1 return V1 whitout the patt
                else return V1
                V1="lspwd" ; ${V1%"pwd"}  # exec ls



EXAMPLES
- Explode a command for use parameters counter
    set `who -r` ; [ "$8" != "0" ] && exit

- declare a variable for only uppercase/lovercase chars
    typeset -u VAR ; VAR="lower" ; echo $VAR   -> LOWER
    typeset -l VAR ; VAR="UPPER" ; echo $VAR   -> upper

- exec

- eval - esegue il comando dato come argomento

- let - esegue le operazioni matematiche che passate come argomento
 let "x = x * 5"
 ((x = x * 5))  .. altra forma di let




REGULAR EXPRESSION
- ritorna la prima lettera dopo il segno - all'inizio di una stringa
    VAR="-ciao"
    RESULT=`expr "$VAR" : "-\(.\)"`
    echo $RESULT        .. -c
- toglie il '-' iniziale
    VAR="-ciao"
    VAR=`expr "$VAR" : "-*\(.*\)"`
    echo $VAR           .. ciao
- ritorna la lunghezza di una stringa
    VAR="ciao"
    echo `expr length $SHELL`      .. 4
- ritorna l'indice di dove incontra una substringa
    echo `expr index abcdef de`    .. 4
- ritorna 6 caratteri a partire dall'11
    expr substr "Goodnight Ladies" 11 6     .. Ladies



ARRAY
- definisce un array
 set -A Week Sat Sun Mon Tue Wed Thu Fri
- ritorna un elemento dell'array
 echo ${Week[3]}       .. Tue
 id=3 ; echo ${Week[id]}     .. Tue
- stampa tutti gli elemti di un array
 echo ${Week[@]}       .. Sat Sun Mon Tue Wed Thu Fri
- scandisce un array
    for day in ${Week[@]}
    do
        echo $day
    done
- ritorna il numero di elementi in un array
 nelem=${#Week[@]} ; echo $nelem   .. 7

Quick Unix Tutorial

Quick Unix Tutorial

Examples of Basic Commands



 Action 


 Command 


 Examples 
appent to file cat >> cat >> file1
combine 2 files cat cat file1 file2 > file3
copy files cp cp myfile copymyfile
create a file cat cat > newfile
edit files vi vi file
list files ls ls bin/
move a file mv mv file1 doc/chapter1
remove a file rm rm unwantedfile
rename a file mv mv oldfilename newfilename
view files cat
pg
more
less
view
cat file
pg file2 file3
view file6 file7


 Directories 


 Command 


 Examples 
change to another directory cd cd example/first/
create a directory mkdir mkdir example1
find out where you are pwd pwd
go to your home directory cd cd
remove an emplty eirectory rmdir rmdir junk
Redirection of Output or Input
>
   redirects the output of a command to a file
>>  redircts the output of a command to the end of an existing file
<    takes the input of a command form a file, not the terminal

Summary of Basic Commands

  • apropos  locate commands by keyword lookup
  • arch  display the architecture of the current host
  • cal  display a calendar
    cal [month] year
    • month   number between 1 and 12
    • year      number between 1 and 9999
    Examples:
    cal 1996      print calendar for year 1996
    cal 1 1997   print calendar for January 1997
  • cancel  send/cancel requests to an LP print service
  • cat  concatenate and display files (To view files, create files, append to files and combine files)
    cat [options] [files]
    Examples:
    cat files                read file(s)
    cat > file              create file (reads form terminal; terminate input with ^D)
    cat >> file            append to file (reads form terminal; terminate input with ^D)
    cat file2 >> file1   appends contents of file2 to file1
  • cd  shell built-in functions to change the current working directory
  • chdir  shell built-in functions to change the current working directory
  • chgrp  change the group ownership of a file
  • chmod  change the permissions mode of a file
  • chown  change owner of file
  • clear  clear the terminal screen
  • cp  copy files
  • date  print and set the date
  • dc  arbitrary precision desktop calculator
  • dos2unix  convert text file from DOS format to ISO format
  • eject  eject media such as CD-ROM and floppy from drive
  • exit  shell built-in functions to enable the execution of the shell to advance beyond its sequence of steps
  • file  determine the type of a file by examining its contents
  • head  display first few lines of files
  • lp  send/cancel requests to an LP print service
  • lpstat  print information about the status of the LP print service
  • ls  list the contents of a directory
    ls [options] [directories]
    the current working directory used if no directories specified
    A few options:
    • -a    list all entries includeing hidden files (starting with .)
    • -i    print inode numbers
    • -l    long list (mode, links, owner, group, size, timeof last modification, and name
    • -t    sort by modification time
    • -x    multi-column list, sorted across each row
  • Mail, mailx, mail rmail interactive message processing system to read mail or send mail to users
    mail [options] users
    Examples:
    mail                           with no options, to read your mail
    mail user                     to send mail to user
    mail user < filename     mail a file to another user
  • mkdir  make directories
  • more  browse or page through a text file
  • mv  move files
  • nispasswd  change NIS+ password information
  • page  browse or page through a text file
  • pg  files perusal filter for CRTs
  • pr  print files
  • ps  display the status of current processes
  • pwd  working directory name
  • rm  remove files or directories
  • rmdir  remove files or directories
  • spell  find spelling errors
  • tail  deliver the last part of a file
  • umask  shell built-in function to restrict read/write/execute permissions
  • unix2dos  convert text file from ISO format to DOS format
  • vi  screen-oriented (visual) display editor based on ex
  • view  screen-oriented (visual) display editor based on ex
  • w who is logged in, and what are they doing
  • wc  display a count of lines, words and characters in a file
  • which  locate a command; display its pathname or alias
  • who  who is on the system
  • whoami  display the effective current username
  • whois  Internet user name directory service
  • write write to another user

Basic UNIX commands

Basic UNIX commands

Note: not all of these are actually part of UNIX itself, and you may not find them on all UNIX machines. But they can all be used on turing in essentially the same way, by typing the command and hitting return. Note that some of these commands are different on non-Solaris machines - see SunOS differences.
If you've made a typo, the easiest thing to do is hit CTRL-u to cancel the whole line. But you can also edit the command line (see the guide to More UNIX).
UNIX is case-sensitive.

Files

  • ls --- lists your files
    ls -l --- lists your files in 'long format', which contains lots of useful information, e.g. the exact size of the file, who owns the file and who has the right to look at it, and when it was last modified.
    ls -a --- lists all files, including the ones whose filenames begin in a dot, which you do not always want to see.
    There are many more options, for example to list files by size, by date, recursively etc.
  • more filename --- shows the first part of a file, just as much as will fit on one screen. Just hit the space bar to see more or q to quit. You can use /pattern to search for a pattern.
  • emacs filename --- is an editor that lets you create and edit a file. See the emacs page.
  • mv filename1 filename2 --- moves a file (i.e. gives it a different name, or moves it into a different directory (see below)
  • cp filename1 filename2 --- copies a file
  • rm filename --- removes a file. It is wise to use the option rm -i, which will ask you for confirmation before actually deleting anything. You can make this your default by making an alias in your .cshrc file.
  • diff filename1 filename2 --- compares files, and shows where they differ
  • wc filename --- tells you how many lines, words, and characters there are in a file
  • chmod options filename --- lets you change the read, write, and execute permissions on your files. The default is that only you can look at them and change them, but you may sometimes want to change these permissions. For example, chmod o+r filename will make the file readable for everyone, and chmod o-r filename will make it unreadable for others again. Note that for someone to be able to actually look at the file the directories it is in need to be at least executable. See help protection for more details.
  • File Compression
    • gzip filename --- compresses files, so that they take up much less space. Usually text files compress to about half their original size, but it depends very much on the size of the file and the nature of the contents. There are other tools for this purpose, too (e.g. compress), but gzip usually gives the highest compression rate. Gzip produces files with the ending '.gz' appended to the original filename.
    • gunzip filename --- uncompresses files compressed by gzip.
    • gzcat filename --- lets you look at a gzipped file without actually having to gunzip it (same as gunzip -c). You can even print it directly, using gzcat filename | lpr
  • printing
    • lpr filename --- print. Use the -P option to specify the printer name if you want to use a printer other than your default printer. For example, if you want to print double-sided, use 'lpr -Pvalkyr-d', or if you're at CSLI, you may want to use 'lpr -Pcord115-d'. See 'help printers' for more information about printers and their locations.
    • lpq --- check out the printer queue, e.g. to get the number needed for removal, or to see how many other files will be printed before yours will come out
    • lprm jobnumber --- remove something from the printer queue. You can find the job number by using lpq. Theoretically you also have to specify a printer name, but this isn't necessary as long as you use your default printer in the department.
    • genscript --- converts plain text files into postscript for printing, and gives you some options for formatting. Consider making an alias like alias ecop 'genscript -2 -r \!* | lpr -h -Pvalkyr' to print two pages on one piece of paper.
    • dvips filename --- print .dvi files (i.e. files produced by LaTeX). You can use dviselect to print only selected pages. See the LaTeX page for more information about how to save paper when printing drafts.

Directories

Directories, like folders on a Macintosh, are used to group files together in a hierarchical structure.
  • mkdir dirname --- make a new directory
  • cd dirname --- change directory. You basically 'go' to another directory, and you will see the files in that directory when you do 'ls'. You always start out in your 'home directory', and you can get back there by typing 'cd' without arguments. 'cd ..' will get you one level up from your current position. You don't have to walk along step by step - you can make big leaps or avoid walking around by specifying pathnames.
  • pwd --- tells you where you currently are.

Finding things

  • ff --- find files anywhere on the system. This can be extremely useful if you've forgotten in which directory you put a file, but do remember the name. In fact, if you use ff -p you don't even need the full name, just the beginning. This can also be useful for finding other things on the system, e.g. documentation.
  • grep string filename(s) --- looks for the string in the files. This can be useful a lot of purposes, e.g. finding the right file among many, figuring out which is the right version of something, and even doing serious corpus work. grep comes in several varieties (grep, egrep, and fgrep) and has a lot of very flexible options. Check out the man pages if this sounds good to you.

About other people

  • w --- tells you who's logged in, and what they're doing. Especially useful: the 'idle' part. This allows you to see whether they're actually sitting there typing away at their keyboards right at the moment.
  • who --- tells you who's logged on, and where they're coming from. Useful if you're looking for someone who's actually physically in the same building as you, or in some other particular location.
  • finger username --- gives you lots of information about that user, e.g. when they last read their mail and whether they're logged in. Often people put other practical information, such as phone numbers and addresses, in a file called .plan. This information is also displayed by 'finger'.
  • last -1 username --- tells you when the user last logged on and off and from where. Without any options, last will give you a list of everyone's logins.
  • talk username --- lets you have a (typed) conversation with another user
  • write username --- lets you exchange one-line messages with another user
  • elm --- lets you send e-mail messages to people around the world (and, of course, read them). It's not the only mailer you can use, but the one we recommend. See the elm page, and find out about the departmental mailing lists (which you can also find in /user/linguistics/helpfile).

About your (electronic) self

  • whoami --- returns your username. Sounds useless, but isn't. You may need to find out who it is who forgot to log out somewhere, and make sure *you* have logged out.
  • finger & .plan files
    of course you can finger yourself, too. That can be useful e.g. as a quick check whether you got new mail. Try to create a useful .plan file soon. Look at other people's .plan files for ideas. The file needs to be readable for everyone in order to be visible through 'finger'. Do 'chmod a+r .plan' if necessary. You should realize that this information is accessible from anywhere in the world, not just to other people on turing.
  • passwd --- lets you change your password, which you should do regularly (at least once a year). See the LRB guide and/or look at help password.
  • ps -u yourusername --- lists your processes. Contains lots of information about them, including the process ID, which you need if you have to kill a process. Normally, when you have been kicked out of a dialin session or have otherwise managed to get yourself disconnected abruptly, this list will contain the processes you need to kill. Those may include the shell (tcsh or whatever you're using), and anything you were running, for example emacs or elm. Be careful not to kill your current shell - the one with the number closer to the one of the ps command you're currently running. But if it happens, don't panic. Just try again :) If you're using an X-display you may have to kill some X processes before you can start them again. These will show only when you use ps -efl, because they're root processes.
  • kill PID --- kills (ends) the processes with the ID you gave. This works only for your own processes, of course. Get the ID by using ps. If the process doesn't 'die' properly, use the option -9. But attempt without that option first, because it doesn't give the process a chance to finish possibly important business before dying. You may need to kill processes for example if your modem connection was interrupted and you didn't get logged out properly, which sometimes happens.
  • quota -v --- show what your disk quota is (i.e. how much space you have to store files), how much you're actually using, and in case you've exceeded your quota (which you'll be given an automatic warning about by the system) how much time you have left to sort them out (by deleting or gzipping some, or moving them to your own computer).
  • du filename --- shows the disk usage of the files and directories in filename (without argument the current directory is used). du -s gives only a total.
  • last yourusername --- lists your last logins. Can be a useful memory aid for when you were where, how long you've been working for, and keeping track of your phonebill if you're making a non-local phonecall for dialling in.

Connecting to the outside world

  • nn --- allows you to read news. It will first let you read the news local to turing, and then the remote news. If you want to read only the local or remote news, you can use nnl or nnr, respectively. To learn more about nn type nn, then \tty{:man}, then \tty{=.*}, then \tty{Z}, then hit the space bar to step through the manual. Or look at the man page. Or check out the hypertext nn FAQ - probably the easiest and most fun way to go.
  • rlogin hostname --- lets you connect to a remote host
  • telnet hostname --- also lets you connect to a remote host. Use rlogin whenever possible.
  • ftp hostname --- lets you download files from a remote host which is set up as an ftp-server. This is a common method for exchanging academic papers and drafts. If you need to make a paper of yours available in this way, you can (temporarily) put a copy in /user/ftp/pub/TMP. For more permanent solutions, ask Emma. The most important commands within ftp are get for getting files from the remote machine, and put for putting them there (mget and mput let you specify more than one file at once). Sounds straightforward, but be sure not to confuse the two, especially when your physical location doesn't correspond to the direction of the ftp connection you're making. ftp just overwrites files with the same filename. If you're transferring anything other than ASCII text, use binary mode.
  • lynx --- lets you browse the web from an ordinary terminal. Of course you can see only the text, not the pictures. You can type any URL as an argument to the G command. When you're doing this from any Stanford host you can leave out the .stanford.edu part of the URL when connecting to Stanford URLs. Type H at any time to learn more about lynx, and Q to exit.

Miscellaneous tools

  • webster word --- looks up the word in an electronic version of Webster's dictionary and returns the definition(s)
  • date --- shows the current date and time.
  • cal --- shows a calendar of the current month. Use e.g., 'cal 10 1995' to get that for October 95, or 'cal 1995' to get the whole year.
You can find out more about these commands by looking up their manpages:
man commandname --- shows you the manual page for the command For further ways of obtaining help, look at the pages with electronic sources of information and non-electronic sources.