Oracle UNIX Administration sadc Utility Oracle UNIX/Linux Tips by Burleson Consulting

Using the UNIX sadc Utility

The sadc utility is short for System Activity Report Package.  The sadc utility is a popular package that can be used inside cron to schedule collections of server statistics.

The sadc utility is designed for use by the UNIX System Administrator and all sadc reports must be run as root and provide detailed server information. 

The sacd sa1 report

One of the most popular sadc reports is sa1.  Here is a Bourne shell script that is commonly submitted as the root user to capture server details.

#! /usr/bin/sh
# @(#) $Revision: 72.3 $
#       sa1.sh

DATE=`date +%d`
ENDIR=/usr/lbin/sa
DFILE=/var/adm/sa/sa$DATE
cd $ENDIR
if [ $# = 0 ]
then
        exec $ENDIR/sadc 1 1 $DFILE
else
        exec $ENDIR/sadc $* $DFILE
fi

Using the glance utility

Another popular utility for UNIX is the glance utility. The glance utility originated on HP/UX systems, but it is now becoming available for other UNIX dialects including Solaris and AIX. The glance screens can be driven either by clicking function keys or by a command driver (figure 2-6).  Here we see the basic letter commands for glance.

Figure 6: The glance command summary screen

You start the glance utility by entering the glance command from the UNIX prompt.  The glance utility provides a graphical display of UNIX server performance (Figure 2-7).  Note at the bottom of the screen that we have various function keys for display additional details.  We will start with the initial display screen and then move on to address each major glance function.

The glance process summary screen

The first screen in glance displays the current CPU, memory, disk and swap consumption, and also reports on the top processes.

Figure 7: The glance utility screen

The top of the glance screen shows stacked histograms for CPU, disk, memory and swap.  Within each histogram we see two portions, the system (s) and user (U) values.

The default glance screen is the process screen.  Underneath the histograms, we see a list of all of the top processes on our Oracle server.  These glance screen are from an Oracle Applications system, and we see the top processes. Here we see multiple tasks of f45runw, which is the driving task for Oracle SQL*Forms.  The tasks on the process screen are displayed in the order of current CPU consumption and we also see data on cumulative CPU consumption (Cum CPU) and the disk I/O rate.  In this example, we see high activity for the Oracle checkpoint background process (ora_ckpt_PR).

The glance CPU screen

The glance CPU screen shows CPU consumption for the major UNIX system calls.  These columns sum to 100% and allow you to view the totals for the CPU components (Figure 2-8).  The major CPU components include:

* user() ? Show total usage for tasks in user execution mode

* nice() ? Sequences UNIX tasks for the run queue

* interrupts() ? Show activity for task interrupted to perform  a UNIX system call.

* Idle() ? Shows the amount of the CPU in idle state

In this screen shot, we note that our server is 90% idle.

Figure 8: The glance CPU screen

The glance memory screen

The memory screen in glance show total RAM memory usage on our UNIX server. The important rows in this display are the Page In row and the KB Paged In row.

As we noted earlier in this chapter, RAM page in operations are highly undesirable, and this screen show a clear shortage of RAM on this server.  In this case I know that this server has a 32-bit Oracle system running on a 64-bit server, and 32-bit Oracle does not address SGA sizes in excess of 1.2 gigabytes.  Hence, we see competition for RAM memory, and significant RAM memory paging.  By the way, this server should be upgraded to 64-bit Oracle where UNIX will be able to address high memory.

Figure 9: The glance memory screen

The glance disk screen

The F4 key invokes the glance disk display. This screen shows both the logical and physical disk reads.  A logical disk read is a read request while a physical read is a logical read request that was forced to perform a disk read because the data block did not exist in the UNIX JFS data buffer.  Here we see that UNIX has made 1662 logical request which resulted in 157 physical disk reads.  This gives the UNIX JFS a buffer cache hit ratio of 94%.

Figure 10: The glance disk screen

We will go into great detail on using glance for monitoring disk activity in Chapter 8.

The glance global waits screen

The global waits screen show all major UNIX wait events as a percentage of total waits on the server.  Here we see waits on semaphores, pipes and sleep operations.

Figure 11: The glance global wait state screen

The glance global system calls screen

This glance screen shows all system call for the major UNIX system calls that we discussed earlier in this chapter.  Here we see the summaries for fork(), read(), write(), open(), and close() system calls.

On this screen we see a system call signature that is typical of Oracle databases.  The read() and write() system calls are consuming well over 90% of the server.  This signature is often quite different for Oracle app server where the majority of work is computational rather than I/O related.

Figure 12: The glance global system calls screen

The glance system table screen

This screen displays the system table for the UNIX server.  Unlike Oracle tables, UNIX system tables are in-memory structures that allow UNIX to govern internal operations.  Here we see a typical signature for an Oracle database with the majority of activity in the file table.

Figure 13: The glance system tables screen

The glance swap screen

The glance swap screen is very important for monitoring the usage of RAM on the UNIX server.  Here we see all of the defined swap disk space, the size of the swap disk area, and the amount of swap that has been used.  This screen is very important when monitoring your UNIX server for RAM overloads.

Figure 14: The glance swap screen

Next, let?s look at the most popular monitoring command, vmstat, and see how vmstat can quickly show UNIX server performance.

Overview of the vmstat Utility

The vmstat utility is the most common UNIX monitoring utility, and it is found in the majority of UNIX dialects (Note that vmstat is called osview on the IRIX dialect of UNIX). The vmstat utility displays various server values over a given time interval. The vmstat utility is invoked from the UNIX prompt, and it has several numeric parameters. The first numeric argument to vmstat represents the time interval (expressed in seconds) between server samples. The second argument specifies the number of samples to be reported. In the example that follows, vmstat is executed to take five samples at 2-second intervals:

root> vmstat 2 5

Almost all UNIX servers have some version of vmstat. Before we look at the details for this powerful utility, let?s explore the differences that you are likely to see.

Dialect Differences in vmstat

Because each hardware vendor writes their own vmstat utility, there are significant differences in vmstat output. The vmstat output is different depending on the dialect of UNIX, but each dialect contains the important server metrics.

Because vendors have written their own versions of the vmstat utility, it can be useful to consult the online UNIX documentation to see the display differences. In UNIX, you can see your documentation by invoking the man pages. The term man is short for manual, and your can see the documentation for your particular implementation of vmstat by entering man vmstat from your UNIX prompt.

Below is a sample of vmstat output for the four most popular dialects of UNIX. In each example below, the important metrics appear in bold.

vmstat for Solaris

In the Sun Solaris operating environment, the output from vmstat will appear like this:

root> vmstat 2 5

 procs     memory            page    disk          faults      cpu
 r b w   swap  free  re  mf pi po ?  s6 -- --   in   sy   cs us sy id
 0 0 0 2949744 988800 0   4  0  0 ?  0  0  0  148  200   41  0  0 99
 0 0 0 2874808 938960 27 247 0  1 ?  0  0  0  196  434   64  1  2 98
 0 0 0 2874808 938960 0   0  0  0 ?  0  0  0  134   55   32  0  0 100
 0 0 0 2874808 938960 0   0  0  0 ?  0  0  0  143  114   39  0  0 100
 0 0 0 2874808 938960 0   0  0  0 ?  0  0  0  151   86   38  0  0 100

vmstat for Linux

In the Linux operating environment, the output from vmstat will appear like this:

root> vmstat 2 5

   procs                      memory  swap    io     system         cpu
 r  b  w   swpd   free   buff  cache  si  ? bi    bo   in    cs  us  sy  id
 1  0  0    140  90372 726988  26228   0  ?  0     0   14     7   0   0   4
 0  0  0    140  90372 726988  26228   0  ?  0     2  103    11   0   0 100
 0  0  0    140  90372 726988  26228   0  ?  0     5  106    10   0   0 100
 0  0  0    140  90372 726988  26228   0  ?  0     0  101    11   0   0 100
 0  0  0    140  90372 726988  26228   0  ?  0     0  102    11   0   0 100

vmstat for AIX

In the IBM AIX operating environment, the output from vmstat will appear like this:

root> vmstat 2 5

kthr     memory             page              faults        cpu    
----- ----------- ------------------------ ------------ -----------
 r  b   avm    fre  re  pi  po  fr   sr  cy  in     sy  cs  us sy id wa
 7  5 220214   141   0   0   0  42   53   0 1724 12381 2206 19 46 28  7
 9  5 220933   195   0   0   1 216  290   0 1952 46118 2712 27 55 13  5
13  5 220646   452   0   0   1  33   54   0 2130 86185 3014 30 59  8  3
 6  5 220228   672   0   0   0   0    0   0 1929 25068 2485 25 49 16 10

vmstat for HP/UX

In the Hewlett Packard HP/UX operating environment, the output from vmstat will appear like this:

root> vmstat 2 5
r  b  w   avm    free re at  pi po  ?     in   sy    cs  us sy id
1  0  0 70635  472855 10  5   2  0  ?   2024 2859   398   4  1 96
1  0  0 74985  472819  9  0   1  0  ?   1864 1820   322   0  0 100
0  0  0 83056  472819  2  0   0  0  ?   1846 1684   302   0  0 100
0  0  0 81390  472819  0  0   0  0  ?   1847 1571   288   0  0 100
0  0  0 78788  472819  0  0   0  0  ?   1852 1608   291   0  0 100

Now that we have seen the different display options for each dialect of vmstat, let?s take a look at the data items in vmstat and understand the common values that we can capture in STATSPACK tables.

 

If you like Oracle tuning, see the book "Oracle Tuning: The Definitive Reference", with 950 pages of tuning tips and scripts.  You can buy it direct from the publisher for 30%-off and get instant access to the code depot of Oracle tuning scripts.