Mapping Disk Architectures Administration
Oracle UNIX/Linux Tips by Burleson Consulting
Mapping Oracle Disk Architectures
Today?s disk devices are normally delivered
as complete I/O subsystems, complete with their own memory cache,
channels, disk adapters and SCSI adapters. Understanding the
architecture requires mapping the number of ports, the size of the
disk cache, the number of disk adapters, and the mapping of I/O
channels between the disks and the disk cache. Figure 4-10 shows a
sample of a disk architecture map for a disk array.
Figure 10: A sample architecture of a disk
Developing this type of disk map is very
important to load balancing within Oracle because there are many
possible bottlenecks within the disk array subsystem that can cause
slowdowns. In addition to monitoring for disk waits, we also need to
monitor for SCSI contention, channel contention and contention
between the disk adapters. Fortunately, many of the major disk
vendors (EMC, IBM) provide their own proprietary disk utilities
(e.g., NaviStar, Open Symmetrics Manager) to perform these disk
The Multiple RAM Buffer Issue
We are also seeing disk arrays being
delivered with a separate RAM cache for the disk arrays, as shown in
Figure 4-11. These RAM caches can be many gigabytes in size and
contain special software tools for performing asynchronous writes
and minimizing disk I/O.
Figure 11: Multiple RAM caches with an
The Oracle DBA needs to consider the RAM
cache on the disk array, because it changes the basic nature of disk
I/O. As you know, when Oracle cannot find a data block in one of the
data buffers in the SGA, Oracle will issue a physical read request
to the disk array. This physical read request is received by the
disk array, and the disk RAM cache is checked for the desired block.
If the desired block is in the RAM cache, the disk array will return
the block to Oracle without making a physical disk I/O.
The fact that Oracle physical requests may
not match actual read requests is a very important point, because it
can lead to misleading statistics. For example, the stats$filestatxs
table shows the number of reads and writes to files. If you are
using a disk array such as EMC, these I/O statistics will not
correspond to the actual disk reads and writes. The only conclusive
way to check ?real? disk I/O is to compare the physical I/O as
measured on the disk array with Oracle?s read and write statistics.
In many cases, the disks are performing less than half the I/O
reported by Oracle, and this discrepancy is due to the caching of
data blocks on the disk array RAM memory.
Next, let?s look at file striping and see
how it can be used to load balance the I/O subsystem.
File Striping with Oracle
File striping is the process of splitting a
tablespace into small datafiles and placing these datafiles across
many disks. With the introduction of RAID (redundant arrays of
inexpensive disks), we also have the option of block-interleaf
striping (RAID 1), which places each data block in the tablespace on
a separate disk.
Other methods of Oracle file striping
involve taking a large tablespace and splitting it into many Oracle
datafiles. These files may then be spread across many disks to
reduce I/O bottlenecks, as shown in Figure 4-12.
Figure 12: Striping a tablespace across
However, manual file striping has become
obsolete because of the large size of disks. In 1990, a 20GB
database would probably have been composed of 20 physical disks,
each within 1GB of storage. With many disks in a database, the
Oracle DBA could improve throughput by manually striping the busiest
tablespaces across many disks.
Commercials disks are getting larger every
year, and it is very difficult to find small disk devices that
contain less than 36GB of storage. Just ten years ago, the IBM 3380
disk was considered huge at 1GB of storage. Today, the smallest
disks available are 18GB. The larger disks mean that there are fewer
disk spindles, and fewer opportunities for manual file striping.
Since it is often not possible to isolate Oracle tablespaces on
separate disks without wasting a huge amount of disk space, the
Oracle administrator must balance active with inactive tablespaces
across their disks.
Note: There is a new feature in Oracle8i
called ?single table clusters.? By using a cluster, the keys are
grouped in the same physical block?reducing IO and speeding data
retrieval by key.
Using RAID with Oracle
As you may know, there are more than six
different types (called ?levels?) of RAID architectures, and each
has its own relative advantages and disadvantages. For the purposes
of an Oracle database, many of the RAID schemes do not posses the
high performance required for an Oracle database, and are omitted
from this discussion.
Please note that RAID 5 is not considered
for database that perform write activity since the processing
overhead for updates makes it too slow for most Oracle applications.
Below are the most commonly used RAID architectures for Oracle
* Raid 0?RAID 0 is commonly referred to as
block-level striping. This is an excellent method for performing
load balancing of the Oracle database on the disk devices, but it
does nothing for high availability since none of the data is
duplicated. Unlike manual datafile striping, where the Oracle
professional divides an Oracle tablespace into small datafiles, with
RAID 0, the Oracle datafile is automatically striped one block at a
time across all of the disk devices. In this fashion, every datafile
has pieces residing on each disk, and the disk I/O load will become
very well balanced. Note that a disk failure in RAID 0 results
in the loss of the datafiles storage on this device. A good
recommendation is to only put temporary tables on this that can be
easily recovered in the case of a disk failure.
* RAID 1?RAID 1 is commonly called disk
mirroring. Since the disks are replicated, RAID 1 may involve double
or triple mirroring. The RAID 1 architecture is designed such that a
disk failure will cause the I/O subsystem to switch to one of the
replicated disks with no service interruption. RAID 1 is use when
high availability is critical, and with triple mirroring, the mean
time to failure (MTTF) for an Oracle database is measured in
decades. (Note that disk controller errors may cause RAID 1
failures, although the disks remain healthy.)
* RAID 0+1?Raid 0+1 is the combination of
block-level striping and disk mirroring. The advent of RAID 0+1 has
made Oracle-level striping obsolete since RAID 0+1 stripes at the
block level, dealing out the table blocks, one block per disk,
across each disk device. RAID 0+1 is also a far better striping
alternative since it distributes the load evenly across all of the
disk devices, and the load will rise and fall evenly across all of
the disks. This relieves the Oracle administrator of the burden of
manually striping Oracle tables across disks and provides a far
greater level of granularity than Oracle striping, because adjacent
data blocks within the same table are on different disks.
* RAID 5 - Some of the newer hardware based
Raid 5 storage does extremely well in performance in data
warehouses. RAID 5 is a good approach for Oracle data warehouses
where the load speeds are not important and where the majority of
the system I/O is read-only activity.
Note that the use of RAID does not guarantee
against catastrophic disk failure. Oracle specifically recommends
that all production databases be run in archivelog mode regardless
of the RAID architecture, and that periodic Oracle backups should be
performed. Remember that there are many components to I/O
subsystems?including controllers, channels, disk adapters, SCSI
adapters?and a failure of any of these components could cause
unrecoverable disk failures of your database. RAID should only be
used as an additional level of insurance, and not as a complete
If you like Oracle tuning, see the
Tuning: The Definitive Reference", with 950 pages of tuning
tips and scripts.
You can buy it direct from the publisher for 30%-off and get
instant access to the code depot of Oracle tuning scripts.