Call now: 252-767-6166  
Oracle Training Oracle Support Development Oracle Apps

 
 Home
 E-mail Us
 Oracle Articles
New Oracle Articles


 Oracle Training
 Oracle Tips

 Oracle Forum
 Class Catalog


 Remote DBA
 Oracle Tuning
 Emergency 911
 RAC Support
 Apps Support
 Analysis
 Design
 Implementation
 Oracle Support


 SQL Tuning
 Security

 Oracle UNIX
 Oracle Linux
 Monitoring
 Remote s
upport
 Remote plans
 Remote
services
 Application Server

 Applications
 Oracle Forms
 Oracle Portal
 App Upgrades
 SQL Server
 Oracle Concepts
 Software Support

 Remote S
upport  
 Development  

 Implementation


 Consulting Staff
 Consulting Prices
 Help Wanted!

 


 Oracle Posters
 Oracle Books

 Oracle Scripts
 Ion
 Excel-DB  

Don Burleson Blog 


 

 

 


 

 

 

 

 

The Plague of Large Oracle Disks

Oracle Database Tips by Donald Burleson


Oracle is a data delivery engine, and accessing data from disks often imposes a system bottleneck.

Also see ASM Optimal Disk Placement (ODP)

In my book "Oracle Disk I/O Tuning" that placing too much data on a single disk spindle will impose enqueues because the mechanical device can only locate to a single cylinder at a time.  On busy Oracle databases on a single disk spindle, the disk can shake like an out-of-balance washing machine as competing tasks enqueue for data service.

Originally, RAID was an acronym for INEXPENSIVE DISKS, and Oracle professionals enjoyed the ability of spreading their Oracle data across multiple disks and spreading the load.  Today's large disk arrays commonly utilize asynchronous write and multi-gigabyte RAM buffers to minimize this latency, but it still is a major concern for thousands of Oracle shops.

According to Mike Ault's book "Oracle Solid State Disk Tuning", smaller solid-state RAM disks have far less bandwidth issues because the RAM architecture of SSD allow high concurrent access that is impossible on a mechanical platter.

This issue of single channel access also imposes a bottleneck on Oracle disk devices, and the large disks (over 144 gigabytes) often perform more slowly for high concurrent access than their smaller predecessors.

  • Oracle's standard SAME (Stripe and Mirror Everywhere, RAID 10) is largely useless for load balancing if the whole database resides on just a few physical disks.
     

  • Seek delay (movement of the read-write heads) composes over 80% of disk access latency, and high concurrent requests against large devices make them very slow.

Disk enqueues can occur when the disk is unable to quickly service concurrent requests.  Super-large disks can be problematic, and the most popular Oracle data files can be placed on the middle absolute track of the device to minimize read-write head movement.

As manufacturing costs continue to fall, disk vendors are offering larger and larger disks and this is imposing some serious large disk performance issues with seek delay is 1.5 times slower:

"In the "good old days" when 9G disks were big, we didn't have this problem. Really, this problem is new since then. Back then, if we wanted 200G of storage RAID1, we needed about 45 of those disks.

Controllers could only handle 7 of them, you see (the 8th device on the bus was the controller itself) and that meant we had proportionally lots more access roads, and lots more loading docks per square foot of warehouse space than you typically have today. . .

I note:

  • The seek time is actually slower today.
  • The bandwidth performance is maximum only 10 times better
  • Your controller is no more than 32 times faster
  • The disk, however is about 83 times bigger!"

As a result of this trend, many Oracle professionals experience external I/O waits and they see that the top-5 waits events (from a STATSPACK or AWR Report) show "db file sequential reads" and "db file scattered reads" as a main system bottleneck.

Solutions to the large disk plague?

Obviously, the Oracle professional must take action to relieve disk I/O bottlenecks.  There are several solutions to this issue:

  • Use large data buffer caches - The majority of the Oracle 10g benchmarks ( www.tpc.org ) use 64-bit Oracle with a db_cache_size over 50 gigabytes.  Other large shops segregate I/O into multiple data buffers by using Oracle multiple blocksizes. 

  • Get higher bandwidth storage - Some Oracle shops purchase the more-expensive smaller devices or disk with fixed read-write heads (Winchester technology).  Other embrace SSD arrays which have unprecedented bandwidth for high concurrent access since Oracle SSD clobbers disk access speed.

  • ASM intelligent file placement:  Oracle 11gr2 introduces ASM intelligent file placement a new approach to speeding-up disk throughput on very large disk platters.  With ASM, the DBA marks "hot" data files (or ASM templates) and Oracle relocates them to the outermost sectors where you can retrieve more data per disk revolution:

Oracle 12c automatic data optimization

In a manual system, popular data ages-out and become less popular and read-only.  In this approach to data lifecycle management, aged-out popular data is automatically compressed and move to a lower-tier tertiary storage (disk or tape).

 

My other notes on Oracle disks include:

Market Survey of SSD vendors for Oracle:

There are many vendors who offer rack-mount solid-state disk that work with Oracle databases, and the competitive market ensures that product offerings will continuously improve while prices fall.  SearchStorage notes that SSD is will soon replace platter disks and that hundreds of SSD vendors may enter the market:

"The number of vendors in this category could rise to several hundred in the next 3 years as enterprise users become more familiar with the benefits of this type of storage."

As of January 2015, many of the major hardware vendors (including Sun and EMC) are replacing slow disks with RAM-based disks, and Sun announced that all of their large servers will offer SSD.

Here are the major SSD vendors for Oracle databases (vendors are listed alphabetically):

2008 rack mount SSD Performance Statistics

SearchStorage has done a comprehensive survey of rack mount SSD vendors, and lists these SSD rack mount vendors, with this showing the fastest rack-mount SSD devices:

manufacturer model technology interface performance metrics and notes
IBM RamSan-400 RAM SSD

Fibre Channel
InfiniBand

3,000MB/s random sustained external throughput, 400,000 random IOPS
Violin Memory Violin 1010 RAM SSD

PCIe

1,400MB/s read, 1,00MB/s write with ×4 PCIe, 3 microseconds latency
Solid Access Technologies USSD 200FC RAM SSD

Fibre Channel
SAS
SCSI

391MB/s random sustained read or write per port (full duplex is 719MB/s), with 8 x 4Gbps FC ports aggregated throughput is approx 2,000MB/s, 320,000 IOPS
Curtis HyperXCLR R1000 RAM SSD

Fibre Channel

197MB/s sustained R/W transfer rate, 35,000 IOPS

Choosing the right SSD for Oracle

When evaluating SSD for Oracle databases you need to consider performance (throughput and response time), reliability (Mean Time Between failures) and TCO (total cost of ownership).  Most SSD vendors will provide a test RAM disk array for benchmark testing so that you can choose the vendor who offers the best price/performance ratio.

Burleson Consulting does not partner with any SSD vendors and we provide independent advice in this constantly-changing market.  BC was one of the earliest adopters of SSD for Oracle and we have been deploying SSD on Oracle database since 2005 and we have experienced SSD experts to help any Oracle shop evaluate whether SSD is right for your application.  BC experts can also help you choose the SSD that is best for your database.  Just  call 800-766-1884 or e-mail.:  for SSD support details.

DRAM SSD vs. Flash SSD

With all the talk about the Oracle “flash cache”, it is important to note that there are two types of SSD, and only DRAM SSD is suitable for Oracle database storage.  The flash type SSD suffers from serious shortcomings, namely a degradation of access speed over time.  At first, Flash SSD is 5 times faster than a platter disk, but after some usage the average read time becomes far slower than a hard drive.  For Oracle, only rack-mounted DRAM SSD is acceptable for good performance:

Avg. Read speed

Avg. write speed

Platter disk

10.0 ms.

  7.0 ms.

DRAM SSD

 0.4 ms.

  0.4 ms.

Flash SSD    

 1.7 ms.

 94.5 ms.

 

 


 

 

��  
 
 
Oracle Training at Sea
 
 
 
 
oracle dba poster
 

 
Follow us on Twitter 
 
Oracle performance tuning software 
 
Oracle Linux poster
 
 
 

 

Burleson is the American Team

Note: This Oracle documentation was created as a support and Oracle training reference for use by our DBA performance tuning consulting professionals.  Feel free to ask questions on our Oracle forum.

Verify experience! Anyone considering using the services of an Oracle support expert should independently investigate their credentials and experience, and not rely on advertisements and self-proclaimed expertise. All legitimate Oracle experts publish their Oracle qualifications.

Errata?  Oracle technology is changing and we strive to update our BC Oracle support information.  If you find an error or have a suggestion for improving our content, we would appreciate your feedback.  Just  e-mail:  

and include the URL for the page.


                    









Burleson Consulting

The Oracle of Database Support

Oracle Performance Tuning

Remote DBA Services


 

Copyright © 1996 -  2017

All rights reserved by Burleson

Oracle ® is the registered trademark of Oracle Corporation.

Remote Emergency Support provided by Conversational