A brief history of database disk storage Oracle Database Tips by Donald BurlesonOctober 28, 2015

 

The duties of any database administrator include responsibility for the management, control and safety of mission-critical information, and a primary concern of the Oracle DBA is managing their computer disk storage. 

Winston Churchill elegantly described the important of knowing history:

"The farther you can look into the past, the farther that you can see into the future". 

With this sage wisdom in-mind, lets review the history of this all-important component of database tuning.  Also see my important notes on tuning Oracle full-scan operations.

In the 1970's, punched cards were the prominent data storage device, and the ubiquitous cards were even used as income tax refund checks.  People even made cute Christmas trees from punched cards in the "Data Processing" department.

College-aged kids have no idea what the term "Do not fold, spindle or mutilate" means and they missed-out on the fun of dropping your card deck on the floor and using giant sorting machines to re-sequence them (assuming you had the sequence numbers in all of the cards).

• Punched Cards: (1960-1970)

• Drums: (1970-1980)

• Disks: (1980-2000)

• Solid-State Disk: (2000-2010)

More than fifty years ago, IBM introduced RAMAC (random access accounting & control), the first in a long line of cumbersome and expensive database storage architectures.  Disk platter storage continued to evolve through the 1990's, when engineers hit the physical limits of mechanical devices and RAM was introduced as a front-end cache.  In the early 21st century we see RAM-SAN replacing the antiquated mechanical platters.

We also see that old habits die hard, especially with regard to database technology.  In database management software, IBM's flagship IMS database still has a significant market share after 40 years on-the-job, IBM's mainframe DB2 database still stores most of the world's data, and Oracle, the world's most flexible and robust DBMS, will soon celebrate their 20th birthday. 

Lets take a quick history trip to understand the exact nature of disk storage within large database management systems.  Also see my related notes on the future of database management oracle database management.
 

The age of spinning platters

One of the first commercial uses of computer storage involved paper tape and punched cards, and these archaic data storage methods were superseded by Direct Access Storage Devices, DASD (pronounced "daz-dee").  The most popular disk of the 1980's was the refrigerator-sized 3380 disks, which contained only 1.2 gig of storage at the astronomical cost of over $200,000.  In today's 2015 dollars, disk in the 1980's costs more than $4,000.00 per megabyte.

Today, you can buy 100 GB disks for $200, and 100 GB of RAM Disk (solid-state disk) for $100,000.

A DBA in 1980's spent a great deal of time managing their DASD spindles, huge mechanical devices with these giant rotating platters of magnetic-coated media.  RAID (and the whole concept of "inexpensive" disk) was a decade away, and the DBA of the 1980's fought to make every byte count, while still delivering acceptable performance to their end-user.

In order to reduce the latency of access, The 1980's DBA would place high-use datafile near the middle absolute track of the device, and data placement was always a pain because of the huge size of the disks and the high expense.  See my book Oracle Disk I/O Tuning for details:
 

So, with the technology constantly re-vamping, where have we come in the past half-century of disks?  I think that it has gotten worse, not better.  Sure, disks are incredibly cheap compared to their ancestors, but they still have shortcomings, and disks are getting worse, not better:

The Plague of Large Oracle Disks

Placing too much data on a single disk spindle can impose enqueues on any disk because the mechanical device can only locate to a single cylinder at a time.  On busy Oracle databases on a single disk spindle, the disk can shake like an out-of-balance washing machine as competing tasks enqueue for data service.

Originally, RAID was an acronym for INEXPENSIVE DISKS, and Oracle professionals enjoyed the ability of spreading their Oracle data across multiple disks and spreading the load.  Today's large disk arrays commonly utilize asynchronous write and multi-gigabyte RAM buffers to minimize this latency, but it still is a major concern for thousands of Oracle shops.

This problem of "single channel access" (a mechanical bottleneck caused by the read-write heads of the disk platters) also imposes a bottleneck on Oracle disk devices, and the large disks (over 144 gigabytes) often perform more slowly for high concurrent access than their smaller predecessors.

• Oracle's standard SAME (Stripe and Mirror Everywhere, RAID 10) is largely useless for load balancing if the whole database resides on just a few physical disks.
   

• Seek delay (movement of the read-write heads) composes over 80% of disk access latency, and high concurrent requests against large devices make them very slow.

Disk enqueues can occur when the disk is unable to quickly service concurrent requests.  Super-large disks can be problematic, and the most popular Oracle data files can be placed on the middle absolute track of the device to minimize read-write head movement.

As manufacturing costs continue to fall, disk vendors are offering larger and larger disks and this is imposing some serious large disk performance issues with seek delay is 1.5 times slower:

As a result of this trend, many Oracle professionals experience external I/O waits and they see that the top-5 waits events (from a STATSPACK or AWR Report) show "db file sequential reads" and "db file scattered reads" as a main system bottleneck.

The disk vendors say that disk is not dead, and that they will continue to get smaller and smaller:

"Drives with mere hundreds of gigabytes will be small enough to wear as jewelry.

"You'll have with you every album and tune you've ever bought, every picture you've ever taken, every tax record,"

Reducing physical disk reads

We see the following trends for Oracle disk in this decade:

• Disk storage costs fall 10x every year.

• Magnetic-coated spinning platters are obsolete and cannot go faster

• RAM-SAN replaces disk by 2015.

Note the change to Moore's Law for disks show the limitations of the spinning platter technology (figure 3).

Moore's Law for disk

Platters can only spin so fast without becoming aerodynamic and the disk vendors were hard-pressed to keep their technology improving in speed. Their solution was to add a RAM front-end to their disk arrays and sophisticated asynchronous read-write software to provide the illusion of faster hardware performance.

In the 21st century, the Oracle professional must take action to relieve disk I/O bottlenecks.  There are several solutions to this issue:

• Use large data buffer caches - The majority of the Oracle 10g benchmarks ( www.tpc.org ) use 64-bit Oracle with a db_cache_size over 50 gigabytes.  Other large shops segregate I/O into multiple data buffers by using Oracle multiple blocksizes. 
   

• Get higher bandwidth storage - Some Oracle shops purchase the more-expensive smaller devices or disk with fixed read-write heads (Winchester technology).  Other embrace SSD arrays which have unprecedented bandwidth for high concurrent access since Oracle SSD clobbers disk access speed.

It's clear that solid-state devices are making huge headway, and several books on Oracle SSD tuning and stepping-up to the new technology.  With prices falling rapidly, I expect that most database will be solid-state in the next few years and that disk will become the "new tape", offline tertiary storage for backups.

Disk history References

My other notes on Oracle disks include: