Oracle 11g release 2 includes a feature called
flash_cache, a table and index argument that resembles the assignment
of high-use objects to the KEEP pool.
However, unlike the KEEP pool which uses volatile RAM
disk, the flash_cache is used for tertiary storage on solid-state
disk (SSD).
In plain English, Oracle is recognizing a hierarchy of
storage hardware, organized by access speed:
-
Data Buffer RAM - The KEEP pool
can cache objects for super-fast access at RAM speeds. Of course,
this data is not persistent.
-
Solid-state disk - SSD is not as
fast as native RAM, but it remains hundreds of times faster than platter
disks. However, all SSD suffers from a severe write penalty and is
not ideal for high DML data.
-
Disk spindles - The old
magnetic-coated spinning platters from the 1960's comprise the bulk of
Oracle data storage, with stone-age access speeds expressed in
milliseconds.
The docs suggest that the flash_cache refers to "flash"
SSD memory, but it appears that this approach is using SSD as a temporary
cache and not as permanent home for the data, as we see in SSD flash memory.
Many people misunderstand that the "flash memory" is very
different from the rack-mounted industrial strength solid-state disk.
For complete timing details on rack-mount SSD, see the books "Oracle
tuning with solid-state disk" and
"Oracle
RAC and Grid tuning with SSD".
In traditional SSD, the flash memory is an external rack
mount device and it looks like just a bunch of disks (JBOD), and the Oracle
data files are allocated directly onto the flash media. Conversely, in
Oracle's approach, the flash_cache refers to internal flash SSD, and the
flash_cache syntax treats it like "temporary" caching instead of permanent
storage.
Note:
Guy Harrison describes a patch that
allows you to use Oracle flash_cache without a million dollar
Exadata server. He also offers this script for displaying flash_cache
waits:
SELECT
name,
value
FROM
v$sysstat
WHERE
name IN
('physical read flash cache hits',
'physical reads',
'consistent gets',
'db
block gets',
'flash cache inserts');
Guy also notes that you can see flash_cache
entries in the data buffers with the v$bh view, where the stats
starts with flash:
select
owner||'.'||object_name
from
v$bh, dba_objects
where
v$bh.status
like ('flash%');
He also lists the new flash_cache system
statistics:
Flash_cache buffer changes
The 11g r2 docs note that the flash_cache is also
used to segregate objects, away from the "main memory buffer":
A flash cache is an extension of the database buffer cache
that lives on a flash disk, which is a solid state storage device that uses
flash memory.
-
Without flash cache, the database re-uses each
clean buffer in main memory as needed, overwriting it. If the
overwritten buffer is needed later, then the database must read it from
magnetic disk.
-
With flash cache, the database can write the
body of a clean buffer to the flash cache, enabling reuse of its main
memory buffer. The database keeps the buffer header in an LRU list in
main memory to track the state and location of the buffer body in the
flash cache. If this buffer is needed later, then the database can read
it from the flash cache instead of from magnetic disk.
Oracle’s
Exadata whitepaper also notes that there is a substantial difference in
the way that Oracle manages SSD inside the new flash_cache buffer area:
The Exadata
Smart Flash Cache manages active data from regular disks in the Exadata cell
– but it is not managed in a simple Least Recently Used (LRU) fashion.
The Exadata
Storage Server Software in cooperation with the Oracle Database keeps track
of data access patterns and knows what and how to cache data and avoid
polluting the cache.
This
functionality is all managed automatically and does not require manual
tuning.
If there are
specific tables or indexes that are known to be key to the performance of a
database application they can optionally be identified and pinned in cache.
Speed improvement using flash_cache
On Oracle, SSD is up to 600 times faster than platter
disks, and at only $2,000 per gigabyte. SSD technology is going to replace
platter disks for most systems by 2015, and the 2009 Sun server have
on-board SSD flash memory. Here are current
Oracle SSD vendors,
and here is a good book on using SSD with Oracle:
Oracle’s
Exadata whitepaper notes that the flash cache SSD is several orders of
magnitude faster than platter based disk.
Each Exadata
cell comes with 384 GB of Exadata Smart Flash Cache. This solid state
storage delivers dramatic performance advantages with Exadata storage. It
provides a ten-fold improvement in response time for reads over regular
disk; a hundred-fold improvement in IOPS for reads over regular disk; and is
a less expensive higher capacity alternative to memory. Overall it delivers
a ten-fold increase performing a blended average of read and write
operations.
When to use the flash_cache feature
The Oracle documentation suggests enabling the
flash_cache when the data buffer advisor suggests that Oracle wants
more RAM (a case which is almost always true!), when you are disk I/O bound
and when you have spare CPU cycles. They mention CPU because moving to
solid-state flash storage removes disk I/O, but changes the workload
bottleneck to CPU in many cases. Here is a
benchmark of
SSD showing how flash differs from platter storage.
In other words, flash_cache is for every
system that is not already running a SSD back end!
- The Buffer Pool Advisory section of your Automatic Workload
Repository (AWR) report or STATSPACK report indicates that doubling the
size of the buffer cache would be beneficial.
- db file sequential read is a top wait event.
- You have spare CPU.
Of course, many Oracle shops have been running
solid-state disks for many years now, it's not really a new technology.
In traditional SSD, the SSD looks just like a platter disk, except that the
I/O can be hundreds of times faster!
KEEP pool vs. flash_cache
It's important to note that flash_cache is different
from the KEEP pool in both the media type (RAM vs. flash SSD) as well as the
arguments. When using the KEEP pool, you can write automated
scripts to detect popular object and cache them:
The flash_cache is a new storage clause
argument, very similar to the KEEP pool:
alter|create table|index
objectname
storage
(
buffer_pool { keep | recycle | default }
flash_cache { keep | none |
default }
);
One confusing aspect of the flash_cache is that
Oracle is treating the onboard flash memory as if it were RAM, and not disk!
For example, note that the flash_cache syntax also include a KEEP
argument, just like the buffer_pool assignment:
alter
table
fred
storage (flash_cache keep);
alter
table
fred
storage (buffer_pool keep);
Enabling the flash_cache
Obviously, your server must be equipped with onboard
flash SSD, but you must also specify the flash_cache in the
init.ora deck.
Unlike the KEEP pool, the new flash_cache
option requires setting two new flash_cache parameters,
db_flash_cache_file=/dev/miountpoint and db_flash_cache_size=32G,
to specify the total size of the onboard flash cache:
SQL> show parameter flash
NAME
TYPE VALUE
————————————
———– ——————————
db_flash_cache_file
string /dev/mountpoint
db_flash_cache_size
big integer 1048576000
In sum, the flash_cache is not much more than
yet another method for segregating high-impact objects, and it is not clear
how the flash_cache differs from traditional SSD flash system where the SSD
is mounted just like a disk.
Note: If you do not have the onboard
flash_cache Sun server and you try to enable the flash_cache, you get
this error message:
ORA-00439:
feature not enabled: Server Flash Cache
ORA-01078: failure in processing system parameters
In addition to object segregation, the flash_cache
may also be beneficial to RAC systems, since RAC databases love to run
on flash drives because it removes
bandwidth saturation at the cache fusion layer as a source of
contention:
The Oracle documentation says that mounting a platter
disk as a flash_cache device will make performance suffer, but they
do not say how this flash_cache feature is optimized for SSD flash
storage.
Note: As of September 2009 the flash_cache
features is not available for Linux but Metalink note 8974084: META BUG FOR
FLASH CACHE 11.2 PL BUGS TO BACKPORT TO 11.2.0.1 OEL
