Oracle 11g release 2 includes a feature called flash_cache, a table and index argument that resembles the assignment of high-use objects to the KEEP pool.

However, unlike the KEEP pool which uses volatile RAM disk, the flash_cache is used for tertiary storage on solid-state disk (SSD).  Normally, when a data block is aged-out of the data buffer cache, it must be reloaded from disk the next time it are required. With Oracle Smart Flash Cache feature, the aged out blocks are placed in the Smart Flash Cache, so if they are needed again they are reloaded more quickly into the buffer cache from SSD, rather than having to be reloaded from the slower disks.

In plain English, Oracle is recognizing a hierarchy of storage hardware, organized by access speed:

• Data Buffer RAM - The KEEP pool can cache objects for super-fast access at RAM speeds.  Of course, this data is not persistent.

• Solid-state disk - SSD is not as fast as native RAM, but it remains hundreds of times faster than platter disks.  However, all SSD suffers from a severe write penalty and is not ideal for high DML data.

• Disk spindles - The old magnetic-coated spinning platters from the 1960's comprise the bulk of Oracle data storage, with stone-age access speeds expressed in milliseconds.

The docs suggest that the flash_cache refers to "flash" SSD memory, but it appears that this approach is using SSD as a temporary cache and not as permanent home for the data, as we see in SSD flash memory. 

Many people misunderstand that the "flash memory" is very different from the rack-mounted industrial strength solid-state disk.  For complete timing details on rack-mount SSD, see the books "Oracle tuning with solid-state disk" and "Oracle RAC and Grid tuning with SSD".

In traditional SSD, the flash memory is an external rack mount device and it looks like just a bunch of disks (JBOD), and the Oracle data files are allocated directly onto the flash media.  Conversely, in Oracle's approach, the flash_cache refers to internal flash SSD, and the flash_cache syntax treats it like "temporary" caching instead of permanent storage. 

Note: Guy Harrison describes  a patch that allows you to use Oracle  flash_cache without a million dollar Exadata server. He also offers this script for displaying flash_cache waits:

SELECT
   name,
   value
FROM
   v$sysstat
WHERE
   name IN
   ('physical read flash cache hits',
   'physical reads',
   'consistent gets',
   'db block gets',
   'flash cache inserts');

Also see my db_flash_cache_file and db_flash_cache_size parameters tips.

Guy also notes that you can see flash_cache entries in the data buffers with the v$bh view, where the stats starts with flash:

select
   owner||'.'||object_name
from
   v$bh, dba_objects
where
 v$bh.status like ('flash%'); 

He also lists the new flash_cache system statistics:

 

Flash_cache buffer changes

The 11g r2 docs note that the flash_cache is also used to segregate objects, away from the "main memory buffer":

A flash cache is an extension of the database buffer cache that lives on a flash disk, which is a solid state storage device that uses flash memory.

• Without flash cache, the database re-uses each clean buffer in main memory as needed, overwriting it. If the overwritten buffer is needed later, then the database must read it from magnetic disk. 
  
• With flash cache, the database can write the body of a clean buffer to the flash cache, enabling reuse of its main memory buffer. The database keeps the buffer header in an LRU list in main memory to track the state and location of the buffer body in the flash cache. If this buffer is needed later, then the database can read it from the flash cache instead of from magnetic disk.

Oracle's Exadata whitepaper also notes that there is a substantial difference in the way that Oracle manages SSD inside the new flash_cache buffer area:

The Exadata Smart Flash Cache manages active data from regular disks in the Exadata cell - but it is not managed in a simple Least Recently Used (LRU) fashion.

The Exadata Storage Server Software in cooperation with the Oracle Database keeps track of data access patterns and knows what and how to cache data and avoid polluting the cache.

This functionality is all managed automatically and does not require manual tuning.

If there are specific tables or indexes that are known to be key to the performance of a database application they can optionally be identified and pinned in cache.

I/O against the flash cache is recorded separately from platter-disk I/O, and we now see two new types of I/O on AWR and STATSPACK reports:

• db flash cache single block physical read: The flash equivalent of db file scattered read waits
• db flash cache multiblock physical read:  The flash equivalent of db file sequential read waits
• write complete waits: flash cache: The flash equivalent of write complete waits

 

Speed improvement using flash_cache

On Oracle, SSD is up to 600 times faster than platter disks, and at only $2,000 per gigabyte.  SSD technology is going to replace platter disks for most systems by 2015, and the 2015 Sun server have on-board SSD flash memory.  Here are current Oracle SSD vendors, and here is a good book on using SSD with Oracle:

• Oracle solid state disk tuning 

Oracle's Exadata whitepaper notes that the flash cache SSD is several orders of magnitude faster than platter based disk. 

Each Exadata cell comes with 384 GB of Exadata Smart Flash Cache. This solid state storage delivers dramatic performance advantages with Exadata storage. It provides a ten-fold improvement in response time for reads over regular disk; a hundred-fold improvement in IOPS for reads over regular disk; and is a less expensive higher capacity alternative to memory. Overall it delivers a ten-fold increase performing a blended average of read and write operations.

When to use the flash_cache feature

The Oracle documentation suggests enabling the flash_cache when the data buffer advisor suggests that Oracle wants more RAM (a case which is almost always true!), when you are disk I/O bound and when you have spare CPU cycles.  They mention CPU because moving to solid-state flash storage removes disk I/O, but changes the workload bottleneck to CPU in many cases.  Here is a benchmark of SSD showing how flash differs from platter storage.

In other words, flash_cache is for every system that is not already running a SSD back end!

• The Buffer Pool Advisory section of your Automatic Workload Repository (AWR) report or STATSPACK report indicates that doubling the size of the buffer cache would be beneficial.
  
  
• db file sequential read is a top wait event.
  
  
• You have spare CPU.

Of course, many Oracle shops have been running solid-state disks for many years now, it's not really a new technology.  In traditional SSD, the SSD looks just like a platter disk, except that the I/O can be hundreds of times faster!

KEEP pool vs. flash_cache

It's important to note that flash_cache is different from the KEEP pool in both the media type (RAM vs. flash SSD) as well as the arguments.  When using the KEEP pool, you can write automated scripts to detect popular object and cache them to stay in RAM as long a the instance is running:

• Oracle Automating KEEP Pool Assignment

Conversely, with smart flash cache, Oracle moves aged-out data blocks from the RAM data buffer into the flash cache area, instead of letting it age-out to disk.

The flash_cache is a new storage clause argument, very similar to the KEEP pool:

alter|create table|index
   objectname
storage
(
   buffer_pool { keep | recycle | default }
   flash_cache { keep | none    | default }
);

One confusing aspect of the flash_cache is that Oracle is treating the onboard flash memory as if it were RAM, and not disk!  For example, note that the flash_cache syntax also include a KEEP argument, just like the buffer_pool assignment:

alter table
   fred
storage (flash_cache keep);

alter table
   fred
storage (flash_cache none);

alter table
   fred
storage (flash_cache default);

 

Enabling the flash_cache

Obviously, your server must be equipped with onboard flash SSD, but you must also specify the flash_cache in the init.ora deck.

Unlike the KEEP pool, the new flash_cache option requires setting two new flash_cache parameters, db_flash_cache_file=/dev/miountpoint and db_flash_cache_size=32G, to specify the total size of the onboard flash cache:

SQL> show parameter flash

NAME                                 TYPE         VALUE
————————————                         ———–         ——————————
db_flash_cache_file                  string       /dev/mountpoint
db_flash_cache_size                  big integer  1048576000

In sum, the flash_cache is not much more than yet another method for segregating high-impact objects, and it is not clear how the flash_cache differs from traditional SSD flash system where the SSD is mounted just like a disk. 

Note:  If you do not have the onboard flash_cache Sun server and you try to enable the flash_cache, you get this error message:

ORA-00439: feature not enabled: Server Flash Cache
ORA-01078: failure in processing system parameters

Disabling Smart Flash Cache

Resetting the initialization parameters disables the Smart Flash Cache.

ALTER SYSTEM RESET DB_FLASH_CACHE_FILE SCOPE=SPFILE;
ALTER SYSTEM RESET DB_FLASH_CACHE_SIZE SCOPE=SPFILE;
SHUTDOWN IMMEDIATE;
STARTUP;

SELECT * FROM v$flashfilestat;

no rows selected

You can also display details about the statistics for the flash file statistics:

set lines 100
col name format a20

select
   *
from
   v$flashfilestat;

In addition to object segregation, the flash_cache may also be beneficial to RAC systems, since RAC databases love to run on flash drives because it removes bandwidth saturation at the cache fusion layer as a source of contention:

• Oracle RAC tuning with solid-state disk

The Oracle documentation says that mounting a platter disk as a flash_cache device will make performance suffer, but they do not say how this flash_cache feature is optimized for SSD flash storage.

Note: As of September 2015 the flash_cache features is not available for Linux but MOSC note 8974084: META BUG FOR FLASH CACHE 11.2 PL BUGS TO BACKPORT TO 11.2.0.1 OEL

DRAM SSD vs. Flash SSD

With all the talk about the Oracle "flash cache", it is important to note that there are two types of SSD, and only DRAM SSD is suitable for Oracle database storage.  The flash type SSD suffers from serious shortcomings, namely a degradation of access speed over time.  At first, Flash SSD is 5 times faster than a platter disk, but after some usage the average read time becomes far slower than a hard drive.  For Oracle, only rack-mounted DRAM SSD is acceptable for good performance:

	Avg. Read speed	Avg. write speed
Platter disk	10.0 ms.	7.0 ms.
DRAM SSD	0.4 ms.	0.4 ms.
Flash SSD	1.7 ms.	94.5 ms.