Call now: 252-767-6166  
Oracle Training Oracle Support Development Oracle Apps

 
 Home
 E-mail Us
 Oracle Articles
New Oracle Articles


 Oracle Training
 Oracle Tips

 Oracle Forum
 Class Catalog


 Remote DBA
 Oracle Tuning
 Emergency 911
 RAC Support
 Apps Support
 Analysis
 Design
 Implementation
 Oracle Support


 SQL Tuning
 Security

 Oracle UNIX
 Oracle Linux
 Monitoring
 Remote s
upport
 Remote plans
 Remote
services
 Application Server

 Applications
 Oracle Forms
 Oracle Portal
 App Upgrades
 SQL Server
 Oracle Concepts
 Software Support

 Remote S
upport  
 Development  

 Implementation


 Consulting Staff
 Consulting Prices
 Help Wanted!

 


 Oracle Posters
 Oracle Books

 Oracle Scripts
 Ion
 Excel-DB  

Don Burleson Blog 


 

 

 


 

 

 
 

Oracle RAC block size & cache size Configuration tips

Oracle Tips by Burleson Consulting
January 27, 2004 - Updated October 4, 2014

Block Size and RAC

The choice of the database block size can have an impact on the Cluster Interconnect. One normally considers the block size parameter, db_block_size in relation to disk I/O operations. For Oracle RAC databases, Cache Fusion will also be performing cross-instance I/O as well. It should come as no surprise that the database block size can impact global cache transfers as well. See these notes on Oracle_block_size benefits.

NOTE:  RAC likes small blocksizes:  A 2k db_block_size is recommended for all OLTP RAC databases.

 

The general rule of thumb is that databases in support of data warehouses should use large blocks sizes and databases in support of online transaction systems should use smaller block sizes. Oracle RAC database often follow this convention as well.

 

The smaller the database block, the quicker a block can be read from disk. Just like disk I/O, the smaller the database block, the quicker it is transferred from one instance to another via Cache Fusion. This is just a matter of physics. No matter how fast the Cluster Interconnect is, 2 kilobytes of data can transfer across the network faster than 8 kilobytes. There are studies found which indicate transfer rates similar to the following.

 

 

Block Size

Gigabit Ethernet

Infiniband

2K

300 microseconds

100 microseconds

4K

350 microseconds

150 microseconds

8K

450 microseconds

200 microseconds

Table 3.1 Block Size Transfer Times

 

The actual transfer times do not matter and may vary considerably from environment to environment. The numbers in the table above are just meant to illustrate the effect of larger block sizes on global cache transfers.

 

Initially, one might look at the table above and conclude that for Oracle RAC databases, we should always choose the smaller block size. While a 2KB block transfers faster than a 4KB block, the amount of data being transferred per millisecond is improved as the block size increases. Using the figures from the table above, easy mathematics calculates the transfer rate as shown in the next table.

 

Block Size

Gigabit Ethernet

Infiniband

2K

0.00667 KB/ μ s

0.020000 KB/ μ s

4K

0.01142 KB/ μ s

0.026667 KB/ μ s

8K

0.01778 KB/ μ s

0.040000 KB/ μ s

table 3.2 Block Size Throughput

 

The values in the table above were derived by simple math, i.e. block size divided by time in microseconds. In any case, the larger the block size, the more data per microsecond is transferred.

 

So based on the information in both tables, which block size should be used? The answer all depends on the application and how it accesses data. If the application accesses few rows at a time, then the smaller block size is preferred as it returns the data back to the application quicker than larger block sizes. If the application regularly accesses multiple rows or large amounts of data then a larger block size is preferred.

 

To illustrate the concept using the example times above, consider an online transaction processing system where a query selects one row from the table and the table has an average row length of 148 bytes. A database block size of 8KB would contain, on average, 55 rows of data. A database block size of 2KB would contain, on average, 13 rows of data. If the application is seeking 1 row, the 8KB block size would force a read of 54 unneeded rows of data while the 2KB block size would force a read of 12 unneeded rows of data. The one row would be returned in 300 microseconds if the 2KB block size is used or 450 microseconds for the 8KB block size. For applications that exhibit this behavior, a smaller block size is preferred. The application would benefit from the metrics seen in the first table of this section.

 

On the flip side is the application that processes many rows of data and each row is larger than as described in the previous paragraph. Assume the application selects thousand rows of the table at a time and each row in the table averages 533 bytes. The typical data access pattern would seek 533,000 bytes at a time. For the 2KB block size, 261 blocks would be needed whereas for the 8KB block size, 66 blocks would be needed. In this example, the application would benefit from the metrics in the second table of this section.

 

The examples above assumed that each data block was completely full, i.e. no space reserved for future update or insert operations. The examples further assume that all rows were the average size and they packed nicely into the block. Data in blocks rarely cooperate in this fashion in the real world. The examples are used to illustrate the points made in this section. However, the main ideas of each example will hold true even if a parameter like pctfree is used to reserve space for future updates. In the end, the only thing that matters is end user application performance. If you are considering the block size to be used, perform adequate testing of the application with different block sizes to determine the most optimal setting. Remember that Oracle has one default block size but you can also employ non-default block sizes as well to have the best of all worlds. Many of today?s applications do not exhibit true OLTP or DW behavior but have characteristics of both. One could choose a block size that is in the middle of allowable values or one could deploy multiple block sizes for their application.

All Oracle DEFAULT caches and block sizes in a RAC environment (as defined by db_block_size) must be identical, but there are other important differences between the configuration of a RAC database and a single-instance Oracle database, especially when using grid server blades.

Beginning with Oracle RAC Oracle allowed sub-caches to be configured that have different blocksizes than the default db_cache_size, allowing db_2K_cache_size, db_4k_cache_size, db_8k_cache_size, db_16k_cache_size and db_32k_cache_size (not available on all platforms) as sub-caches.

 For example, if your default db_block_size is 4K, you could define a db_2k_cache_size to hold additional objects.  However, all RAC nodes must have the sub-cache defined.

We must remember that the rules for caching in a RAC instance are very different than for a standard Oracle system.  For example, numerous studies have shown that Oracle indexes often benefit from using the largest supported blocksize.  This is not always true in RAC systems.

With Oracle system now employing RAM-SAN instead of disk, and systems having giant db_cache_size regions, it is tempting to configure RAC with large data buffers.  However, we must remember that multi-instance Oracle is very different from a single-instance system:

  • Blade clusters have small buffer caches: Most blade servers have only 2-gig or 4-gig of RAM.  Hence, each RAC node is limited in the available db_cache_size.
     

  • RAC likes small blocksizes:  Because of the inter-instances block transfer via Cache Fusion, smaller block sizes minimize pinging of blocks between instances.  Most RAC DBA's will define the default db_block_buffers to 2k and then add a 4k buffer to isolate other data objects, such as indexes.  See MOSC Note: 422844.1 for details.
     

  • RAC scales by the sum of RAM caches: Unlike a single Oracle database with a 32 gigabyte RAM data cache, RAC system achieve high caching by the sum of the individual RAM caches.  Hence, a 32 node RAC cluster with a 2 gigabyte RAM cache on each node would effectively have a total cache of 128 gigabytes.

See my related notes on RAC caching and block sizes:
 

 
 
 
Learn RAC Tuning Internals!

This is an excerpt from the landmark book Oracle RAC Performance tuning, a book that provides real world advice for resolving the most difficult RAC performance and tuning issues.

Buy it  for 30% off directly from the publisher.

 

 

��  
 
 
Oracle Training at Sea
 
 
 
 
oracle dba poster
 

 
Follow us on Twitter 
 
Oracle performance tuning software 
 
Oracle Linux poster
 
 
 

 

Burleson is the American Team

Note: This Oracle documentation was created as a support and Oracle training reference for use by our DBA performance tuning consulting professionals.  Feel free to ask questions on our Oracle forum.

Verify experience! Anyone considering using the services of an Oracle support expert should independently investigate their credentials and experience, and not rely on advertisements and self-proclaimed expertise. All legitimate Oracle experts publish their Oracle qualifications.

Errata?  Oracle technology is changing and we strive to update our BC Oracle support information.  If you find an error or have a suggestion for improving our content, we would appreciate your feedback.  Just  e-mail:  

and include the URL for the page.


                    









Burleson Consulting

The Oracle of Database Support

Oracle Performance Tuning

Remote DBA Services


 

Copyright © 1996 -  2017

All rights reserved by Burleson

Oracle ® is the registered trademark of Oracle Corporation.

Remote Emergency Support provided by Conversational