Call now: 252-767-6166  
Oracle Training Oracle Support Development Oracle Apps

 
 Home
 E-mail Us
 Oracle Articles
New Oracle Articles


 Oracle Training
 Oracle Tips

 Oracle Forum
 Class Catalog


 Remote DBA
 Oracle Tuning
 Emergency 911
 RAC Support
 Apps Support
 Analysis
 Design
 Implementation
 Oracle Support


 SQL Tuning
 Security

 Oracle UNIX
 Oracle Linux
 Monitoring
 Remote s
upport
 Remote plans
 Remote
services
 Application Server

 Applications
 Oracle Forms
 Oracle Portal
 App Upgrades
 SQL Server
 Oracle Concepts
 Software Support

 Remote S
upport  
 Development  

 Implementation


 Consulting Staff
 Consulting Prices
 Help Wanted!

 


 Oracle Posters
 Oracle Books

 Oracle Scripts
 Ion
 Excel-DB  

Don Burleson Blog 


 

 

 


 

 

 
 

RAC Private Network tips

RAC tuning tips

October 3,  2015

 

Question:  Can you explain what a RAC private network does?  Why is it important to have the private network tuned properly in a RAC cluster interconnect?

Answer:  The Cluster Interconnect is also referred to as the private network. This is a private interconnect and the only traffic on this network should be for cluster operations, including Cache Fusion. There is a misconception that the private network is used only for the cluster heartbeat, a pulse sent from one node to another to signify that the node is alive. A heartbeat is a lightweight operation on a network so why devote a network switch only usable for the private network?

This incorrect line of thinking leads to some Oracle RAC deployments that combine the public and private networks on the same switch.  Other RAC deployments may separate the public and private networks, but put the private network on a switch that is used for other network purposes. Each of these configuration options is a mistake that many people later regret when Oracle RAC performance suffers. The private network needs to remain private, solely and exclusively for the use of the Cluster Interconnect.

 

Chapter 2 has already shown how the private network is used for so much more than just a cluster heartbeat, and it should be clear that the private network is used for any cross-instance communications. When two sessions on different instances want to modify the same block, Global Enqueue Services 'talk? across the Cluster Interconnect.  Global cache transfers of data blocks will dominate the private network traffic. Examine these metrics from an AWR report.

 

 

Global Cache Load Profile

~~~~~~~~~~~~~~~~~~~~~~~~~                  Per Second       Per Transaction

                                      ---------------       ---------------

  Global Cache blocks received:              1,279.68                 79.62

    Global Cache blocks served:                637.30                 39.65

     GCS/GES messages received:              4,367.95                271.76

         GCS/GES messages sent:              7,440.72                462.94

            DBWR Fusion writes:                 21.15                  1.32

 Estd Interconnect traffic (KB)             17,642.23

 

The metrics above were taken from one instance of a three-node RAC database over a one-hour period of time. The metrics above show an average of 1,916.98 global cache blocks sent and received per second. The database block size is 8 Kbytes, which means there are 15,335.84 kilobytes of traffic each second on the private network just to perform global cache transfers. The total private network traffic is 17,642.23 kilobytes is for all cross-instance traffic including the global cache transfers.

 

The total Cluster Interconnect traffic for this one-hour snapshot is about 17.6 megabytes. These metrics prove that the Cluster Interconnect is used for so much more than just heartbeat traffic. These metrics should also reinforce the notion that the private network needs to remain private, as it can be busy. It is not uncommon to see some Oracle RAC deployments that have many gigabytes per hour of traffic on the Cluster Interconnect.

 

On the Oracle RAC nodes, it is very easy to determine which interfaces support the public and private networks for the cluster. The Oracle Interface Configuration Tool (oifcfg) can provide that information.  

 

[root@host02 oracle]# cd /u01/app/crs12.1.0.1/bin

[root@host02 bin]# ./oifcfg getif

eth0  192.168.56.0  global  public

eth1  192.168.10.0  global  cluster_interconnect

 

 

From the output above, we can see that device eth0 is for the public network and eth1 is for the Cluster Interconnect. It is a requirement that the device names be the same on all nodes so if we execute the same commands on another node, the output should be the same. If another node has a different interface, cluster communications will not work correctly. It is also a requirement that the public and private interfaces be on different subnets.

 
 
 
Learn RAC Tuning Internals!

This is an excerpt from the landmark book Oracle RAC Performance tuning, a book that provides real world advice for resolving the most difficult RAC performance and tuning issues.

Buy it  for 30% off directly from the publisher.


Hit Counter

 

Burleson is the American Team

Note: This Oracle documentation was created as a support and Oracle training reference for use by our DBA performance tuning consulting professionals.  Feel free to ask questions on our Oracle forum.

Verify experience! Anyone considering using the services of an Oracle support expert should independently investigate their credentials and experience, and not rely on advertisements and self-proclaimed expertise. All legitimate Oracle experts publish their Oracle qualifications.

Errata?  Oracle technology is changing and we strive to update our BC Oracle support information.  If you find an error or have a suggestion for improving our content, we would appreciate your feedback.  Just  e-mail:  

and include the URL for the page.


                    









Burleson Consulting

The Oracle of Database Support

Oracle Performance Tuning

Remote DBA Services


 

Copyright © 1996 -  2020

All rights reserved by Burleson

Oracle ® is the registered trademark of Oracle Corporation.

 

 

��  
 
 
Oracle Training at Sea
 
 
 
 
oracle dba poster
 

 
Follow us on Twitter 
 
Oracle performance tuning software 
 
Oracle Linux poster