This is an excerpt from the bestselling book
Oracle Grid & Real Application Clusters. To get immediate
access to the code depot of working RAC scripts, buy it
directly from the publisher and save more than 30%.
Database systems provide data
concurrency by enabling multiple users to access the same data
without compromising data consistency. Data consistency means that
each user sees a consistent view of the data, including the visible
changes made by the users? own transactions, as well as the
transactions of other users. Oracle automatically supplies a query
with read-consistent data, so that all data that the query sees
comes from a single point in time (statement-level read
Optionally, Oracle can provide
read consistency to all queries in a transaction (transaction-level
read consistency). Oracle maintains undo records to manage such
consistent views. The undo segments contain the old data values that
have been changed by uncommitted or recently committed transactions.
In a RAC system, users can
connect with multiple instances to run database queries. Typically,
users will be connected to different nodes but access the same set
of data or data blocks. This situation demands that the data
consistency, formerly confined to a single instance, be effectively
extended to multiple instances. Therefore, buffer cache coherence
from multiple instances must be maintained.
Instances require three main
types of concurrency:
* Concurrent reads on multiple
instances ? When users on two different instances need to read the
same set of blocks.
* Concurrent reads and writes on
different instances - A user intends to read a data block that was
recently modified, and the read can be for the current version of
the block or for a read-consistent previous version.
* Concurrent writes on different
instances ? When the same set of data blocks are modified by
different users on different instances.
Whether the database is a
single-instance stand-alone system or a multi-instance RAC system,
maintaining data consistency is a fundamental requirement. If data
blocks are already available in the local buffer cache, then they
are immediately available for user processes. And, if they are
available in another instance within the cluster, they will be
transferred into the local buffer cache.
Maintaining the consistency of
data blocks in the buffer caches of multiple instances is called
cache coherency. The Global Cache Service (GCS), implemented by a
set of Oracle processes requires an instance to acquire cluster-wide
data before a block can be modified or read. In this way, cache
coherency is ensured and maintained. This resource can be explained
in terms of enqueue and/or lock.
GCS synchronizes global cache
access, allowing only one instance at a time to modify the block.
Thus, cache coherency is maintained in the RAC system by
coordinating buffer caches located on separate instances. GCS
ensures that the data blocks cached in different cache buffers are
maintained globally. That is why some people prefer to call cache
fusion a diskless cache coherency mechanism. This is true in a
sense, because the previous Oracle parallel server version (OPS)
utilized forced disk writes to maintain cache coherency.
Global Cache Service (GCS)
GCS is the main controlling
process for cache fusion. It tracks the location and status (mode
and role) of the data blocks, as well as the access privileges of
the various instances. GCS guarantees data integrity by employing
global access levels. It maintains block modes for data blocks in
the global role. It is also responsible for block transfers between
instances. As shown in Figure 7.2, upon a request from an instance,
GCS organizes the block shipping and the appropriate lock mode
conversions. Various background processes, such as global cache
service processes (LMSn) and the global enqueues service daemon (LMD),
implement the global cache service.
Figure 7.2: Message/Resource
Exchange controlled by GCS
Before going further into a
detailed examination of the cache fusion mechanism and how GCS
operations are performed in different scenarios, the next section
will take a look at and recap some basic SGA structures and locking
concepts. A more detailed review of SGA structures was provided in
RAC Cache Coherency
The Oracle data blocks are a global resource
that is available to any user or process in the clustered
database, and each instance needs to work with the other instances
in a cohesive effort to manage the blocks.
Coherency is defined in many dictionaries as
?the act of uniting or sticking together?. For Oracle RAC
databases, cache coherency is defined as the collective effort of the instances
to appear as if the Buffer Caches are one larger, logical unit.
Implementing cache coherency in a clustered environment is no
small task. There is a lot of overhead involved in making sure
that each instance knows what is in the other caches. This process
overhead increases overall response time.
The trick, (which Oracle RAC performs very
well), is keeping cache coherency impacts to a minimum for most
applications. Unfortunately, cache coherency can also act as a
magnifying glass and make some performance problems worse than if
the application were run on a single-instance database.
At the heart of Oracle RAC?s cache coherency is
Global Cache Services
(GCS). GCS manages resources and everything in Oracle RAC is a
resource. If a row needs to be modified in a table, that resource
needs to be acquired before the transaction is allowed to make the
change. If this were a
single-instance database and a simultaneous transaction needed to
modify the same row, the second transaction would need to wait for
the transaction lock (TX) to be released.
In Oracle RAC, the TX locks
still exist in all instances.
Consider the scenario where the second
transaction is executing in a different instance than the first
transaction. GCS and its partner,
Global Enqueue Services
(GES) need to ensure that the second transaction cannot be
performed until the first session releases its TX lock in the
first instance. GCS tracks the locations of the blocks in all of
the collective buffer caches. To do this, all blocks in the Buffer
Cache have a master copy
of the block in only one instance, but other instances may have
copies of the block as well. The GCS ensures cache coherency by
keeping track of these copies and ensuring the blocks are
consistent across the cluster. There can be multiple versions of a
block to support consistent reads for end users. GES is
responsible for keeping track of the block versions.
In order to keep track of the blocks and their
versions, Oracle RAC sets up a
Global Resource Directory
(GRD). Each instance?s
Shared Global Area (SGA) sets aside a portion of memory for
the GRD. Whenever a resource is used, GCS will assign one instance
as the resource master for that resource. All of the instances can then
determine the resource master for any particular resource.
Remember, it is not necessary that the master copy of the block be
in the same instance as the resource master. All of these working
components, GCS, GES, and GRD, combine to form cache coherency for