Cluster Volume Management (CVM)

The cluster volume manager is a software module that virtualizes the physical storage and presents the usable entities for the applications, such as the database at the operating system level. It is a storage administrator layer at the host level.

The Veritas cluster volume manager (CVM) uses objects for storage management. The two types of objects used by Volume Manager are physical objects and virtual objects.

* Physical objects ? Volume Manager uses two physical objects: physical disks and partitions. Partitions are created on the physical disks (on systems that use partitions).

* Virtual objects - Volume Manager creates virtual objects, called volumes. Each volume records and retrieves data from one or more physical disks. A volume is a virtual disk device that appears to applications, databases, and file systems as a physical disk. Volumes are also composed of other virtual objects such as sub-disks and plexes. These are used to change volume configuration features such as concatenation, mirroring, and striping. Volumes and their virtual components are called virtual objects.

Volume management involves using physical disks or LUNS to create disk groups. These disks are combined to create volumes with a layout or configuration appropriate to their purpose, for example, mirroring or striping etc. The volumes are flexible objects that can easily be extended or modified. The volume management operations include:

* Placing the shared disks or LUNS from the shared storage array into the Volume Manager control. When the LUNS are placed under the VM control, they are known as VM disks.

* Creating the Disk Groups ? the disk groups that contain the database application must be available to all nodes in the cluster. A disk group allows disks, volumes, and file systems that are relevant to a single database to be arranged into a logical collection for easy administration.

* Creating volumes with the appropriate layout (mirror, stripe. etc.) and size from the disk group storage pool. Volumes are flexible objects that can be extended or modified.

* Making file systems from the volumes and mounting them for database use. Volumes can also be used for database applications, in which case they are referred to as raw devices.

It is important to understand that volumes are the basic storage components that an Oracle database utilizes. The volumes, as created by the CVM, are accessible by all members or nodes of the cluster. This is an important distinction in the RAC environment. A volume can be accessed either directly by the RAC database as a raw partition, or through mounted file systems such as a CFS, as shown in Figure 15.16.

Figure 5.16: Veritas Cluster Volume Management

Configure NFS file system

Using NFS mount for the shared storage for the RAC database system, provides yet another option. In this section, the usage of NFS type shared storage structures will be covered by looking at the Filer System supplied by Network Appliance System.

Storage Infrastructure

* One Network Appliance F2XX/F7XX/F8XX/F9XX filer with Data ONTAP? 6.1.3R2 or later

* One Gigabit switch with at least four ports

* One Gigabit NIC in the filer

* One or more disk shelves, based on the disk space requirements

Once the hardware connection is made to the nodes in the Linux Cluster, install Data ONTAP 6.1.3 R2 and give it a name, such as FDATA1, and an IP Address for the filer storage unit.

Create a volume on the FDATA1 filer for storing Oracle Database files. This volume will be used to store all the data files, control files, and log files for the Oracle Database.

Use the at the filer console:

FDATA1> vol create oradata 14

Then Edit the /etc/exports file on FData1 and add the following entries to that file:

/vol/racdata -anon=0

Execute the following command at the filer console:

FData1> exportfs ?a

Now create mount points and mount volumes. Update the /etc/fstab file on all the server nodes and add the following entries:

FData1:/vol/racdata  -  /oradb nfs ? Yes rw , bg , hard , nointr , rsize=32768,wsize=32768,tcp,noac,vers=3,timeo=600

Where:

* FData1 is the name of the filer.

* oradb is the mount point on the cluster nodes.

* The mount options that are required for the Oracle RAC are:

* noac: This mount option disables caching on the client side.

* tcp: Mount the file system using the tcp option.

Veritas Cluster Volume Management

The Veritas Volume Management (VxVM) tool manages physical disks and presents logical disks for application use. Logical disks are known as volumes. A volume is the disk space that appears to the database as a physical partition. VxVM operates as a sub-system between the operating system and the database system.

We need to create one or more shared disk groups besides the rootdg disk group. rootdg must exist and cannot be shared between the systems. At least one disk must be in the rootdg disk group. The shared disk group is accessible for all the nodes in the cluster.

The cluster functionality of a VxVM works together with the cluster manager daemon. The CM informs VxVM of the changes in cluster membership based on which access is controlled. When a node joins a cluster it gains access to shared disks. When a node leaves a cluster it no longer has access to shared disks. A node joins a cluster when the cluster manager is started on that node.

To the cluster manager, all nodes are the same. All nodes that join the cluster can potentially access VxVM objects configured within shared disk groups. However, the cluster functionality of VxVM requires that one node act as the master node. All other nodes in the cluster are slave nodes. Any node is capable of being the master node and it is responsible for coordinating certain VxVM activities.

We must run all the commands that configure or reconfigure VxVM objects on the master node. Tasks that must be initiated from the master node include setting up shared disk groups, creating and reconfiguring volumes, and performing snapshot operations. VxVM determines that the first node to join a cluster performs the function of master node. If the master node leaves a cluster, one of the slave nodes is chosen to be the new master.

There are three interfaces to use with the volume manager:

• Volume Manager Storage Administrator (VMSA) is a graphical user interface to Volume Manager.

• Command Line Interface (CLI): The command line interface (CLI) consists of UNIX utilities that are invoked from the command line to perform Volume Manager and standard UNIX tasks. You can use the CLI not only to manipulate Volume Manager objects, but also to perform scripting and debugging functions.

• Volume Manager Support Operations (vxdiskadm): The Volume Manager Support Operations interface, commonly known as vxdiskadm, is a menu-driven, text-based interface that you can use for disk and disk group administration functions.

Placing the physical disks into the group creates a disk group and then individual volumes can be created for the various Oracle files. Let us look at some examples.

To create a disk group called oradg over 4 LUNs:

# vxdg -s init oradg1 ora01=c1t1d0s2 ora02=c1t2d0s2 ora03=c1t3d0s2 ora04=c1t4d0s2

To create a RAID 0+1 (mirror/stripe) volume across 4 disk LUNs:

# vxassist -g oradg1 make rac_system_500m 500m layout=stripe stripeunit=256 ncolumn=4 user=oracle group=dba alloc="ora01 ora02 ora03 ora04"