Tuning Linux for proper CPU setup is actually quite easy there are just two items to address. First, make sure to install the correct address space version to match your CPU architecture (i.e. 32-bit vs. 64-bit). I have not found this to make a substantial and directly measureable impact. It is simply that 64-bit environments offer larger address spaces and thus larger SGAs, which can often potentially reduce overall I/O. Second, make sure to always install the SMP kernel regardless of your hardware. A single CPU running an SMP kernel will suffer a very small but acceptable performance penalty. But remember, think more generically as you tune. Today's single may well be tomorrows double or even quad. To best reap those potential performance rewards, you should err on the side of potentially more CPUs in the future.

Optimizing Linux memory usage is much like the CPU - assume you have the minimum requirements to meet your business SLA needs, but keep in mind that it could grow at any time. However, unlike the CPU where you can simply choose the single option for SMP kernel to cover all bases, you potentially have two choices with regard to memory configuration.

If using 32-bit Linux, then the address space is limited by hardware to 4GB with a Linux imposed split of 3GB for user space and 1GB for kernel space. However, there are some well known workarounds as shown in the table on the following page (all of which are well documented on Oracles MOSC web site and in numerous Oracle book & papers). But the days of 32-bit are numbered, so it is better to accept the inevitable and adopt 64-bit as soon as possible.

Table 1: Examples of Workarounds

Of course, if you are using 64-bit Linux, this entire memory issue is essentially eliminated. However, there is one more little tweak that can be applied to either 32-bit or 64-bit Oracle servers and that is the use of Huge Pages. This Linux 2.6 kernel feature simply utilizes larger than the 4K pages to reduce virtual memory I/O operations when working with lots of memory. Here are some documented limits:

The process to enable huge pages is as follows:

• X = grep Hugepagesize /proc/meminfo

• Y = Largest (MB of all client SGAs) * 1024

• Z = # Huge Pages needed = Y / X

• Set Huge Page Pool size

edit /etc/sysctl.con
vm.nr_hugepages = Z

• Increase ulimit parameter memlock for oracle user

  • edit /etc/security/limits.conf

  • oracle soft   memlock          Y

  • oracle hard  memlock          Y

• reboot

Important note:  Some experts DO NOT recommend using Automatic Memory Management (AMM, e.g. setting memory_target) with Linux hugepages.  See MOSC note 749851.1 "HugePages and Oracle Database 11g Automatic Memory Management (AMM) on Linux". 

In sum, AMM is not compatible with Linux Hugepages.  Also, beware that AMM re-size operations can cripple Oracle performance in some cases.

Interestingly enough, many people with killer 64-bit servers increase their SGA size without implementing Huge Pages. The results have been well documented (see Oracles MOSC document id = 361670.1) where SGA sizes greater than 10GB have displayed decreases in performance! So as a general practice, always implement Huge Pages.

To improve I/O for file system based Oracle data files, Linux offers a little known and seldom used option that can yield between 50-150% performance improvements in standard database benchmarks like the TPC-C by simply changing the /etc/fstab file entries for our Oracle data file mount points as follows:

• For ext2 and 3 file systems, add ,noatime to the third column.

What this does is tell the operating system that it is not necessary to update the last access time for directories and files under that mount point, which translates into radically reduced total I/O. Since the Oracle background processes are already accessing the data files every three seconds and have their own headers with timestamps within them, why spend I/O resources to update time attributes for files or directories?

To improve I/O for ASM based oracle data files, simply double the default SGA sizing parameter for the ASM instance from 64MB to 128MB. Memory is far too cheap these days to haggle over such a small amount. And the obvious results will more than justify the cost.
 

This is an excerpt from Oracle on VMWare: Expert tips for database virtualization by Rampant TechPress.