VMware Performance Tuning

While in search of tuning information for VMware, I came across much of the following information and have found it to be extremely beneficial.

Windows Configuration

Windows VM:
1024MB Ram 10GB SCSI disk, 1 Ethernet device, USB

It's recommended by VMWare that Windows VMs be configured to use IDE, however in my reading,  the Virtual IDE devices use tons more CPU time than the SCSI device. This is due to the emulation level done and lack of I/O threading in VMWare's IDE controller. I have to assume that this is a problem with IDE in general, as it's never been very good at multithreaded I/O (this is one big reason it's never been used for servers). Additionally,it is recommended to use the LSILogic controller. It supports multithreaded IO while the Buslogic controller doesn't.

Print Server Setup
Printing to a USB printer directly connected to the VMWare server.  Ubuntu doesn't configure USBFS out of the box. This can be corrected by editing a few files:

Add to fstab:
usbfs /proc/bus/usb usbfs auto 0 0

Edit /etc/init.d/mountdevsubfs.sh, and uncomment the following lines:

#mkdir -p /dev/bus/usb/.usbfs
#domount usbfs "" /dev/bus/usb/.usbfs -obusmode=0700,devmode=0600,listmode=0644
#ln -s .usbfs/devices /dev/bus/usb/devices
#mount --rbind /dev/bus/usb /proc/bus/usb

This is done by removing the # from the front of each line. Once this is done, go ahead and run the script.

/etc/init.d/mountdevsubfs.sh start

In the Virtual Machine configuration, to ensure that the printer was always connected on startup, insert the following configuration into that Virtual Machine's VMX file:

usb.present = "TRUE"
usb.generic.autoconnect = "FALSE"
usb.autoConnect.device0 = "0x0000:x0000"
usb.autoConnect.device1 = "0x04e8:0x327e"
usb.generic.skipsetconfig = "TRUE"

You can get the IDs for your devices by issuing an lsusb on the VMWare Server, this command will output similar to the following:

Bus 002 Device 002: ID 04e8:327e Samsung Electronics Co., Ltd
Bus 002 Device 001: ID 0000:0000
Bus 001 Device 001: ID 0000:0000

Additionally, I blacklisted usblp on the VMWare Server so the host didn't connect to the printer making it unavailable to the guest.

echo "blacklist usblp" >>/etc/modprobe.d/blacklist

Printing now "just works" after rebooting. Of course for it to "just work" you also need to configure CUPS or Windows printer shareing, but that is out of the scope of this article.

Add each of these to /etc/vmware/config:

mainMem.useNamedFile tells VMWare where to put it's temporary workspace file. This file contains the content of the Virtual Machine memory which is not used. By default it is placed in the directory with the virtual machine, however that can seriously impact performance so we'll turn it off.

mainMem.useNamedFile = FALSE

tmpDirectory is the default path for any temp files. We need to change that to be a shared memory filesystem (in RAM).

tmpDirectory="/dev/shm"

prefvmx.useRecommendedLockedMemSize and prefvmx.minVmMemPct tell VMWare to either use a fixed sized memory chunk or balloon and shrink memory as needed. Since I have 8GB of memory in this server I want to make sure that I use a fixed chunk of memory to reduce disk IO.

prefvmx.useRecommendedLockedMemSize="TRUE"
prefvmx.minVmMemPct="100"

To tune each Virtual Machine, I installed VMWare tools and then made the following changes to each VMX file:

Set the time in the Virtual Machine to the hosts time (I use NTP on the host):

tools.syncTime = "TRUE"

When I reboot the host, I want to gracefully stop each VM instead of just powering it off:

autostop = "softpoweroff"

This tells the VM to not share which saves CPU cycles:

mem.ShareScanTotal=0
mem.ShareScanVM=0
mem.ShareScanThreshold=4096
sched.mem.maxmemctl=0
sched.mem.pshare.enable = "FALSE"

This basically performs the same action as the configuration I put in /etc/vmware/config by telling the VM to eliminate the temp files and not to balooning and shrink memory, however it doesn't hurt anything to have it in both locations:

mainMem.useNamedFile = "FALSE"
MemTrimRate = "0"
MemAllowAutoScaleDown = "FALSE"

In order for the VMWare configuration to work properly with shared memory, you'll need to increase the default shared memory size for tmpfs to match the amount of memory in your system. This can be done by editing /etc/default/tmpfs:

SHM_SIZE=5G

You can use 'mount -o remount /dev/shm' and 'df -h' to implement and verify the change.

Or if needed...Create an in-memory temp drive

Configure /etc/sysctl.conf on the VMWare Server which configures the kernel to perform better as a Virtual Server by inserting the following configuration:

vm.swappiness = 0
vm.overcommit_memory = 1
vm.dirty_background_ratio = 5
vm.dirty_ratio = 10
vm.dirty_expire_centisecs = 1000
dev.rtc.max-user-freq = 1024

Lastly, I disable the tickless kernel option in kernel 2.6.26 which further reduces the Virtual Machine I/O constraints by reverting back to using ticks which is better supported by VMWare. This can be done by adding the following option to the kernel options line in /boot/grub/menu.lst or /etc/lilo.conf:

nohz=off

Example:   
kernel   /vmlinuz-2.6.26-1-686-bigmem root=/dev/sda3 ro quiet hdb=noprobe nohz=off