Wednesday, March 03, 2010

Quick Notes and Tips On VMware Tuning

This is by no means a comprehensive of detailed list of tuning of a VM running in VMWare's ESX server.  However, I wanted to add a couple of hints or tips that others my find useful.

Windows as the Guest OS:
  •  High Context Switching:  If you are experiencing high context switching, it is best to try to understand why.  Several Things can cause this:
    • COM+ calls to out of process COM+ servers
      You may want to sacrifice memory and security by making the COM+ server an in-process call. This can reduce context switches.
    • System Calls
      Not much you can do here. 
    • Large Numbers of Threads running
      Reduce the number of threads that are servicing requests, for IIS you can just reduce threads, for a home grown application this will be done progromatically and should be configurable.
    • Page Faults
      This is a tricky one:  you may want to understand your "Working Set" size and ensure that your applications "Working Set" size is set large enough to accomodate all memory pages. Windows uses a 4K page Size or 4096 bytes per page, and working sets by default hold about 300 or so pages of memory.
ALL Virtulization Environments:
  • Disable all Screensavers, animations, or Xservers on Linux or windows.  Any background processing will be counted by VMware's virtulization statistics.  This also means that you should have any Virus Scanners well-tuned such that you are not doing on-access demain scanning for known secure directories.
  • Disconnect all unused devices on both the host and guest operating system.
  • CPU Bound Applications:  for applications that are CPU bound, CPU Virtulization will cause a degradation in performance.  For applications that are not CPU bound CPU Virtulization will have little perfromance degradation.
    • Use the minimum number of Virtual CPUs possible.
    • Note that in some guest operating systems, the Idle loop (nothing on the processor) still counts against that Virtual CPU in the VMware CPU statistics.
    • Multithreaded applications - Pin guest threasd or processes to Virtual CPUs.
    • Monitor the Idle Loop Spin Parameter.
    • Monitor the Service Console Activity with esxtop.
  • Memory Overhead
    • Virtulization Translation Activities.  OS (the guest os) will abstract memory for each process.   When the process needs to access memory, it will request that memory.  Here is the High level of what happens:
      • Request memory -> translate to Relative Page.
      • Guest OS Segments page -> get OS's physical Address.
      • VMWare translates OS's physical -> Get Real Physical Page.
    • Usage Overhead.  
      • Service Console uses < 300M.
      • VMWare kernal  uses a small amount of memory for various data structures.
    • Host memory Sizing - make sure that the host OS has more physical memory than all the ESX overhead plus the maximum of all the working set sizes of all the running applications in all the VMs.
    • Allocate Enough memory to each VM to hold all the workings sets of all the applications that will be run in that VM.
    • If Over Committed Memory is too large, the ESX sever may start swapping.  This will degrade performance significantly.
These are just some of the simple low-haning fruit.
VMWare has published a very comprehensive document:
www.vmware.com/pdf/vi_performance_tuning.pdf