Backup Consistency Types

This post is to summarize the various backup consistency types:

• Inconsistent Backup
  • Any file changed after it was backed up but before the job completed, the result is an inconsistent backup
  • e.g. File A and B, File A is backed up, then File A and B are changed, then File B is backed up. Now the backup of File A and B is inconsistent
  • The content in memory or pending I/O is not backed up
• Crash-Consistent Backup
  • All data is backed up at exactly the same time via techniques like Volume Shadow Copy Service (VSS) to take a block-level snapshot, and then the backup software pulls its backup from that snapshot
  • This backed up data is in the same state it would have been if the system had crashed
  • The content in memory or pending I/O is not backed up
  • Many applications, like Active Directory, have an automated recovery mechanism and will attempt to handle the inconsistent problem without administrator intervention. If these automated systems aren’t successful, a manual process is needed. For Microsoft SQL, you may need to know how to replay logs into a database file.
• Application-Consistent Backup
  • For Windows applications, the application manufacturer provide a VSS writer. When the VSS service is triggered, it will notify these writers that a backup is occuring. Then it’s up to the VSS writer how to handle it.
  • A proper VSS writer will make the application flushing all of its memory and I/O operations to the disk, as it would be if the application were properly closed
  • When the VSS snapshot is complete, it signals the VSS writers, then application resume normal operation and the backup software pulls its backup from that snapshot
  • If an application does not provide or properly register a VSS provider but its data resides on a volume with VSS enabled, the data is backed up in a crash-consistent state
• Image-Level Backup
  • The other backups back up when a machine is actively running
  • An image-level backup backs up when the machine is shut down

Source: VSS Crash-Consistent vs Application-Consistent VSS Backups

vCenter “Hardware monitoring service on this host is not responding or not available” in Hardware Status tab

 

Fix: In vCenter, select the VM host, Configuration, Security Profile (under Software), Firewall Properties, CIM Server, Options, Restart.

After restarting the CIM Server service, it can take a few minutes before the hardware status is updated.

Tested in ESXi 5.0

Reference: http://communities.vmware.com/thread/219556?start=15&tstart=0

Configure USB Devices in Windows Virtual PC

Windows Virtual PC has integration features that allow certain types of devices to be shared between the host computer and the virtual machine (VM), including audio devices, clipboard, printers, drivers, and smart cards.

However, the VM may use the generic driver for the USB device, and the device may not work correctly.  To install the driver for the USB device,

1. Disable Integration Features.
2. Log on again.  Under the USB menu, attach the USB device and click Continue to remove the device from the host computer.
3. Install the USB driver, and test it.
4. Release the USB device from the VM using the USB menu.
5. Enable the integration features again, and the USB device should work because the right driver is loaded in the VM.

Upgrade vSphere Virtual Machine

Recap vSphere upgrade process:

• Upgrade vCenter Server
• Upgrade ESXi Hosts (3.5 to 4.1, 4.0 to 4.1)
• Upgrade Virtual Machine Clients (optional.  But it’s required if you want to take advantage of the new drivers and features)

The following article is from Scott Lowe with some of my update.

Upgrading a VMware Infrastructure 3.x environment to VMware vSphere 4 involves more than just upgrading vCenter Server and upgrading your ESX/ESXi hosts (as if that wasn’t enough). You should also plan on upgrading your virtual machines. VMware vSphere introduces a new hardware version (version 7), and vSphere also introduces a new paravirtualized network driver (VMXNET3) as well as a new paravirtualized SCSI driver (PVSCSI). To take advantage of these new drivers as well as other new features, you’ll need to upgrade your virtual machines. This process I describe below works really well.

Please note that this process will require some downtime. I personally tested this process with both Windows Server 2003 R2 as well as Windows Server 2008; it worked flawlessly with both versions of Windows.

1. Record the current IP configuration of the guest operating system. You’ll end up needing to recreate it.

2. Upgrade VMware Tools in the guest operating system. You can do this by right-clicking on the virtual machine and selecting Guest > Install/Upgrade VMware Tools. When prompted, choose to perform an automatic tools upgrade. When the VMware Tools upgrade is complete, the virtual machine will reboot.

3. After the guest operating system reboots and is back up again, shutdown the guest operating system. You can do this by right-clicking on the virtual machine and selecting Power > Shutdown Guest.

4. Upgrade the virtual machine hardware by right-clicking the virtual machine and selecting Upgrade Virtual Hardware.

5. In the virtual machine properties, add a new network adapter of the type VMXNET3 and attach it to the same port group/dvPort group as the first network adapter.

6. Remove the first/original network adapter.

7. Add a new virtual hard disk to the virtual machine. Be sure to attach it to SCSI node 1:x; this will add a second SCSI adapter to the virtual machine. The size of the virtual hard disk is irrelevant.

8. Change the type of the newly-added second SCSI adapter to VMware Paravirtual.

9. Click OK to commit the changes you’ve made to the virtual machine.

10. Power on the virtual machine. When the guest operating system is fully booted, log in and recreate the network configuration you recorded for the guest back in step 1. Windows may report an error that the network configuration is already used by a different adapter, but proceed anyway. Once you’ve finished, shut down the guest operating system again.

11. Edit the virtual machine to remove the second hard disk you just added.

12. While still in the virtual machine properties, change the type of the original SCSI controller to VMware Paravirtual (NOTE: See update below.)

13. Power on the virtual machine. When the guest operating system is fully booted up, log in.

14. Create a new system environment variable named DEVMGR_SHOW_NONPRESENT_DEVICES and set the value to 1.

15. Launch Device Manager and from the View menu select Show Hidden Devices.

16. Remove the drivers for the old network adapter and old SCSI adapter. Close Device Manager and you’re done!

If you perform these steps on a template, then you can be assured that all future virtual machines cloned from this template also have the latest paravirtualized drivers installed for maximum performance.

UPDATE: Per this VMware KB article, VMware doesn’t support using the PVSCSI adapter for boot devices. That is not to say that it doesn’t work (it does work), but that it is not supported. Thanks to Eddy for pointing that out in the comments!  The VMware KB article has been updated, VMware support using the PVSCSI adapter for boot devices on Windows Server 2008 and Server 2003.

Reference: Scott Lowe – vSphere Virtual Machine Upgrade Process

Install HP ESXi Offline Bundle for VMware ESXi 4.1

Prerequisites

• vSphere Command-Line Interface (vSphere CLI) on a Microsoft Windows computer.
• HP ESXi bundle zip file (e.g. hp-esxi4.1uX-bundle-1.0a.zip)

Procedure

1. Power off any VMs that are running on the host.

2. Place the host into maintenance mode.

3. Find out which bulletins are applicable to the ESXi host.

vihostupdate.pl --server <server> --scan --bundle <local_path>/hp-esxi4.1uX-bundle-1.0a.zip

4. (Optional) List all the bulletins that are available in the bundle.

vihostupdate.pl --server <server> --list --bundle <local_path>/hp-esxi4.1uX-bundle-1.0a.zip

5. Install bulletins from the bundle on the ESXi host.

vihostupdate.pl --server <server> --install --bundle <local_path>/hp-esxi4.1uX-bundle-1.0a.zip --bulletin bulletin1,bulletin2

if you omit the --bulletin argument, this command installs all the bulletins in the bundle.

6. Verify that the bulletins are installed on your ESXi host.

vihostupdate.pl --server <server> --query

Reference: see HP Installation Instructions for more details

Upgrade VMware ESXi 3.5 to vSphere 4.1 Using Host Update Utility

Upgrade ESXi 3.5 to 4.1 summary:

• upgrade ESXi 3.5 to 4.0 Update 1
• upgrade ESXi 4.0 Update 1 to 4.1
• I cannot upgrade EXSi 3.5 to 4.1 directly using Host Update Utility

Upgrade procedure:

1. Download “VMware vSphere Client and Host Update Utility 4.0 Update 1”.  (VMware vSphere Client 4.1 download package does not include Host Update Utility)
2. Install VMware vSphere Client and Host Update Utility 4.0 Update 1.
3. Download “Upgrade package from ESXi Server 3.5 to ESXi Server 4.0 Update 1” ZIP package.
4. Launch Host Update Utility 4.0 to upgrade ESXi Server 3.5 to 4.0 Update 1 with the ZIP package.  This will upgrade ESXi to 4.0.0 build-208167.
5. Then follow the instruction on “Upgrade vSphere ESXi from 4.0 to 4.1 Using vihostupdate”

Upgrade vSphere EXSi from 4.0 to 4.1 Using vihostupdate

1. Run vSphere Host Update Utility 4.0 to install all 4.0 patches.
2. If the host is managed by vCenter Server, you must Upgrade vCenter Server to v.4.1 first.
3. Install vSphere CLI on a computer.
4. Download the upgrade ZIP bundle to the vSphere CLI computer.  Make sure download the ESXi (ESX Installable) ZIP bundle.
5. Power off any virtual machines that are running on the host and place the host into maintenance mode.
6. (Optional) Configure a scratch directory (see KB1004177).  Create a directory (scratch_dir) on the VMFS volume.
7. (Optional) Set the scratch directory to /vmfs/volumes/datastore1/scratch_dir
8. (Optional) Select  the “ScratchConfig.ConfiguredSwapState” option.
9. (Optional) Reboot the ESXi host.
10. On vSphere CLI computer, vihostupdate.pl --server <host-ip-addr> -i -b "upgrade-from-ESXi4.0-to-4.1.0-0.0.260247-release.zip" -B ESXi410-GA-esxupdate
11. vihostupdate.pl --server <host-ip-addr> -i -b "upgrade-from-ESXi4.0-to-4.1.0-0.0.260247-release.zip" -B ESXi410-GA
12. Reboot the ESXi host.

vCenter Server 4.1 Installation

1. Install Windows Server 2008 R2 (64-bit)
2. Create a vCenter SQL database and a vCenter database user; set the db owner to vCenter database user; grant DBO right on the MSDB database.
3. Install Microsoft SQL Server Native Client 2005 or 2008.
4. Create a SQL DSN (make sure select SQL Server Native Client driver.
5. Install vCenter Server 4.1

Linux Guest Different MAC Address Error on VMware vSphere

Converted a Linux (Fedora 5) PC to a VMware vSphere guest.  The Linux guest OS shows a failed message when shutting down interface eth0.

Fix: edit /etc/sysconfig/network-scripts/ifcfg-eth0’s HWADDR to match the MAC address assigned the Linux guest OS.

Linux Guest Hangs at “Starting udev” on VMware vSphere

Converted a Linux (Fedora 5) PC to a VMware vSphere guest.  The Linux guest OS hangs at “Starting udev”.

Fix:

1. Restart the Linux guest OS;
2. Press any key at the GRUB boot menu, press e to edit, and add the highlighted words at the “kernel” line; press enter, and then b to boot;
   kernel /vmlinuz-2.6.18-1.2257.fc5smp ro root=/dev/sdb1 clock=pmtmr divider=10 hgb quiet
3. Once it boots in the console, edit /boot/grub/grub.conf with the same setting.

Reference: http://itsecureadmin.com/2010/03/linux-guest-hangs-at-starting-udev-vmware-vsphere/
or http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1006427

Windows Server 2008 R2 Hyper-V Server License

Q. Which edition of Windows Server 2008 R2 should I buy for my Hyper-V server?
John Savill

A. The decision about whether to purchase the Standard, Enterprise, or Datacenter edition of Windows Server 2008 R2 should depend on two major factors: the number of virtual machines (VMs) you intend to run and your high availability requirements.

High availability is only available with the Enterprise and Datacenter SKUs, so if you want clusters and features like Live Migration, you have to use Enterprise or Datacenter edition.

The next factor is the number of VMs. Standard Edition supports one physical OS and one virtual OS (VM), Enterprise supports one physical OS and four virtual OSs, and Datacenter supports one physical OS and an unlimited number of virtual OSs. Note that Standard and Enterprise are purchased on a per-server basis while Datacenter is purchased on a per-processor basis, and at least two processors (sockets) must be licensed on each server with Datacenter.

You can assign multiple licenses to a single physical server. For example, I could purchase two copies of Enterprise Edition and assign them to a single physical server, which would allow me to run eight VMs. I could also buy eight copies of Standard Edition or just two of Datacenter (I need two because two is the minimum number purchasable with Datacenter—two sockets).

• Generally, the following is a good guideline for the most cost effective SKU to buy, but remember to consider future growth.
• Standard Edition is most cost efficient for one to three VMs per server. Note that if you run three VMs, you'll need to buy three copies of Standard Edition.
• Enterprise Edition is most cost efficient from four VMs on a server up to four VMs per processor. For example, if I have a dual processor box and want to run eight VMs, I could buy two copies of Enterprise edition.
• Datacenter Edition is most cost efficient for more than four VMs per processor, because you can run an unlimited number of VMs per processor and license each processor. While Datacenter is more expensive than Enterprise when running four VMs per processor, you have more scalability and support for future growth, so you could, potentially, adopt Datacenter over Enterprise when you consider future requirements. Remember that you have to license all processors in the server.

Microsoft offers an online calculator that can help you choose the right SKU to buy.

Remember that if you're using the maximum number of virtual instance rights, you can't run any workloads other than Hyper-V in the parent partition. If you run additional workloads in the parent partition (which isn't recommended), you lose one of your virtual instance rights. So if you're running Enterprise Edition and you're also running a file and print server on the Hyper-V host, you can only run three virtual OS instances.

When you purchase a SKU of Windows Server, you have the right to run that version and any lower version in your virtual environments. For example, if I buy Datacenter Edition, I can run Datacenter, Enterprise, or Standard in my VMs. If I buy Enterprise edition, I can run Enterprise or Standard in my VMs.

Finally, remember you can't move these virtual OS environment rights between servers. If you want to run four VMs on each server and have the ability to live migrate them to another server (which would mean it would run eight VMs), you need two licenses of Enterprise on the target server (or Datacenter). This is why when you're using Live Migration and clusters, it's normally advised to purchase Datacenter.

Q. If I'm performing a physical to virtual migration, can I move my physical Windows license to the virtual environment?

A. Assuming you're turning off the physical box and wiping the OS, you can move a Windows license from a physical to a virtual environment based on Microsoft's 90 day move rights. These rights allow licenses to be moved once every 90 days between servers, including moving to a virtualization host.

VMware vSphere 4 Notes

1. vSphere Editions for Small Businesses Comparison

Basic ESXi physical hardware limit: 2 physical processors, 6 cores per processors, 256GB memory.

 

2. VMware vCenter Server

VMware vCenter Server Installation

• Hardware requirements (for up to 50 ESX(i) hosts and 250 VMs):
• 2 CPUs
• 4 GB RAM
• 3 GB Disk
• Software requirements:
• Windows Server 2003 or 2008 x32 or x64
• SQL Server 2005 or 2008 or Express Edition
• Can be installed in a physical or virtual machine.
• Reference:
• What Is New in VMware vCenter Server 4
• Upgrading to ESX 4.0 and vCenter 4.0 best practices

VMware vCenter Server Administration

• After installing vCenter Server:
• Create a datacenter
• Join ESX(i) host to vCenter Server

Allocate Virtual Processors

The recommendation is to allocate more virtual processors than you have physical processors and let the hypervisor take care of scheduling the CPU work across the physical cores. Microsoft typically recommends you have no more than eight virtual processors per physical core.