Backup Consistency Types

This post is to summarize the various backup consistency types:

• Inconsistent Backup
  • Any file changed after it was backed up but before the job completed, the result is an inconsistent backup
  • e.g. File A and B, File A is backed up, then File A and B are changed, then File B is backed up. Now the backup of File A and B is inconsistent
  • The content in memory or pending I/O is not backed up
• Crash-Consistent Backup
  • All data is backed up at exactly the same time via techniques like Volume Shadow Copy Service (VSS) to take a block-level snapshot, and then the backup software pulls its backup from that snapshot
  • This backed up data is in the same state it would have been if the system had crashed
  • The content in memory or pending I/O is not backed up
  • Many applications, like Active Directory, have an automated recovery mechanism and will attempt to handle the inconsistent problem without administrator intervention. If these automated systems aren’t successful, a manual process is needed. For Microsoft SQL, you may need to know how to replay logs into a database file.
• Application-Consistent Backup
  • For Windows applications, the application manufacturer provide a VSS writer. When the VSS service is triggered, it will notify these writers that a backup is occuring. Then it’s up to the VSS writer how to handle it.
  • A proper VSS writer will make the application flushing all of its memory and I/O operations to the disk, as it would be if the application were properly closed
  • When the VSS snapshot is complete, it signals the VSS writers, then application resume normal operation and the backup software pulls its backup from that snapshot
  • If an application does not provide or properly register a VSS provider but its data resides on a volume with VSS enabled, the data is backed up in a crash-consistent state
• Image-Level Backup
  • The other backups back up when a machine is actively running
  • An image-level backup backs up when the machine is shut down

Source: VSS Crash-Consistent vs Application-Consistent VSS Backups

vMotion Microsoft Cluster VMs

vSphere supports three different configurations of Microsoft Cluster Service (MSCS):

• Clustering MSCS VMs on a single host (aka a cluster in a box - CIB)
• Clustering MSCS VMs across physical hosts (aka a cluster across boxes - CAB)
• Clustering physical machines with VM

see Setup for Failover Clustering and Microsoft Cluster Service, ESXi 6.0 for more information.

However, vMotion is supported only for CAB with pass-through RDMs. Do not vMotion MSCS VMs in the other two configuration.

In addition, do not vMotion MSCS VMs to a VSAN storage, because VSAN does not support thick provision and SCSI bus sharing on the VM SCSI adapter. The VM will not be able to power on with the error message “Cannot use non-thick disks with clustering enabled (sharedBus='physical'). The disk for scsi1:0 is of the type thin.”

Add Fonts to Windows Command Prompt

By default, Windows 10 Command Prompt only offers two font choices - Consolas and Lucida Console, even other fonts are installed. To add more a font to selection, add a String Value registry key in

HKLM\Software\Microsoft\WindowsNT\CurrentVersion\Console\TrueTypeFont

In the following screenshot, I added Bitstream Vera Sans Mono to my command prompt.

Source: http://www.watchingthenet.com/how-to-add-and-change-fonts-in-windows-command-prompt.html

Multiple vCPUs Can Cause Performance Issues

Assigning more hardware resource (aka overprovisioning) on a VM can cause performance issues.

According to KB1005362, we can use esxtop to check the %CSTP value to determine vCPUs overprovisioning. If the value of a VM is higher than 3.00, the performance issue may be caused by the vCPU count. Try lowering the vCPU count of the VM by 1.

Understand Three Important Files in vSphere HA

This post is the note I take as I am reading the vSphere 6.x HA Deepdive book, plus my understanding of the materials.

Here we focuse on the files stored on the shared datastore, aka. remote files. Each host also restores some configuration files in a directly accessible datastore, aka. local files)

• Protectedlist file
  • Naming: protectedlist
  • Owner: The master locks this protectedlist file
  • The master uses this file to claim the “ownership” of the datastores stored the VM configuration file. When a host is isolated, if the host can access the datastore, then it will validate whether a master owns the datastores. If no master owns the datastores, the isolation response will not be triggered and restarts will not be initiated. (see page 29 & 30 of the book about isolation response).
  • The master uses this protectedlist file to track the VMs protected by HA and the states of the VMs (powered on / off)
  • The master distributes this protectedlist file to all datastores in use by the VMs in the cluster
• "poweron" file
  • Naming: host-<number>-poweron
  • Owner: per-host (master & slaves)
  • The host uses this "poweron" file to track the powered on virtual machines on a host
  • The slaves use this "poweron" file to inform the master that it is isolated from the management network
    • No datastore heartbeat: the master determines a host has failed
    • The top line of "poweron" file is 1 (means isolated); if 0 means not-isolated
• Heartbeat file
  • Naming: host-<number>-hb
  • Owner: per-host
  • Each host creates a heartbeat file on the designed heartbeating datastores
    • On VMFS datastore, "heartbeat region" is used to check the heartbeat update
    • On NFS database, the time-stamp of the file is check (each host writes to its heartbeat file once every 5 seconds)

Disable Mac Startup Sound

A way to disable the Mac startup sound is to mute the speaker before shutting down the computer. But you have to remember to do that every time you unmute the speaker.

When I search on the web, some third party utilities are mentioned, e.g. StartupSound.prefPane or Psst or StartNinja or use sudo nvram SystemAudioVolume=%80. All these utilities are for the older version of Mac; and the nvram command doesn’t work my OS X El Capitan (version 10.11.3).

Finally, I find a way to mute the speaker automatically when log off. See this post for the full detail. I only set up the mute.sh, and don’t even bother the unmute.sh.

1. create a mute.sh script with the following lines
• #!/bin/bash
• osascript -e ‘set volume with output muted’
2. sudo chmod u+x ~/Documents/mute.sh
3. sudo mv ~/Document/mute.sh /Library/Scripts/
4. sudo defaults write com.apple.loginwindow LogoutHook /Library/Scripts/mute.sh

To restore the startup sound

• sudo defaults delete com.apple.loginwindow LogoutHook

VMware vFlash Read Cache (VFRC) Notes

• Available in vSphere 5.5 and 6.0 for basic caching solution
• VMware introduces vSphere APIs for IO Filtering (VAIO) in vSphere 6.0 U1 as a new API framework for third party’s innovation
• VAIO is not a feature or product.
• It seems that VMware has no plan to create new caching product based on VAIO
• VAIO can be used not only caching but also for replication
• Some third party product can provide more advanced caching solution (both read and write-back cache, e.g. PernixData FVP)
• Require Enterprise Plus License
• Use local SSDs (must be on the HCL) to form a new file system called VFFS (Virtual Flash File System) to provide two types of write through (read only) caching
• Per-VMDK cache
• Host Swap cache
• VFRC limitation (KB2057206)
• Default VRFC maximum size is 200GB. Can be set to 400GB
• Configuration
1. Set up VFRC resouce on a host
• vSphere Web Client, host, Manage, Settings, Virtual Flash Read Cache Resource Management, Add capacity
2. Allocate VFRC to a VM’s VMDK
• vSphere Web Client, VM,  Edit Settings, select and expand the hard disk,and enter the amount of VFRC
• The block size can impact performance, but it’s not easy to determine (see this post for detail)
3. Set up Host Swap Cache
• vSphere Web Client, host, Manage, Settings, under Virtual Flash, edit Virtual Flash Host Swap Cache Configuration, Enable virtual flash host swap cache and enter the amount

Use Dig to Troubleshoot DNS Issue on Windows

Get dig.exe on Windows

• Download and extract the BIND binary zip file to a folder from ISC.
• Open a command prompt and navigate to the folder
• Enter dig, no installation is required.

Basic syntax

• dig @dns-server name type
• dig @dns-server -q name -t type

Simplify the output

• dig +nocmd +noall +answer @dns-server name
• dig +nocmd +noall +answer +multiline @dns-server name any

Windows Page File Purposes and Sizing

Consider the following for page file sizing

• System crash dump: the page file must exist and be larger than the physical memory (RAM)
• Peak system committed memory:
  • system committed memory = physical memory + all page files combined size
  • 
  • The system with 8GB RAM + 1.3GB page file = 9.2GB committed RAM
• Infrequently accessed pages: Infrequently accessed modified pages are removed from physical memory and stored in the page files. The “\Memory\Modified Page List Bytes” performance counter measures, in part, the number of infrequently accessed modified pages that are destined for the hard disk.

Extend or add a page file if all the following performance counters are true:

• \Memory\Available MBytes is low (more available phyiscal memory is in use): less than 4 MB 
• \Memory\Modified Page List Bytes is high 
• \Paging Files(_Total)\%Usage is high: more than 70%

General recommendation

• Windows Server DC and DFS replication, certificate, and LDS servers are not supported without a configured page file
• Exchange Server requires a page file
• Set a static page file size rather than dymantically managed by Windows
• For Windows Server 2012 or R2 Hyper-V, the page file of the management OS (the host OS) should be left at the default of setting of “System Managed”
• For 64-bit SQL Server, enable “Lock Page in Memory” for the SQL Server service account
• For 64-bit SQL Server, set Page File to be a static 6 GB

Source:

• How to determine the appropriate page file size for 64-bit versions of Windows
• The Windows Page File and SQL Server (performance counter number)
• Windows Page File Explained (example of page file size)