Friday, August 29, 2014

Snapshot consolidation failed with error


Virtual machine was showing following alert:
Virtual machine disk consolidation is needed.  Virtual machine consolidation needed status

After trying to consolidate the virtual machine snapshots I received the following error:
Status: An error occurred while consolidating disks: Could not open/create change tracking file
Cannot complete the operation because the file or folder ds:///vmfs/volumes/*.vmdk already exists

Troubleshooting:

SSH to ESXi host and browse the volume where the VM resides on.
cd into the VM folder and found flat files for each of the vmdk disks but the VM does not show that any snapshots exists.


Tried to vMotion the VM and received same error
Tried to storage vMotion the VM and received same error
Tried creating a new snapshot and delete it again but still could not consolidate afterwards

Found the following KB 2013520 which outlines the same problems but just for committing a snapshot that already exists.

The issue seems to exist due to corrupted CTK files.  These files are associated with each delta disk and flat file for the virtual machine.

Resolution:

  • Power off the VM
  • Create a temp folder in the VM folder on the datastore
  • Move all the CTK files into the temporary folder.  The files names will look like this "*-ctk.vmdk"
  • Right click the VM and select snapshot -> Consolidate

On a side note:

After doing some further investigation I found that the there was a very large snapshots on this VM (> 150Gb) and after committing the snapshot it seems to have corrupted the CTK files.  

CTK file description
Always nice learning something new each day :)

The CTK files describe the VMDK characteristics/attributes of which the actual 'data' is stored in the in the *-flat.vmdk file.
Like in our case if you have multiple disks attached to a VM, then you'll have multiple descriptor files that link to their actual disks.  You can actually view the contents of the file  which provides information such as chain IDs, the type of VMDK, it's data disk, hw version, etc.
These files are also used for change block tracking (CBT) and VMware has a good KB to read on this:

Links: