Symptoms
- Cannot observe the progress when deleting a snapshot.
- It is difficult to determine status of snapshot deletion.
- The hostd process does not respond.
- Storage vMotion operation stays at 18% for a long time.
Purpose
This article provides information on monitoring directories using the watch command and waiting for the snapshot deletion operations to complete in ESX and ESXi.
For more information on how snapshots work, see Understanding virtual machine snapshots in VMware ESXi and ESX (1015180).
Resolution
Note: Time involved to commit snapshots is environmental and subjective.
ESX 3.5/4.x and ESXi 4.1/5.0
To monitor directories during snapshot deletion in ESX 3.5/4.x and ESXi 4.1/5.0:
- Log in as root to the ESX host using SSH. For more information, see Connecting to an ESX host using a SSH client (1019852) or Using Tech Support Mode in ESXi 4.1 and ESXi 5.0 (1017910)
- Navigate to the virtual machine directory containing vmdk virtual disk files.
- List files in the directory by executing:
# ls -al
- Determine any VM_NAME-00000#.vmdk or VM_NAME-00000#-delta.vmdk snapshot files. Look for numbered files following the hyphen(-) in the name.
- To monitor the VMDK snapshot and base disks which are currently being updated use the following watch command:
# watch -d 'ls -luth | grep -E "delta|flat"'
where:
-d highlights the differences between successive updates
t sorts by modification time
l shows a long listing which displays additional information regarding the files themselves.
u sorts by and shows access time
h prints sizes in a readable format like 1K 234M 2G
You can also run the following command to monitor the time stamp update of the base disks to confirm if the process is working or not.
ls -lrt |grep -E "flat|delta"
This command monitors the contents of a directory and displays files by their modification date.
Note: In ESX 3.5 and 4.0 (pre Update 2) the snapshot delta files will be written to the previous snapshot delta file and so on and are finally written to the base disk (flat). In ESX/ESXi 4.0 Update 2 and later the process works differently in that the data in snapshots (deltas) are written directly to the base disk (flat). For more information on the snapshot process, see Understanding virtual machine snapshots in VMware ESXi and ESX (1015180) and Consolidating snapshots (1007849).
If there are more than 10 snapshots, use this command to monitor the snapshot commit process and to prevent the screen from filling with too many files:
# while true;do date;ls -lht *vmdk|head -10;echo ________;sleep 3;done
Note: You can quit the monitoring of the consolidation process with CTRL + C.
ESXi 3.5/4.0
To monitor directories during snapshot deletion in ESXi 3.5/4.0:
- Log in as root to the ESXi host using the Tech Support mode. For more information, see Tech Support Mode for Emergency Support (1003677).
- Change to the virtual machine directory /vmfs/volumes/datastore/VM.
- Open a text editor such as vi or nano and create the file snapmon. For more/related information, see Editing files on an ESX host using vi or nano (1020302)
- Add this script the the file:
snapmon
clear
while [ 1 ]
do
date;ls -luth *.vmdk
sleep 2
clear
done - Save and close the file.
- Run this command to make this file executable:
# chmod ug+x snapmon - Commit the snapshot from the Snapshot Manager.
- Execute the snapmon script to monitor the snapshot:
# ./snapmon
Note: This process can be sent to background and brought to foreground using the fg command.
You can locate most recent pid (/bin/sh) using the ps ax command.
To kill a running snapmon, run the command:
# kill
Based on VMware KB 1007566
