Pages

Wednesday 27 January 2010

Determining if a VM is orphaned

Please check out www.kaztechsolutions.co.uk for more of my technical posts, alternately please call us on 01932 268289. 

Symptoms

  • Virtual machines showing as invalid or orphaned in VirtualCenter.
  • Error when trying to start the virtual machine: Could not power VM, no swap file, failed to power on VM .
  • Virtual machines showing as invalid or orphaned after a VMware High Availability (VMware HA) host failure occurs
  • Virtual machines showing as invalid or orphaned after an ESX Server host comes out of maintenance mode
  • Error when trying to start the virtual machine: VMControl error -11: No such virtual machine .
  • Error when trying to start the virtual machine: A general system error occurred. The system returned on error. Communication with the virtual machine may have been interrupted .
  • Virtual Machines showing as invalid or orphaned after a failed DRS migration.
Purpose

The purpose of this article is to explain what orphaned virtual machines are, how they occur, and how you can fix them. The article outlines the most common error messages seen that relate to orphaned virtual machines and how these issues can be resolved.

Resolution
Note: Before you begin, refer to Restarting the Management agents on an ESX Server (1003490) for important information on restarting the mgmt-vmware service.
Note: The following solutions may require the use of the vmware-cmd command. For more information on the vmware-cmd command, see the VMware Scripting API .
In VirtualCenter, you may find that you have a virtual machine that has an orphan designation or has become invalid. An orphan virtual machine is one that exists in the VirtualCenter database but is no longer present on the ESX Server host. A virtual machine also shows as orphaned if it exists on a different ESX Server host than the ESX Server host expected by VirtualCenter.
A virtual machine can become orphaned:
After a VMotion or VMware DRS Migration (ESX 3.x).

Connect to the source and destination ESX Server hosts using SSH. For more information, see Opening a command or shell prompt (1003892) .

Check with the vmware-cmd -l command if the orphaned virtual machine is registered on the same ESX Server host as reported by VirtualCenter, this is likely the source machine. If the virtual machine is not registered on that host, use the vmware-cmd -l command to check if it is registered on the destination ESX Server host.

If the virtual machine is registered on the destination ESX Server host:

Run the following commands to restart the vpxa and ESX Server Management services:

service mgmt-vmware restart. For more information, see Virtual Machines Unexpectedly Reboot after Issuing the "mgmt-vmware restart" Command (7301769) .
service vmware-vpxa restart.

Restart the VirtualCenter Server Service. For more information, see Stopping, starting or restarting the vCenter server service (1003895).

Note: If you are not using VirtualCenter only the first step is required.
Note: For the majority of problems related to orphaned virtual machines on ESX Server 3.x, these steps resolve the issue.
Note : Ensure there is no time difference between the source and destination ESX Server hosts.

If after completing these steps you see the following errors when attempting to start your virtual machine:

A general system error occurred. The system returned on error. Communication with the virtual machine may have been interrupted.
VMControl error -11: No such virtual machine.

Try to start the virtual machine from the command line using: vmware-cmd start

Where is the path to the configuration file as determined by ‘vmware-cmd –l ’.

Try to register the virtual machine through the vmware-cmd -s command. If it does not fail with the error VMControl error -11: No such virtual machine , go to step F.
You can also try to register a virtual machine by right-clicking on its .vmx file in the datastore browser and choosing Register Guest.
View the .vmx file of the virtual machine and verify that the file has valid configuration parameters. Ensure the file does not contain non UTF-8 characters.
If possible, compare the .vmx file with the .vmx file of another virtual machine.
Create a new virtual machine, choosing to use the existing virtual disks of the original virtual machine.
Power on the virtual machine.

If the virtual machines on a particular ESX Server host are showing as orphaned after a VMware HA host failure occurs or after the ESX Server host comes out of maintenance mode:

Remove the ESX Server host from VirtualCenter.

To remove the ESX Server host from VirtualCenter:

Select the specified ESX Server host in the VirtualCenter inventory.
Right-click on the ESX Server host and choose Disconnect.
Right-click and selecting the Remove option after the ESX Server host has been disconnected.

Remove the VirtualCenter agent and VMware HA agents by running the commands from the service console of the ESX Server host:

export LGTO_AAM_VMWARE_REMOVAL=1
rpm -e LGTOaama
rpm -e LGTOaamvm
rpm -e VMware-vpxa

Re-add the ESX Server host to VirtualCenter.

To re-add the ESX Server to VirtualCenter:

Select the cluster, datacenter, or farm to which you want to add the ESX Server host.
Right-click on the cluster, datacenter, or farm and choose Add.
Enter the ESX Server host's IP address. You must have a username and password with sufficient permissions to add the host. Complete the steps in the Add Host wizard.

The Virtual machines is no longer orphaned.

If you see the following error when trying to start the Virtual Machines:

Could not power VM, no swap file, failed to power on VM when trying to start them

Run the ps -auxwww | grep -i [vmname] command on each ESX Server host until you find the server host that is running the virtual machine's process and locking its files.
Run the service vmware-vpxa restart command on that ESX Server host
Restart the VirtualCenter Server service

If you delete a virtual machine outside of VirtualCenter. A user can delete a virtual machine through the VMware Management Interface while VirtualCenter is down, through the Virtual Infrastructure (VI) Client directly connected to an ESX Server host, or by deleting the virtual machine's configuration file through the service console.

If the configuration file was deleted and the virtual disk remains, you can recreate the virtual machine using the VMware Management Interface or the VI Client and choosing to attach the existing virtual disk to a newly created .vmx file.

If VirtualCenter is restarted while a migration is in progress, a virtual machine may show as orphaned. This is a temporary situation. During startup, VirtualCenter reconnects to all hosts. If a migration completed while VirtualCenter was down, a virtual machine can be reported as an orphan until VirtualCenter establishes a connection to the target host for the virtual machine.
If you schedule too many virtual machines to be relocated at the same time. An ODBC timeout can cause errors in the database. To resolve an ODBC timeout, delete the orphans after ensuring the original virtual machines can power on.

Click Delete VM on the VirtualCenter console to delete the virtual machine and its orphan from the VirtualCenter database.

No comments:

Post a Comment