Summary

Zerto Virtual Replication keeps track of relevant vSphere components (host, VMs, networks, and etc). In the event that an object is removed from vCenter, Zerto is unable to continue to track the host, even if the object is later re-added.

In the case of a VRA, this means that replication will stop for any VPGs that are using the affected VRA, and the VRA on that host will become a "Ghost VRA." Once a Ghost VRA exists, the situation that caused it to become a Ghost VRA either needs to be resolved, or the VRA needs to be replaced in order for the replication to resume.

Symptoms

When a VRA will enter ghost status, it will be clearly marked in the ZVM UI. Unter the setup tab, the VRA will be marked with red, and the VRA status will show "Ghost VRA". Also, the relevant VPGs will be in an error state.

Here is an example of how it will look like in the ZVM UI.
 

Root Cause

There are several ways for Ghost VRA status to appear:

1. A VRA VM, or the VRA-H (the diskbox) associated with the VRA, has been removed from the VC (either deleted or just removed from inventory).
2. A recovery (mirror) or a journal volume that was attached to the VRA VM, or to the VRA-H, has been removed from the VC (either detached from the VM or completely deleted).
3. The host on which the VRA is running on was re-added to the vCenter inventory (it will get a new internal ID from the VC MOB, thus Zerto will not be able to track the VRA installed on it).
4. Deleted Failover Test VM (testing recovery VM).

Solution

There are several solutions, depending on where the Ghost VRA is reported, or how it was generated.

A Ghost VRA at a source site:

1. vMotion protected VMs from the Ghost VRA’s host to another host, with a VRA installed, in the source cluster.
2. Take note of the ghost VRA’s network/storage configurations for use during re-install.
3. When the VRA is not protecting any VPGs, uninstall the ghost VRA.
4. Install a new VRA on the host.
5. vMotion protected VMs back to the host as desired.
6. Check that the VRA and the VPGs are working as expected

A Ghost VRA due to the removal of a Zerto-managed VM (for example - VRA, VRA-H, or a failover test VM) from vCenter inventory:

1. Take note of the settings of the affected VPGs, as you'll need to delete them. You can export the VPGs details from the recovery ZVM UI. To do so, log into the ZVM UI, click on the VPGs tab, then on the down arrow in the right top corner (this will create a CSV file with the VPGs details).
2. Delete the affected VPGs (you can choose “Keep target disks at peer site" if you'd like to use the pressed option), so the ghost VRA will have 0 VPGs to protect. If the deletion fails, and the VPGs enter “Pending Remove” state, choose to delete them again, and choose the “Force Delete” option.
3. The VPG deletion should cause the VRA to exit the ghost VRA state. 
4. Using the Zerto Diagnostics utility on the production ZVM, choose to import a VPG, then select the configuration XML file to recreate the removed VPGs. By default, the ZVM will export the settings to a file once a day and it will be saved in the Zerto installation folder, in the ExportedSettings folder (For example C:\Program Files (x86)\Zerto\Zerto Virtual Replication\ExportedSettings). 
5. If the recreation of the VPGs fails, create the VPGs manually with the information you've saved during step 1 (you can use the pressed option if needed).
6. Check that the VRA and the VPGs are working as expected

A Ghost VRA at a target site due to the removal of a VRA’s host from vCenter inventory:

1. Take note of the settings of the affected VPGs and VRA, as you'll need to delete them. You can export the VPGs details from the recovery ZVM UI. To do so, log into the ZVM UI, click on the VPGs tab, then on the down arrow in the right top corner (this will create a CSV file with the VPGs details).
2. Delete the affected VPGs (you can choose “Keep target disks at peer site" if you'd like to use the pressed option). If the deletion fails, and the VPGs enter “Pending Remove” state, choose to delete it again, and choose the “Force Delete” option.
3. Remove the ghost VRA VM from the vCenter inventory. This will dissociate disks that are still attached to the VRA, and leave them in their datastore locations.
4. Install a new VRA on the host.
5. Using the Zerto Diagnostics utility on the production ZVM, choose to import a VPG, then select the configuration XML file to recreate the removed VPGs. By default, the ZVM will export the settings to a file once a day and it will be saved in the Zerto installation folder, in the ExportedSettings folder (For example C:\Program Files (x86)\Zerto\Zerto Virtual Replication\ExportedSettings).
6. If the recreation of the VPGs fails, create the VPGs manually with the information you've saved during step 1 (you can use the pressed option if needed).
7. Check that the VRA and the VPGs are working as expected.

A Ghost VRA due to the removal of a recovery/journal volume from vCenter inventory:

1. Take note of the settings of the affected VPGs (the VPGs that their recovery volumes were removed), as you'll need to delete them. You can export the VPGs details from the recovery ZVM UI. To do so, log into the ZVM UI, click on the VPGs tab, then on the down arrow in the right top corner (this will create a CSV file with the VPGs details).
2. Delete the affected VPGs (you can choose “Keep target disks at peer site" if you'd like to use the pressed option), so the ghost VRA will have 0 VPGs to protect. If the deletion fails, and the VPGs enter “Pending Remove” state, choose to delete them again, and choose the “Force Delete” option.
3. The VPG deletion should cause the VRA to exit the ghost VRA state.
4. Using the Zerto Diagnostics utility on the production ZVM, choose to import a VPG, then select the configuration XML file to recreate the removed VPGs. By default, the ZVM will export the settings to a file once a day and it will be saved in the Zerto installation folder, in the ExportedSettings folder (For example C:\Program Files (x86)\Zerto\Zerto Virtual Replication\ExportedSettings).
5. If the recreation of the VPGs fails, create the VPGs manually with the information you've saved during step 1 (you can use the pressed option if needed).
6. Check that the VRA and the VPGs are working as expected.

A Ghost VRA with no environment changes:

1. Take note of the settings of the affected VPGs, as you'll need to delete them. You can export the VPGs details from the recovery ZVM UI. To do so, log into the ZVM UI, click on the VPGs tab, then on the down arrow in the right top corner (this will create a CSV file with the VPGs details).
2. Delete the affected VPGs (you can choose “Keep target disks at peer site" if you'd like to use the pressed option), so the ghost VRA will have 0 VPGs to protect. If the deletion fails, and the VPGs enter “Pending Remove” state, choose to delete them again, and choose the “Force Delete” option.

 If Step 2 did not resolve the issue, continue with the following steps.

1. Note the Ghost VRA’s network/storage configurations for use during re-install.
2. Remove the Ghost VRA VM from the vCenter inventory. This will dissociate disks that are still attached to the VRA, and leave them in their datastore locations.
3. Install a new VRA on the host.
4. Using the Zerto Diagnostics utility on the production ZVM, choose to import a VPG, then select the configuration XML file to recreate the removed VPGs. By default, the ZVM will export the settings to a file once a day and it will be saved in the Zerto installation folder, in the ExportedSettings folder (For example C:\Program Files (x86)\Zerto\Zerto Virtual Replication\ExportedSettings).
5. If the recreation of the VPGs fails, create the VPGs manually with the information you've saved during step 1 (you can use the pressed option if needed).
6. Check that the VRA and the VPGs are working as expected.