VxRail: Migration of VMs from host fails with “The target host does not support the virtual machine’s current hardware requirements”

Article Number: 524619 Article Version: 3 Article Type: Break Fix

VxRail Appliance Family

When attempting to migrate VMs from a certain host in the cluster, the following compatibility error is encountered in the migration wizard:

The target host does not support the virtual machine’s current hardware requirements.

To resolve CPU incompatibilities, use a cluster with Enhanced vMotion Compatibility (EVC) enabled. See KB article 1003212.





The above instructions are related to the new Intel Spectre/Meltdown Hypervisor-assisted Guest Mitigation fixes.

CPU features can be compared on the affected host and the other hosts via the following:

> Host-01 (affected):$ cat /etc/vmware/config | wc -l57> Host-02:$ cat /etc/vmware/config | wc -l53

Hence, when VMs start on the affected hosts, they have extra CPU requirement that wont be met when migrating to other hosts.

In order to remove these CPU requirements, you will need to refresh the EVC baseline by disabling EVC and re-enable it again. This will update the /etc/vmware/config on all hosts in the cluster.


RecoverPoint for Virtual Machines: SC production RPAs are in reboot regulation

Article Number: 502032 Article Version: 3 Article Type: Break Fix

RecoverPoint for Virtual Machines

Issue (impact on customer): SC production RPAs are in reboot regulation

Impacted Configuration & Settings [Replication Mode,Splitter Type,Compression,CDP/CLR/Multi,FC/IP/iSCSI, Policies etc.]: RP4VM, failed to register new VM to the VC too many times.

Impact on RP: Control keeps on crashing

Affected versions: 5.0.1

Root cause:

When we register a new VM (upon performing protect) in the VC, to create a shadow VM. In this case, the registration fails and after 5 retry attempts, RPA control give-up and generate a message of the failure with the VM name. Problem is, we deleted the memory allocated for the registration before generating the message, and thus, getting into Assertion (trying to access variable in the memory that no longer exists) and control crashes repeatedly until RPA enters reboot regulation.

Customer deleted a production that protected by RecoverPoint for Virtual Machine cluster without un-protecting it first.


Any other changes that may cause failure for RPA to register VMs.

Fixed at: 5.1


Perform step-by-step procedure:

1. Find out why the VM keep failing to register to the VC and try to solve the error at the VC to prevent this from re-occurring.

2. Unprotect and Re-protect the VMs on the same CG.

3. If 2+3 doesn’t succeed – disable/enable the CG.


Avamar Client for VMware: How to enable Instant Access (IA) restore for multiple VMs with DataDomain 6.0 and Avamar version 7.4

Article Number: 499593 Article Version: 3 Article Type: How To

Avamar Client for VMware

Instant Access (IA) is similar to restoring an image backup to a new virtual machine, except that the restored, virtual machine can be booted directly from the Data Domain system. This reduces the amount of time required to restore an entire virtual machine.

When used with Data Domain systems earlier than release 6.0, in order to minimize operational impact to the Data Domain system, only one instant access is permitted at a time. For Data Domain systems at release 6.0 or greater, 32 instant access processes are permitted at the same time. This requirement is documented inAvamar 7.4 and Service Packs for VMware User Guide

Here is the procedure to enable Multiple IAs restore in Avamar server:

1. Edit /usr/local/avamar/var/mc/server_data/prefs/mcserver.xml in Avamar server, and change value to ‘true’

<entry key=”ddr_can_modify_ir_limit” value=”true” />

2. Restart MCS service to make the change take effect.

dpnctl stop mcs

dpnctl start mcs

dpnctl start sched

3. In Avamar GUI – Server – Modify DD IA limit to 32, and save the change

User-added image

Other Best practices for Instant Access restore

  • It is recommended to change the NFS.MaxVolumes values on the ESXi server to a value greater than 32.

This is because each IA creates an NFS volume on the server. Within vCenter, the setting is under ‘Configuration’->‘Advanced Settings’ of a ESXi server. By default, the value is set to 8.

  • Perform a post-restore migration and clean-up after a system restore.

This is because Avamar resources need to be released for subsequent IA operations.

This also protects the virtual machines’ data. vMotion can be used to migrate a virtual machine from one datastore to another.


RecoverPoint for Virtual Machines: Cannot Enable Image Access Due to a a Virtual Network Device Not Being Visible to an ESXi Host

Article Number: 494877 Article Version: 3 Article Type: Break Fix

RecoverPoint for Virtual Machines 5.0,RecoverPoint for Virtual Machines 5.0 P1

A user attempting to perform an Image Access test is unable to do so. An error is returned stating that one of the configured networks is not visible on the ESXi host.

While validating that the Virtual Machine networks shown in the the settings are correct, this system check ends up failing. Image Access is then prevented since restoring the existing settings may not be possible in this situation.

The following will be present in the vRPA management log:

MgmtAction: setGroupSettings_internal() returning: : (mgmtRS=VectorSet((mgmtRC=MRC_RPVE_IMAGE_ACCESS_NIC_NOT_VISIBLE,additionalInfo=NoOption)),bGood=0)

There is a caching mechanism within the RecoverPoint management process that causes the Virtual Machine to appear as if it’s under the wrong ESXi host.


Configure all ESXi hosts to see the same virtual network, and configure the RecoverPoint for Virtual Machine’s shadow VMs to use only specific network.


RecoverPoint for Virtual Machines: Failover is unsuccessful when shadow VM is storage vMotioned

Article Number: 488925 Article Version: 4 Article Type: Break Fix

RecoverPoint for Virtual Machines 4.3,RecoverPoint for Virtual Machines 4.3 P1,RecoverPoint for Virtual Machines 4.3 SP1,RecoverPoint for Virtual Machines 4.3 SP1 P1,RecoverPoint for Virtual Machines 4.3 SP1 P2


There was a storage vMotion done on a copy VM to a different DataStore.

In order to do so, both shadow VM and replica VM must be migrated.

The first migration is successfully performed but the second one is blocked by VMware.

Symptoms found in the logs:

The second migration is blocked due to an error:

“Relocate virtual machine <vm name> Cannot complete the operation because the file or folder ds:///vmfs/volumes/<first vmUid>/<first vm name>/vmware.log already exists”

Root cause:

Storage vMotion is performed by VMware, the second migration fails due to a conflict of an already existing file.

Affected versions:

4.3, 4.3.1,,, 5.0


In RP4VMs, migrating protected VMs (with shadow based RP4VM solutions) to a different DataStore.


In order to Storage vMotion the shadow and replica VMs, the following steps would need to be performed:

1. Storage vMotion the shadow VM – only the configuration file (VMX) should be migrated, the replica VMDK(s) should be kept on the current DataStore.

By default, Configuration file and all VMDKs are migrated as part of the storage vMotion process so it is vital to click on “Advanced” on the “Select DataStore” step of the “Migrate Virtual Machine” wizard and select a different DataStore specifically for the Configuration file while keeping the VMDKs on the current DataStore.

2. Perform Test a Copy (enable image access) to bring up the replica VM

3. Storage vMotion the replica VM – (Configuration file and all VMDKs) to the new DataStore

If in step 1 a full migration was performed (i.e. to both VMX configuration file and the VMDKs), the second migrate of the replica VM will be denied. In order to solve this, perform a full migration of the shadow VM back to the initial DataStore and follow the storage vMotion steps exactly as described in the procedure above.

Finally, the floppy imageiso file used by the shadow VM would need to be changed as well:

1. Check if the target DataStore has a “RecoverPoint” folder with a RecoverPointVM.flpRecoverPointVM.iso file.

If it does not, copy the RecoverPoint folder from the current DataStore

2. Edit the settings of the shadow VM and point the floppy image to the RecoverPointVM.flp file on the target DataStore.

3. Reset the shadow VM

Remark: RecoverPointVM.iso is relevant to version 4.3.1 only. RecoverPointVM.flp is relevant to and higher versions.

ResolutionFixed at:

Storage vMotion procedure is described above.


RecoverPoint for Virtual Machines: Virtual Machine CPU / RAM resources insufficient on failover

Article Number: 486282 Article Version: 3 Article Type: Break Fix

RecoverPoint for Virtual Machines 4.3,RecoverPoint for Virtual Machines 4.3 P1,RecoverPoint for Virtual Machines 4.3 SP1,RecoverPoint for Virtual Machines 4.3 SP1 P1,RecoverPoint for Virtual Machines 4.3 SP1 P2


When a customer is failing over a Virtual Machine (VM) from production VM to the copy or shadow VM (ex. during disable image access, failover, etc.), the shadow VM is loaded with the reservation for the CPU / RAM (Memory) of the production or copy VM and may prevent the shadow VM from powering on.

Also, when switching back from the shadow VM to the production or copy VM (during image access, failover, etc.), the production or copy VM is loaded with the memory reservation of the shadow VM (default memory reservation of the shadow VM is 64 MB).

Symptoms found in logs:

If the shadow VM can’t power on because of the CPU reservation (as seen in BZ 131448), the following warning will be shown in the Virtual Center:

“insufficient capacity on each physical CPU”

If the shadow VM can’t power on due to the RAM/memory reservation (as seen in BZ 130370), search the VM configuration in the connectors view log in RP (files/home/kos/connectors/logs/vi_connector_view.log):

isShadowVM=truevirtualMemoryReservationInMB=(different from 64)


Due to a bug in the VMS API with VMware, the shadow VM is loaded with the wrong reservations.

Affected versions:

4.3,, 4.3.1

Customer uses RP4VMs to failover a VM or put a CG into image access.


Manually change the shadow/copy/production VM’s RAM / memory and CPU resources.



Monitoring VMware Environments with Unisphere for PowerMax

Application awareness for VMware is a new feature of Unisphere for PowerMax, which allows users to troubleshoot storage related problems to ESXi servers.

Using a read only vCenter user account, a vCenter or an individual ESXi server can be added to Unisphere.

The discovered items are then associated with local storage devices to Unisphere and mapped to storage related objects making it easier to investigate possible storage related problems.

The following are some of the actions a user can view on this new feature:

  • Masking Views related to ESXi server
  • Performance KPIs related to ESXi Server
  • Details of ESXi Server
  • Virtual Machines on ESXi Server
  • Storage Groups related to ESXi Server
  • Local Storage Capacity of ESXi Server
  • Details of Virtual Disk under a VM
  • Allows you to find the ESXi Server a VM resides on and related storage objects.

It is easy to identify potential storage related issues associated with your ESXi server by examining the main vCentre/ESXi server list view.

If any Storage Group is out of compliance an indication will display too highlight, also if there is a problem with Front End Ports utilization then this will able be indicated. Clear relationships between the storage objects and vCenter/Esxi server(s) make diagnosis of a potential problem swift.

The ability to locate an ESXi when given a particular VM name accelerates troubleshooting efforts and can rule in/out the problem having anything to do with the local storage array.

Discovery of VMware related objects can be scheduled by using the Job feature of Unisphere for PowerMax.

** Please Note currently no VMware related statistics are retrieved.

Getting Started with VMware Unisphere

Login to Unisphere for PowerMax

Select HOME > VMWARE > vCenters and ESXi


The user can then register a whole vCenter or add individual ESXi servers. Registering is easy!

All that is required is the vCenter/ESXi server name or IP address and a username and password.

The user then has the option to run the discovery now or Add to Job List and run at the required time.


Once done you get can view the esxi servers listed for the vCenter – note this only shows Esxi servers local to PowerMAX!

Storage group compliance and Front End colour coded status indicators draws your eye quickly to any potential issue. More information is readily available such as the VMware version and build number, CPU cores, Memory by double clicking one to the esxi server.


Investigate a potential SG Compliance Problem

In this use case the user is presented with a yellow warning triangle in the Storage Group compliance column in the main list view.

This means this Storage Group is outside of the maximum response time for it’s defined service level and might indicate a potential problem. Workload Planner calculates its weighted response time for the past 4 hours and for the past 2 weeks, and then compares the two values to the maximum response time associated with its given service level.

If one of them is in compliance and the other is out of compliance, then the compliance state is MARGINAL. If both are out of compliance, then the compliance state is CRITICAL.


To investigate further you can View All Details of an ESXi and navigate to the Performance tab.


Here you can see more KPIs and some other related performance objects.

Once the potential troublesome SG is determined the user can also now navigate to the Performance section of Unisphere and investigate further.


A tip/trick I found useful when trying to locate an Esxi server or VM is to use the Unisphere for PowerMax search icon.


Great to hear your thoughts on this fantastic VMware application awareness in Unisphere!

Blog Authored by Derek O’Mahony @DerekOMah


VxRail: Virtual Machines need to be power cycled for branch target injection mitigation (Spectre v2) to take effect

Article Number: 519601 Article Version: 8 Article Type: How To

VxRail Appliance Family,VxRail Appliance Series,VxRail Software

VxRail release with versions higher than or equal to 4.0.402 and 4.5.152 address Hypervisor-Assisted Guest Mitigation for branch target injection mitigation (Spectre v2). Patching the VMware vSphere hypervisor and updating the CPU Microcode will allow guest operating systems to use hardware support for branch target mitigation.

Please follow below sequence for the security fix to take effect for all Virtual Machines:

  1. Upgrade current VxRail cluster to versions with the security fix completely
  2. Log in vSphere Web Client with an administrative account. For each service Virtual Machine, perform Power -> Shut Down Guest OS action. Please note that if vCenter Server Appliance (VCSA) or vCenter Server Platform Service Controller (PSC) is shut down, the vSphere Web Client will be down. So please take note of the ESXi host that VCSA or PSC resides on and use vSphere Host Client or the steps at end of this article to power them back on. User-added image
  3. Wait until the VM is powered off completely, power it on to finish the power cycle operation
  4. Repeat above steps for all service Virtual Machines [in the following order]:
    1. VxRail Manager
    2. vRealize Log Insight [if deployed]
    3. vCenter Server Platform Service Controller (PSC) [if Internal]
    4. vCenter Server Appliance (VCSA) [if Internal]

For technical details, please refer to VMware KB 52085

Hypervisor-Assisted Guest Mitigation for Branch Target injection


When then the VCSA or PSC is shut down the vSphere Web Client will become inaccessible and therefore you will not be able to power them back on using the vSphere Web Client. Please use vSphere Host Client to power them back on. Alternatively, the following steps can be followed.
  1. In vCenter click the VCSA VM and select Summary Tab –> Related Objects –> Host. This gives the ESXi host running this VM.User-added image
  2. Enable SSH on the ESXi Host: Select ESXi Host –> Configure –> Security Profile –> Services –> Edit –> Enable SSH
  3. Login to ESXi Host using root credentials.
  4. Issue the following commands:
  • vim-cmd vmsvc/getallvms | grep “VMware vCenter Server”
This returns the “vmid” in this case 1 [first number on the line]; in this case the vmid=1 so a value of 1 would be used.
  • vim-cmd vmsvc/power.getstate <vmid>
  • vim-cmd vmsvc/power.off <vmid>
Wait a few mins and then power back on the VM.
  • vim-cmd vmsvc/power.on <vmid>
User-added image

Note: The VCSA could take a while to restart 30 – 60 mins depending on the size of the configuration.

Repeat these steps for the PSC.

For other self-deployed Virtual Machine, customer could decide the proper time to do the power cycle depending on the workload, but we suggest to take the action as earlier as possible. Customer is also responsible for Operating System-Specific Mitigations to take the latest OS patch for self-deployed VM’s.


How to Create Machine Catalog using MCS in Azure Resource Manager


  • Access to the XenApp and XenDesktop Service of Citrix Cloud.
  • An Azure Subscription.
  • An Azure Active Directory (Azure AD) user account in the directory associated with your subscription, which is also co-administrator of the subscription.
  • An ARM virtual network and subnet in your preferred region with connectivity to an AD controller and Citrix Cloud Connector.
  • “Microsoft Azure” host connection
  • To create an MCS machine catalog, XenDesktop requires a master image that will be used as a template for all the machines in that catalog.

User-added image

Creating Master Image from Virtual Machine deployed in Azure Resource Manager

Create a virtual machine (VM) in Azure using the Azure Resource Manager gallery image with either the Server OS or Desktop OS (based on whether you want to create Server OS catalog or Desktop OS catalog).

Refer to Citrix Documentation – install Citrix VDA software on the VM for more information.

Install the applications on the VM that you want to publish using this master image. Shutdown the VM from Azure Portal once you have finished installing applications. Make sure that the power status for the VM in Azure Portal is Stopped (deallocated)

User-added image

When creating MCS catalog we need to use the .vhd file that represents OS disk associated with this VM as master image for the catalog. If you have the experience of using Microsoft Azure Classic connection type in XenDesktop, you would have captured specialized image of the VM at this stage, but for Microsoft Azure connection type you don’t have to capture the VM image, you will only shutdown the VM and use the VHD associated with the VM as master image.

Create MCS Catalog

This information is a supplement to the guidance in the Create a Machine Catalog article. After creating master image, you are all set to create MCS catalog. Please follow the steps as described below to create MCS catalog.

  1. Launch the Studio from your Citrix Cloud client portal and navigate to Machine Catalogs in the left hand pane.

  2. Right click Machine Catalogs and click on Create Machine Catalog to launch the machine creation wizard.

  3. Click Next on the Introduction page.

    User-added image

  4. On the Operating System page Select Server OS or Desktop OS based on what type of catalog you want to create and click Next.

    User-added image

  5. On the Machine Management page, select Citrix Machine Creation Service (MCS) as the deployment technology and select the Microsoft Azure hosting resource and click Next.

    User-added image

Master Image Selection – This page provides a tree view which you can navigate to select the master image VHD. At the topmost level are all the resource groups in your subscription except those which represent the MCS catalog created by XenDesktop. When you select and expand a particular resource group, it shows list of all the storage accounts in that resource group. If there are no storage accounts in that resource group, there will not be any child items under that resource group. If you have manually created number of resource groups and storage accounts to host your manually created VMs in your subscription, the master image page will show all those resource groups, storage accounts, containers and VHDs even though not all those VHDs are master images that you want to use for the provisioning. Select the storage account that has your master image. When you expand the storage account, it shows list of containers inside the storage account. Expand the container that has master image VHD and select the VHD that you want to use as master image for the catalog.

User-added image

You need to know the VHD path in order to select it. If you have stood up a VM in Azure and prepared it to be used as a master image and you want to know the VHD path, follow the steps below:

  1. Select the resource group that has your master image VM.

  2. Select the master image VM and click Settings

  3. Click on Disks then Click OS Disks and copy the disk path.

    User-added image
    User-added image

  4. OS disk path is structured as https://<storage account name>.blob.core.window.net/<container name>/<image name>.vhd

  5. You can use the disk path obtained in the step above to navigate the tree view to select image.

Note: If you don’t shutdown the master image VM and select the corresponding VHD to create a catalog, the catalog creation will fail. So make sure if you are selecting the VHD which is attached to running VM instance, the VM is in Stopped(deallocated) state.

  1. Storage type selection – XenDesktop supports Locally Redundant Standard or Premium storage for provisioning VMs in Azure. Your master image VHD can be hosted in any type of storage account, but for the VMs to be provisioned in Azure, XenDesktop will create new storage accounts based on storage type you selected.User-added image

  2. XenDesktop will provision maximum 40 VMs in single storage account due to IOPS limitations in Azure. For example if you want to create 100 VM catalog, you will find 3 storage accounts created and VM distribution in each storage account will be 40, 40 and 20.

  3. VM instance size selection – XenDesktop will show only those VM instance sizes which are supported for the selected storage type in the previous step. Enter number of VMs and select the VM instance size of your choice and click Next.

    User-added image

  4. Network Card Selection – Select network card and the associated network. Only one network card is supported.

    User-added image

  5. Select resource location domain and enter machine naming scheme.

    User-added image

  6. Enter credentials for your resource location Active Directory.

    User-added image

  7. Review the catalog summary, enter the catalog name and click Finish to start provisioning.

    User-added image

  8. Once the provisioning is complete, you will find new resource group created in your Azure subscription which hosts, all the VMs, storage accounts and network adapters for the catalog you provisioned. The default power state for the VMs after provisioning is Stopped(deallocated).

    User-added image

Once the provisioning is complete, you will find new resource group created in your subscription that has VM RDSDesk-01 as per the naming scheme we provided, NIC corresponding to that VM and a storage account that XenDesktop created to host the OS disk and the identity disk for the VM. The VM will be hosted on the same network as that of the selected hosting resource during catalog creation and the default power state of the VM will be Shutdown(deallocated).

The resource group created by XenDesktop during the MCS provisioning will have following naming convention


To find out which resource group in the Azure portal corresponds to the catalog you created from studio, follow the steps below.

  1. Connect to your XenApp and XenDesktop service using Remote PowerShell SDK. Please visit this link to find our how to interact with your Citrix Cloud environment using Remote PowerShell SDK.
  2. Run command Get-ProvScheme -ProvisioningSchemeName <Catalog Name>
  3. Note down the ‘ProvisioningSchemeUid’ from the output of the above command.
  4. Go to the Azure portal and search for the resource group name that contains ‘ProvisioningSchemeUid’ you obtained in step 3.
  • Note:

    As a best practice you should always create a copy of your master image and use the copied image as input to the provisioning process. In future if you want to update the catalog, you can start the master image VM and make necessary changes, shut it down and again create a copy of the image which will be your update image. This helps you to use the master image VM to create multiple image updates.

    Remember to shutdown the master image VM from Azure portal before starting to create the catalog. The master image needs to be copied into catalog’s storage account once provisioning starts, so we need to make sure it is not in use by any VM, otherwise it will lead to image copy failure and eventually provisioning failure.

  • Make sure you have sufficient cores, NIC quota in your subscription to provision VMs. You are most likely going to run out of these two quotas. You may not be able to check your subscription quota limits,
  • If your master image VM is provisioned in the Premium storage account then just shutting down the VM from the portal isn’t enough. You also need to detach the disk from the VM to use it as master image in provisioning. But in Azure Resource Manager you can not detach the disk while the VM is still available. So you need to delete the VM from the portal, this will only delete the VM but keep the OS disk in the storage account. The NIC corresponding to the VM also needs to be deleted separately.
User-added image


NVP-vProxy: vSphere vCenter virtual machine updates are not automatically populated in the VMware View

Article Number: 501545 Article Version: 3 Article Type: Break Fix

NetWorker 9.1,NetWorker 9.2,NetWorker 9.1.1

The NetWorker VMware Protection integration is configured with the vProxy Appliance. The VMware Administrator adds or removes a Virtual machine to the vCenter inventory. The NetWorker Administrator does not see the virtual machine details updated in the VMware View or in the NSR Protection Group for their active NetWorker Management Console (NMC) session. For example, if a virtual machine is created in the VMware vCenter inventory, it will not show in the VMware View or NetWorker Protection Group details.

If the NMC session is left open for 24 hours, it will not automatically update the virtual machine details. If the NMC VMware View is manually refreshed or re-opened the virtual machine details will be updated properly and the VMware View will reflect the changes.

The NMC is working as designed. When the NMC is opened it runs the nsrvim process for all the configured vCenter servers to update the NetWorker resource database (nsrdb) and populate the VMware View details. Once the VMware View is populated, it does not automatically re-populate the VMware View (or NSR Protection Group) unless the user selects to “Refresh” the VMware View details.

A virtual machine was created or deleted in the VMware vCenter inventory after opening the NMC or refreshing the VMware View.

In the NMC select the VMware View and then the “Refresh” option. The VMware View refresh will run the nsrvim process for all the configured vCenter servers and update the VMware View with the changes.

The VMware vCenter environment details are stored in the NetWorker resource database (nsrdb) under the NSR Hypervisor resource. The NSR Hypervisor resource contains the “environment” variable, which stores the VMware vCenter information in XML format. This information is used by several NetWorker components to obtain details about the configured VMware vCenter servers. The nsrvim process is responsible for connecting to the configured NSR Hypervisor resources and updating the vCenter information in the nsrdb.

The nsrvim execution is called by several processes in the NetWorker environment to ensure the most recent information is available in the NSR Hypervisor attribute. The manual execution of the nsrvim, or execution via another process, will not automatically update the NMC VMware View. The only nsrvim execution that will update the NMC VMware View is via the opening of the NMC or VMware View refresh.