vSphere HA Failed to Create a Configuration VVol

One of the more confusing errors when getting started with implementing vSphere Virtual Volumes is the vSphere HA failed to create a configuration VVol for this Datastore. This error can be symptom of a larger connectivity and access issue with the VVol Storage Container, but more often the issue lies in vSphere HA attempting to use the VVol Datastore as a Heartbeat Datastore before each host has successfully mounted to the VVol Datastore. Which in turn will cause this error to show up.

VMware has been able to identify the root cause for this and a fix has been released in both ESXi 6.7 and 7.0.

PR 2630045/2654686: The vSphere Virtual Volumes algorithm might not pick out the first Config-VVol that an ESXi host requests

In a vSphere HA environment, the vSphere Virtual Volumes algorithm uses UUID to pick out when multiple ESXi hosts compete to create and mount a Config-VVol with the same friendly name at the same time. However, the Config-VVol picked by UUID might not be the first one that the ESXi hosts in the cluster requests and this might create issues in the vSphere Virtual Volumes datastores.

This issue is resolved in this release.

The vSphere Virtual Volumes algorithm uses a timestamp rather than an UUID to pick out when multiple ESXi hosts compete to create and mount a Config-VVol with the same friendly name at the same time.

This fix is in ESXi 7.0 Update 1c (Build 17325551) and ESXi 6.7 Patch Release ESXi670-202011002 (Build 17167734).

In the example below I will be outlining the latter issue, how to troubleshoot the error and then how to correct the error.

Identify and Troubleshoot the Error

Navigate to the VVol Datastore that has the error showing up. Here we can see that the sn1-405-vvol-ds has the vSphere HA configuration error.

Check if all the ESXi hosts in the ESXi Cluster are mounted to the VVol Datastore.

Here each host was mounted and connected to the VVol Datastore. Next, check that the Protocol Endpoint is connected to and accessible to every host in the ESXi Cluster. There is a larger connectivity issue if there are any ESXi hosts in this cluster that show the PE as inaccessible or are missing from the PE connectivity list. Cody Hosterman has put together a great connectivity troubleshooting blog post that can be found here.

Here I am checking that the connectivity with hosts looks good as well, but that was largely in part to my OCD and wanting to triple check the connectivity.

Connectivity look great, now check that the capacity is showing up correctly. With a Pure Storage VVol Datastore the default size of the VVol DS will be 8 PB.

Check that the vSphere-HA config VVol was in fact created. Navigate to the files tab and browse the VVol Datastore. Below you can see that there is indeed a vSphere-HA Config VVol created.

Clearing the Error

Correcting the error can be done a few ways. One way is to disable to vSphere HA and then enable it again. Another would be to reconfigure vSphere-HA on each ESXi Host individually. The method I use below is to configure vSphere HA on all ESXi Hosts in the Cluster.

Navigate to the Hosts Page, select the ESXi Cluster, select the Hosts Tab, highlight all of the ESXi hosts, right click and then choose Reconfigure for vSphere HA.

Once the Reconfiguration tasks are complete navigate back to the VVol Datastore. In my case I didn’t even need to refresh, but if the error is still click on the refresh button and the error should be cleared.

Should there still be an error with the vSphere HA configuration on the VVol Datastore, double check the connectivity of all of the ESXi Hosts in that Cluster again. Even from the command line double check that the vasa provider, storage container, and protocol endpoint all look healthy. Overall, if there are not any connectivity issues, this error should be resolved by running the vSphere HA reconfiguration on each of the ESXi Hosts.