Tuesday, January 19, 2016

vSphere 5.5 - Solution for VASA setup errors with VNX

Over weekend I wanted to review some of the features and functionalities that VASA and VSI provides inside vCenter server for a VNX array(5300).  VASA configuration should be straight forward but ran into some strange issues which took some extensive troubleshooting:


VMware compatibility guide for EMC VASA provider specifies that VASA support is part of VNX Block OE 05.32; no additional software is required.  Our VNX 5300 has FLARE 5.32 which should allow for straight connectivity to the VNX...

On vCenter server web client select your vCenter server in inventory list on left.  Select Manage tab in right pane -> Storage provider sub-tab. Click the green button.

URL for direct VNX block:
https://<IP address of storage controller>/vasa/services/vasaService
URL for direct VNX file:
https://<IP address of control station>:5989/vasa/services/vasaService
URL for SMI-S:
https://<IP address of SMI-S server>5989/vasa/services/vasaService

When trying to connect directly to VNX block I received the following error message:

In vCSA i reviewed the following error messages in sps.log under \var\logs\vmware\vpx\sps\
"Received fatal alert: unknown_ca"

The most common issue that seems to be related to the VASA configuration is a expired certificate for the storage monitoring server.
This can be verified with the following commands on vCSA:
# cd /etc/vmware-vpx/ssl/
# keytool -keystore sms.keystore -storepass testpassword -list -v
Check the valid from and until in the output provided.
This was NOT my issue.

To further test this I installed a windows 2012 server with SMI-S. Same connectivity errors appeared.
So what is going on here!

Some further troubleshooting was in order:
- I regenerated the storage management certificates and restarted the vCenter server appliance
- I restarted the array's management server.
- I added the vCenter server certificate to trusted certificates on VNX
- Verified the user created on VNX has VM administrator role.
- For user authentication if you create a local user on VNX, then be sure to add local\username for authentication in service provider.
- Verify that there is an array connected to SMI-S.  this can be verified by running the command "symcfg list" or logging in with testsmiprovider.exe and executing "dv" command.

Still no luck.
Looking further at the sps.log file I figured the problem had to be with the vCenter server that is sending the certificate to the array.


I sticked with SMI-S since it provides more customization in administration console.
https://<IP of SMI-S>:5989/ECOMConfig/
    Username and password the same   (admin / #1Password)
Under security select "SSL Certification Management.
Here we want to select option 3 and import CA certificate from file...

So which certificate do we import?  well that would be the sms.keystore cert, but we need the cert as PEM data.  Do this perform the following commands on vCSA:

# cd /etc/vmware-vpx/ssl/
# keytool -importkeystore -srckeystore sms.keystore -destkeystore /tmp/sms.pkcs -srcstoretype JKS -deststoretype PKCS12
# cd /tmp
# openssl pkcs12 -in sms.pkcs -out sms.pem

Now open the sms.pem in and editor and copy the information from ---Begin Certificate--- to ---End Certificate---
Paste into the textbox to submit the CA to SMI-S.

Now in vCenter service provider, setup connection to SMI-S again.