Wednesday, July 3, 2013

PowerCLI - Identify LUN names for each datastore

Nice quick and easy script with Powercli for identify the LUN names for each datastore.
This is useful for debugging the log files where only the LUN name is specified.

I am a big fan of Powergui where i run and create all my scripts in!
http://powergui.org/index.jspa
http://communities.vmware.com/community/vmtn/automationtools/powercli


Script:
(replace localhost with your own vcenter server)

Connect-VIServer localhost
Get-VMHost | Get-Datastore | where {$_.Type -eq "VMFS"} |
Select Name,@{N="LUN";E={(Get-ScsiLun -Datastore $_ | Select -First 1).CanonicalName}}


Disclaimer:
Please use this script at your own risk and test it out in your test lab first before using it in production.

vCloud Director 5.1 bug with vCDNI network pools

This is something i picked up during my initial configuration of vCloud director and stumped me for a many days until i was finally able to figure out what was going on.

I created various isolated-backed networks, however during the course of the configuration changes were required and I delete ALL of the network pools to start over. When I then recreated the network pools with the same VLAN.  When I deployed the vApps with local vApp network the VM's did not want to communicate with each other at all.

Debugging the problem:

I tried recreating all of the network pools again, and deployed the vApps multiple times with different configuration.  Tried to manually setup IP address and multiple network cards.  However when i change the VM network to a portgroup that was not part of vcloud director I was able to communicate with the VM which lead me to believe the issue was not with the VM itself but with the vCloud network component. Tested the vShield Manager which seems to be working correctly.
VMware support case opened:

VMware was unable to figure out the problem at the time...
vCloud director 5.1.1.868405
vShield manager 5.1.1-848085


Resolution:

Did some research to see if any other users were experiencing this problem and ran into the a community post which describe the problem to the teeth exactly like mine. Link below.

Solution we both came up with was to re-prepare the physical ESXi host within vCloud director, without needing to do anything else.
Another workaround to this was to always leave one network pool, and this problem only appears when you delete all the isolated-backed network pools.


Links:

http://communities.vmware.com/message/2138225#2138225

Tuesday, July 2, 2013

VMware VCenter Server upgrade to 5.1.1 - Do not click "Cancel.

During the upgrade of VMware VCenter server to 5.1.1, DO NOT click "Cancel" on the following prompt during the upgrade.






I was upgrading from 5.1.0 to 5.1.1 and during the upgrade i came to the following prompt and by reading the prompt i assume i needed to reboot my server right now in order to continue on which i select "Cancel".

This however caused the upgrade to fail, but not without the upgrade for SSO actually trying to upgrade my environment and corrupting the SSO database and server.  Afterwards i could not get Vcenter server to start or any authentication to work.

As with all my work I had a good snapshot and database backup taken before the upgrade.

Resolution:

After a support case with VMware and an hour later or debugging, we could not get SSO working again and we reverted back to the snapshot to start the upgrade process over.

Lesson learned:

Always take snapshot and databases backup before any upgrade or installation!
   This does not just apply for VMware but all applications.


VCenter Server upgrade from 5.1.0 to 5.1.1

Current version:

VCenter server:       5.1.0 build     880146
ESXi hosts:             5.1.0  build    799733


New version:

5.1. U1a

ESXi software file:  VMware-VMvisor-Installer-5.1.0.update01-1065491.x86_64
VCenter server:       VMware-VIMSetup-all-5.1.0-1123966


There are a lot of blogs and post and documentation on upgrading so i am not going to bore you with screenshots and detailed explanation but just give me short point summary of how my upgrade process:



  1. Create snapshot/backup of VCenter Server!
  2. Create backup of VCenter Server, SSO and Update Manager database.
  3. Copy install ISO to server and attached with virtual iso application like "Daemon tools".
    1. I do this because if server is rebooted and you have the ISO attached through Vcenter server client or web client it will loose the connection.
  4. Do not perform Simple install!
  5. Select each VMware production individually by upgrading vCenter Single Sign On first.
  6. Upgrade the new vSphere web client.
  7. Login to new web client and under SSO verify that domain is still setup correctly.
  8. Upgrade vCenter Inventory Service
  9. Upgrade vCenter Server
  10. Upgrade vSphere client if needed
  11. Upgrade vSphere Update Manager
  12. Reboot the server. 
    1. (I was on phone with VMware support due to corruption of upgrade and they said  I that reboot is not required until all applications have been installed and completed)

More on the corruption in another blog!

ESXi 5.1 - network port lost connection without notification - Dell Poweredge

I ran into a very serious bug the other day which caused all our production Virtual Servers to loose network connectivity on a specific ESXi host without any notifications or alarms from VMware VCenter server.  Actually VMware was completed unaware of this problem and just continued working as normal with all our servers offline.

Debugging the problem:

Reviewed all the physical ports and found that a particular NIC lost its VLAN's and VMware did not recognize this so all VM's was left in disconnected state.  No network errors was detected on the Cisco physical switches as well as from VMware so no fail over took place

Monday, July 1, 2013

VMware Vcloud Director - Deploying of VM within vApp renames the vmxnet3 ethernet adapter to #2 and lost connection. (FIXED)

I just ran into a very interesting problem today within vCloud Director when a vApp is deployed from catalog a VM will loose its network connection.  Even when Guest OS customization is disabled.


We have a vApp with multiple VMs, all Windows 2008 R2 SP1.
The vApp was saved into catalog with "make identical copy"
After creating the vApp from catalog template all the VM's come up correctly accept the one installed with Exchange.

Debugging the problem:

It seems the network was lost and then gets recreated but is set to DHCP so no network connections possible.

  • Tried to fix by disabling the "guest OS customization" before powering on the vApp.
  • Recreated the network card on the particular VM but with no luck.

Quick fix:

Shutdown the VM
Delete the Virtual network adapter.
Power on the VM
Use device manager to remove all network interfaces with name "VMXNET".
    ( For more information, see the Microsoft Knowledge Base articles 241257 and 2550978.)
Shutdown the VM
Edit VM properties through Vcloud director interface and under the hardware tab add a new virtual network adapter.
Power on VM with force customization.


Resolution:

The solution to our problem was to install the hotfix from Microsoft:

http://support.microsoft.com/kb/2550978

Direct download of the file is available here:

After install the hotfix and then saving the vApp to catalog we were able to deploy the vApp without any further problems.


Links:

Found the following KB article on VMware: