Friday, October 3, 2014

vCenter Configuration Manager (VCM): jobs fail to start

Currently having some issue with VCM so wanted to provide some quick information on this particular problem.

Trying to install Agent to a Windows Machine with DCOM but when i start the process the task does not appear in the jobs window.   I can however manually install the agent on the windows machine after which i can successfully connect through VCM.

Troubleshooting:

Verified all of the following

  • Collector service is running.  
  • My user account has VCM or SQL permissions to run these jobs.
  • Network authority account has local admin to windows machine
  • DNS and reverse DNS not set up properly for the target marchine or VCM.
    • Use nslookup on VCM to validate NetBIOS, FQDN and reverse IP lookup for the target machine.
  • DCOM not disabled by GPO on domain
  • Firewall not enabled on VMC or client windows machine

Viewing of log files for VCM:


RDP to the VCM server and look at (VCM default install location) C:\Program Files (x86)\VMware\VCM\Tools\EcmDebugEventViewer.

Launch EcmDebugEventViewer.exe and Press F5. This tool will display all the Warnings, Errors, and Exceptions and may reveal some additional conditions we'll need to troubleshoot this issue.


Resolution:

Circled back to agent install task not showing in jobs window at all and traced problem down to broker services not running on my VCM databases.

  • Open SSMS for SQL and connect to database server
  • Run this query to verify which database has broker enabled:
    • select name, is_broker_enabled from sys.databases
  • Note: The broker value should be 1 for all databases, except master and model.
  • For each database on which the broker is disabled, run this SQL command:
    • alter database [Database Name] SET ENABLE_BROKER

What we did find is that if this service has been down for a very long time you could possible receive SQL deadlock errors in the log files and will also be unable to login to environment.  In SQL you will see alot of delete commands by running script "exec_sp3".  Just let this run out and complete.  Could take a couple of hours.