Wednesday, May 25, 2016

vRealize Log Insight: Configuring agents

The vSphere content pack provides powerful insight into your vSphere logs, allowing you to make informed and proactive decisions within your environment.  For the exercise I am just reviewing some of the VMware products and providing notes I took during installation. Sorry if they seems a bit all over the place :)

Log Insight agent now gets pre-installed on some of the appliances which is great and means no need to install agents manually.  Some of the VMware products that has agent pre-installed:

vRealize Business
vRealize Operations Manager (beginning from 6.1)
vRealize Orchestrator (beginning from 7.0.1)
vRealize Automation (beginning from 7.0.1)
vRealize Log Insight

Here are some basic functions which will help a lot for instructions on content packs:

Install Content Packs:
Login to vRealize Log insight.
Select the stack menu button in top right hand corner
Select Content Packs
Installation has been simplified a lot since you do not have to go to VMware solution exchange anymore to download and manually install the content packs, it is available straight from Marketplace window.  Super awesome!
Just click on Install for which ever content pack you want to install. 

How to view setup instructions.
Select the stack menu button in top right hand corner
Select Content Packs 
Select Installed content pack
Click the cog wheel -> Setup instructions

To verify if agent configuration from Log Insight was pushed successful to server:
Check the affective file to see if the correct agent configuration file logs has been pushed to the liagentd.
C:\ProgramDATA\Vmware\Log insight agent\liagent-affective

View the agent configuration settings:
Login to vRealize Log insight.
Select the stack menu button in top right hand corner
Select Content Packs 
Select Installed content pack
Select Agent Groups tab
Find group name and review the Notes and Configuration

Agent Groups

Agent Groups comes as part of the content packs you installed. This is required for dashboard to work correctly. If you use syslog-ng, you will still receive the events but the vSphere content pack dashboards will not work.

  • I would always recommend making a copy of the original 
  • Provide a new name
  • Save it
  • Provide a filtered list of hosts which could be by name, IP address or wildcards.  These hosts should already have been already registered to Log Insight via their Agent configuration.
  • Save the Agent Group. 

The configuration is automatically pushed out to the selected hosts and log messages will begin flowing in.

Install agents on linux:
This is of course not part of VMware products but providing the steps to manually install the agent on a linux box which you still need to do sometimes.
  1. Make sure the hostname is set under /etc/hosts, /etc/HOSTNAMES, hostname <newhostname>  (otherwise server will show up with localhost hostname)
  2. Copy the bin file to appliance (this is SUSE so have to copy the bin)
  3. Chmod +x <agentfile>.bin
  4. ./<agentfile>.bin
  5. Vi /etc/liagent.ini
  7. (/etc/init.d/liagentd status/stop/restart)

In order to download the agent from server and install the agent I use following commands: 
# curl -o /tmp/liagent-current.rpm http://LOGINSIGHT-SERVER:9000/api/v1/agent/packages/types/rpm ; rpm -Uvh /tmp/liagent-current.rpm 


NSX Manager
Sends all audit logs and system events from NSX Manager to the syslog server.
1 Log in to the NSX Manager virtual appliance.
2 Under Appliance Management, click Manage Appliance Settings.
3 From the Settings panel, click General.
4 Click Edit next to Syslog Server.
5 Type the IP address of the syslog server.
6 Required Type the port and protocol for the syslog server.  If you do not specify a port, the default UDP port for the IP address/host name of the syslog server is used.
7 Click OK.

NSX Edge
NSX Edge events and logs related to firewall events that flow from NSX Edge appliances are sent to the syslog servers.
1 Log in to the vSphere Web Client.
2 Click Networking & Security and then click NSX Edges.
3 Double-click a NSX Edge.
4 Click the Manage tab and then click the Settings tab.
5 In the Details panel, click Change next to Syslog servers.
6 Type the IP address of both remote syslog servers and select the protocol.
7 Click OK to save the configuration.

NSX Controllers:
The only supported method on configuring the syslog server on the NSX controllers is through the NSX API which is described in the KB below:
I did however found another way to perform this through SSH but use at own risk and I still recommend using the NSX API!

SSH into NSX controller:
Change Controller cluster password
vCenter server -> Networking & security -> Installation -> Management
NSX controller select
Actions -> Change controller cluster password
12 character min
# show syslog-exporters

add syslog-exporter <exporter-name> <syslog-severity-level> <syslog-facility-list> <name-or-ip-address> <port-number> <syslog-protocol>
    Add a syslog exporter

add syslog-exporter-facility <syslog-exporter> <syslog-facility-list>
    Add a facility to a syslog exporter

Example:  # add syslog-exporter nsx-controller-syslog INFO kern,user,mail,deamon,auth,syslog,lpr,news,uucp,cron,security,ftp,ntp,logaudit,logalert,clock,local0,local1,local2,local3,local4,local5,local6,local7,api,api_request,api_request_content,api_request_header,logical_net,system,transport_net <LI-host> 514 UDP

vRA 7:
Install content packs:
Vrealize orchestrator
Download windows agents from administration -> Management -> Agents -> Right at bottom of screen!
Install agents on windows servers  (management, DEM, Web)
From drop-down agents select vRealize 7 - Windows and create filter for only the windows server for instance hostname = wdvra*
Vra-dem, vra-dem-metrics, vra-deo, vra-deo2
Under agent configuration update the paths where necessary like for instance vra-deo where the directory is normally <hostname>-DEO after Distributed Execution Manager folder  "C:\Program Files (x86)\VMware\vCAC\Distributed Execution Manager\DEO\Logs\"  BUT SHOULD BE C:\Program Files (x86)\VMware\vCAC\Distributed Execution Manager\FQDN-DEO\Logs\
If you have multiple DEM servers or management servers then create  another file log called vra-dem2 to add the 2nd server file location.

For vRA appliances:
Just update the \etc\liagent.ini with the hostname for vrealizeloginsight server.
Restart service \etc\init.d\liagentd restart

vRealize Orchestrator:
Some good information from VMware blog on Orchestrator for vRealize Log insight.

Login to vrealize orchestrator control center.
Select Log -> Logging Integration
Check box for "Enable logging to a remote log server"
Currently only Log4j is supported but upcoming release after 7.0.1 should support Log Insight Agent
Enter Host, Port and protocol. 
Test Connection

Problems experienced:
This did not work and got an error "HTTP Status 500 - Failed to edit Log Insight Agent configuration file!"
I create another blog to show how to fix this problem:

Agents group template does not show up and had to uninstall and reinstall the agent.