This is the web application described in the December 2005 issue of Sys Admin’s article titled “Trap Customization in an Enterprise OpenView Operations/Network Node Manager Environment”. You may want to read that article for more information on the design behind this application.
In the production environment, the OVO event system receives a large volume of SNMP events. Unabated, the events in the browser will grow and grow making it almost impossible for staff to make any sense of what the system is telling them. Consequently, mechanisms are required to make some of these alarms reported in the OVO event browser to be removed automatically (ie changed from active to history events) when the alarm condition no longer exists. Making this happen implies the need to define a clearing event to every alarm event where possible.
The idea is to configure automatic actions in the event system on the NNM collection stations. The automatic actions convert the content of the trap to a corresponding opcmsg command and invoke it to send the information to OVO. The event forwarding process is as follows:
1. NNM collection station receives a trap that needs forwarding to OVO
2. NNM’s event system executes the automatic action, a custom perl script that packages the trap information into an opcmsg command and executes it to forward the event to OVO. The reason to execute a custom perl script as an automatic action and not the opcmsg command directly is that the perl script is able to do dynamic severity translation based on a selected trap varbind instead of hard-coding a fixed OpenView severity although the custom perl script can also be configured to send the opc message at a fixed OpenView severity. By supporting dynamic severity translation, the same trap can be used as both the alarm and clearing event depending on the severity value contained in the varbind. This feature is important because many MIBs do not have separate alarm and clearing events but rely on the severity contained in a particular varbind to indicate the kind of event.
3. the opc message is sent to OVO using opcmsg
4. the message text prefix string (described next) triggers the custom template
5. the template causes the message correlation system to either add the event to the OVO active event browser for an alarm event or remove an existing alarm from the browser for a clearing event.
For our intended purpose, we use each field in the opcmsg command as follows.
The custom OVO template for handling these opcmsg messages must perform the following:
· Identify the special short prefix text string in the Msg_text using Message Key and Message Relation defined from the Message Correlation Window
· Use the object field as the correlator for message correlation
· Remove the special short prefix from msg_text before display
· A less severe event cancels a more severe event, For example, major cancels critical, minor cancels both major and critical, and so on.
The web application is named trapapp. It contains 4 menu items:
As you enter information to customize traps using the trapapp application, the information you entered is saved in different sections of a configuration file in Windows .ini file format using the Config::IniFiles module. A configuration file consists of 5 sections: trap, format, alarm, clearing and opcinfo. The format of each section is described below (please note that you do not have to know the format of the configuration file to use the application):
[trap]
trapnum=trapname~varbind1~varbind2~…~varbindN
The purpose of the trap section is to record the trap name and its varbinds and associate that information with a trap number.
[format]
trapnum=eventType~messageFormat
where eventType can be either alarm, clearing or ignore.
The purpose of the format section is to identify the traps used for generating alarm and clearing events and their message format. Traps with eventType ignore are ignored during script generation.
[alarm]
trapnum=correlatorFormat~varbind/fixedSeverity~toNormal~toWarning~toMinor~toMajor~toCritical
where varbind/fixedSeverity contains either a fixed OpenView severity (one of normal, warning, minor, major or critical) or a varbind. In the latter case, toNormal, toWarning, etc. contains the translation mapping from the varbind’s severity to that of OpenView’s.
The purpose of the alarm section is to define the correlator for alarm and clearing events and translation mapping from MIB-specific severity to that of OpenView’s.
[clearing]
trapnum=alarmTrapNum~fixedSeverity
where alarmTrapNum specifies the alarm trap this event is to clear and fixedSeverity specifies the fixed OpenView severity of the trap.
[opcinfo]
appname=appName
messagegroup=groupName
nodename=nodeName or $0
if a specific nodeName is given, it will be used as the name of the sender of trap. If $0 is specified, the host name of the agent sending the trap will be specified as the source of the message. If you are using a gateway host to send information on devices not configured in the OVO Node Bank, use a fixed nodename instead of $0.
The purpose of this section is to supply the information to complete the opcmsg command.
The trapapp web application has a configuration file named trapapp.ini. Its content is shown below:
[app]
scriptdir=./gen/
opcmsg=/opt/OV/bin/OpC/opcmsg
trapdconf=C:/Program Files/HP OpenView/NNM/conf/C/trapd.conf
proddir=/opt/trapapp/
trustedcmddir=C:/Program Files/HP OpenView/NNM/conf/trustedCmds.conf/
It consists of an [app] section and the following entries:
When you click on the Generate button, the application performs the following tasks:
In summary, the trapapp web application is designed to reduce to a minium the amount of manual configuration required on the NNM collection stations.
It is assumed that the reader is familiar with the configuration of web server, creating OVO templates and configuring NNM automatic actions using the respective OVO and NNM tools. If you do, then setting up the trapapp web application is simple. All it does is generate a custom perl script and a partial trapd.conf file for each MIB you uploaded. To set up trapapp, the following steps are required:
Invoke the trapapp application by using the url:
You can now start experimenting on the trapapp application and examine the generated partial trapd.conf file. Once you are comfortable with the generated partial trapd.conf file. You can update NNM’s trapd.conf file with the information contained in the generated partial trapd.conf file using the NNM command xnmevents as follows:
Xnmevents –replace partialConfigFile
Where partialConfigFile is the full path name of the generated partial trapd.conf file.
It is trivial to include a menu item in the web application to do this. I do not implement it because by providing such a convenience feature makes it too easy to change the production trapd configuration unintentionally while experimenting with the web GUI. It is exactly for the same reason that the trusted command list contained in the $OV_CONF/trustedCmds.conf/trapapp is only updated if the generated perl script name is not is absent in the file.
In addition, there is a one-off setup involving the following tasks on the OVO Server:
When things do not work as expected, the following checklist may help you resolve the problems:
An example configuration for the concord-diagmon.mib is set up when you installed trapapp. In this sample configuration, trap 21 (nhLiveAlarm) and 23 (nhLiveClearAlarm) have been set up as alarm and clearing events respectively. The correlator used is $A-$5-$16 (ie, agentHostName-nhElementIp-nhExceptionId). nhLiveAlarm uses content of the varbind nhSeverity as the severity to use in the opsmsg command. In this case, the possible values of nhSeverity are the same as those for OpenView.
Please note that this web application is intended for OpenView administrator’s personal use only. It is not intended to be used by the casual OpenView user. No security or concurrent access control has been implemented. Hence, if more than one user is updating the configuration for the same MIB at the same time, the result is unpredictable.
You can find trapapp updates/fixes and other free tools under “Free Tools” on the web site of Kardinia Software, http://www.kardinia.com. If you have enhanced and/or fixed any bugs in trapapp, please send me a description and your changes so that I can post it on the Kardinia web site.
Andy Yuen
October 8, 2005.