8. Information on other settings

This chapter provides the information on the other monitor or notification settings.

This chapter covers:

8.1. Alert Service

8.1.1. Alert Service

EXPRESSCLUSTER Alert Service is a function to report failures found in operations on EXPRESSCLUSTER to system administrators in remote locations.

Failures are reported in three ways, each serving a different purpose.

  1. E-mail report
    Alert messages in the Cluster WebUI are sent by e-mail to administrators.
  2. Warning light
    The warning light is a visual display of the status of the server. When the server shuts down successfully, the warning light goes off.
    The e-mail report and the warning light function work independently of each other.
  3. SNMP trap sending
    When a Cluster WebUI alert message is displayed, the contents of the alert are sent with an SNMP trap.
Two servers, Alert Service, a network warning light, and an e-mail

Fig. 8.1 Alert service

Alert Service allows you to:

  • Receive information about failures while not physically located in the place where management PC is. This is achieved via e-mail reporting function.
  • Receive e-mail messages on your mobile phone.
  • Visually be alerted of failures by viewing a light.
  • Recognize a failure audibly by reproducing the audio file for the network warning light.
  • Notify the servers that are configured as the destination of the details of errors by SNMP trap sending.

Mail Report notifies the content of the alert in the following format by e-mail.

Subject:
    EXPRESSCLUSTER
Body:
    Message: Server [down server] has been stopped.
    Type: nm
    ID: 2
    Host: [mail sending source server name]
    Date: [send time stamp]

8.1.2. Notes on Alert Service

  • To use the mail report and warning light function, the EXPRESSCLUSTER X Alert Service 5.1 license must be applied to the system.

  • The task of Alert Service is to send the first report of failure but not to examine or find the cause of failure. When a failure occurs, instead of using the Alert Service, try other methods, such as viewing EXPRESSCLUSTER logs or syslog, to find out the cause of the error.

  • When the warning light function is used, it is necessary to set up the command such as rsh that is supported by the warning light manufacturer.

8.1.3. Mail report actions

8.1.4. Warning Light status

The network warning light performs the following operations.

  1. When the server is started
    When the server starts up successfully, warning light changes to green.
  2. When the server shuts down
    When the server shuts down successfully, warning light goes off.
  3. When the server fails
    When the server fails, its warning light flashes in red. If all servers in the cluster fail, the warning light of the server that failed last will not work because the warning light is controlled by a normal server that monitors other servers.

Once a network warning light is lit or starts flashing, it will not go off until the cluster shuts down. Run the clplamp command introduced in the following section to put the light out. For more information on the clplamp command, see "Switching off network warning light (clplamp command)" in "9. EXPRESSCLUSTER command reference" in this guide.

For a network warning light (specified by NEC) that supports playback of an audio file, the setting also enables audio file reproduction to link to On/Off.

8.1.5. Operations of SNMP trap sending

8.2. SNMP linkage

8.2.1. SNMP linkage

SNMP linkage enables SNMP trap sending from EXPRESSCLUSTER and information acquisition by SNMP from an SNMP manager according to the EXPRESSCLUSTER MIB definitions.

Two servers, Alert Service, and an SNMP manager

Fig. 8.2 SNMP linkage

8.2.2. EXPRESSCLUSTER MIB definitions

The information sent/acquired with SNMP linkage is configured by the MIB definition files.

To use the functions of SNMP trap sending and information acquisition by SNMP, described later, MIB definition files are required.

To receive SNMP traps from EXPRESSCLUSTER by using an SNMP manager, or to acquire cluster statuses from an SNMP manager, set the EXPRESSCLUSTER MIB definition files in the SNMP manager.

For how to set the MIB definition files in an SNMP manager, refer to the manual for the SNMP manager.

The EXPRESSCLUSTER MIB definition files are placed in the following directory on the EXPRESSCLUSTER X DVD-ROM.

<EXPRESSCLUSTER_X_DVD-ROM>\Common\<version number>\common\mib

The MIB definition files provide the functions described below.

No.

MIB definition file

Description

NEC-CLUSTER-SMI.mib

Configures the EXPRESSCLUSTER MIB tree root path.

NEC-CLUSTER-EVENT-MIB.mib

Configures the trap and MIB definitions for the EXPRESSCLUSTER SNMP trap sending function.

NEC-CLUSTER-MANAGEMENT-MIB.mib
Configures MIB definitions for the following EXPRESSCLUSTER information:
- Cluster information
- Server information
- Group information

The available functions depend on the files set in the SNMP manager.

To receive SNMP traps from EXPRESSCLUSTER:

  1. NEC-CLUSTER-SMI.mib

  2. NEC-CLUSTER-EVENT-MIB.mib

To get information by SNMP:

1. NEC-CLUSTER-SMI.mib 3. NEC-CLUSTER-MANAGEMENT-MIB.mib

8.2.3. SNMP trap sending

SNMP trap sending serves to send the contents of Cluster WebUI alert messages to the SNMP manager.

To send a trap, the SNMP trap sending destination is required to be configured. Configure it by referring to Destination Settings of SNMP Trap in "Alert Service tab" in "Cluster properties" in "2. Parameter details" in this guide.

The traps to be sent are defined by NEC-CLUSTER-EVENT-MIB.

NEC-CLUSTER-EVENT-MIB defines the following MIB objects.

clusterEventNotifications group

This group defines the traps to be sent. The MIB objects defined for the group function as described below.

No.

SNMP TRAP OID

Description

clusterEventInformation
Trap for information level alerts.
A clusterEvent group MIB object is attached.

clusterEventWarning
Trap for warning level alerts.
A clusterEvent group MIB object is attached.

clusterEventError
Trap for error level alerts.
A clusterEvent group MIB object is attached.

clusterEvent group

This group defines the information appended to the traps. The MIB objects defined for the group function as described below.

No.

SNMP OID

Description

clusterEventMessage

Indicates the alert message.

clusterEventID

Indicates the event ID.

clusterEventDateTime

Indicates the time at which the alert originated.

clusterEventServerName

Indicates the server from which the alert originated.

clusterEventModuleName

Indicates the module from which the alert originated.

8.2.4. Information acquisition by SNMP

By using the SNMP protocol, some information about the EXPRESSCLUSTER configuration and status can be acquired. However, EXPRESSCLUSTER does not include SNMP agent functions. For an SNMP agent, Windows SNMP Service needs to be implemented separately.

SNMP agent

The SNMP agent serves to return a response about the configuration information or status information (GetResponse) to information acquisition requests (GetRequest, GetNextRequest) from an SNMP manager (network management software).

Note

If Windows SNMP Service has been installed when EXPRESSCLUSTER Server is installed, the SNMP linkage function is automatically registered. Otherwise, it is not automatically registered.
It needs to be manually registered; for details on how to manually register it, refer to "Setting up the SNMP linkage function manually" in "Installing the EXPRESSCLUSTER Server" in "Installing EXPRESSCLUSTER" in the "Installation and Configuration Guide".

8.2.5. MIB objects acquirable with SNMP linkage

The MIB objects that can be acquired with the SNMP linkage function are defined by NEC-CLUSTER-MANAGEMENT-MIB.

NEC-CLUSTER-MANAGEMENT-MIB defines the following MIB objects.

clusterGeneral group

This group is used to acquire cluster information. The MIB objects defined for the group function as described below.

No.

SNMP OID

Description

clusterName

Indicates the name of the cluster.

clusterComment

Indicates the comment of the cluster.

clusterStatus

Indicates the current status of the cluster.

The correspondence between the MIB value and the Cluster WebUI status is as described below.

MIB value   status
---------------------
normal       Normal
caution      Caution
error        Error
unknown       -

clusterServer group

This group is used to acquire server information. Indexes on acquisition of clusterServerTable are sorted by server priority. The MIB objects defined for the group function as described below.

No.

SNMP OID

Description

clusterServerLocalServerIndex

Indicates the index of the server receiving the present SNMP information acquisition request (clusterServerIndex).

clusterServerTable

Indicates the information table for the server.

clusterServerEntry
Indicates the server information list.
The index for the list is clusterServerIndex.

clusterServerIndex

Indicates the index for uniquely identifying the server.

clusterServerName

Indicates the name of the server.

clusterServerComment

Indicates a comment for the server.

clusterServerStatus

Indicates the current status of the server.

The correspondence between the MIB value and the Cluster WebUI status is as described below.

MIB value    status
------------------------------------------------
online       Online
caution      Suspension (Network Partition Unsolved)
isolated     Suspension (Isolated)
offline      Offline
unknown      Unknown

clusterServerPriority

Indicates the priority of the server.

clusterServerProductName

Indicates the name of the EXPRESSCLUSTER product installed on the server.

clusterServerProductVersion

Indicates the version of the EXPRESSCLUSTER product installed on the server.

clusterServerProductInstallPath
Indicates the installation path of EXPRESSCLUSTER on the server.
If the return value is other than an ASCII character, the data might be corrupt.

clusterServerPlatformName

Indicates the name of the platform on the server.

clusterGroup group

This group is used to acquire group information. The MIB objects defined for the group function as described below.

No.

SNMP OID

Description

clusterGroupTable

Indicates the information table for the group.

clusterGroupEntry
Indicates the group information list.
The index for the list is clusterGroupIndex.

clusterGroupIndex

Indicates the index for uniquely identifying the group.

clusterGroupName

Indicates the name of the group.

clusterGroupComment

Indicates a comment for the group.

clusterGroupType
Indicates the type of the group.
The correspondence between the MIB value and the group type is as described below.
MIB value        Group type
--------------------------------------
failover         Failover group
cluster          Management group

clusterGroupStatus
Indicates the current status of the group.
The correspondence between the MIB value and the Cluster WebUI status is as described below.
MIB value        status
-----------------------------------
online           Online
onlineFailure    Online Failure
offlineFailure   Offline Failure
offline          Offline
unknown          Unknown
onlinePending    Online Pending
offlinePending   Offline Pending

clusterGroupCurrentServerIndex
Indicates the index of the server on which the group is currently active (clusterServerIndex).
If the group has been deactivated, the return value is -1

8.3. Grace period dependence at the automatic failover between server groups

8.3.1. What is the grace period dependence?

One server group waits specified time for the other server group to start failover when the automatic failover is executed between server groups. When the grace period elapsed after the server down was detected, the failover is executed.

8.3.2. Condition for the grace period dependence

  • One server group waits for the other server group with any of the following configurations to start the failover.

    • Use Server Group settings in the Info tab is selected.

    • Multiple server groups are specified for Server Groups that can run the Group in the Startup Server tab

    • Prioritize failover policy in the server group is selected and Enable only manual failover among the server groups is not selected for Automatic Failover of Failover Attribute in the Attribute tab.

  • In the following cases, one server group does not wait specified time for the other server group to start failover:

    • One server executes the failover to another server within the same server group.

    • The server down is detected by the server down notification.

    • The forced stop is successfully executed while the type of forced stop is selected for other than Do Not Use, or the condition not to execute the forced stop is met.

    • The NP resolution resource is configured.

8.3.3. Displaying and changing the grace period dependence

Specify the waiting time for Grace period of server group failover policy.

If 0 is specified, one server group does not wait for the other server group to start failover

8.3.4. Notes on the grace period dependence

If any operation is done for the failover target group while the other server group waits during the grace period, the settings to wait during the grace period is canceled and the other server group does not failover.

If the once-failed server is detected to be alive while the other server waits during the grace period, the settings to wait during the grace period is canceled and the failover is not executed.

If the failover target server goes down, the failover may start later than when the grace period ends.

8.4. Witness server service

8.4.1. What is Witness server service?

Witness service is the service to receive Witness heartbeat from each server in the cluster and send back the status information of receiving the heartbeat from each server as a response. It is installed in a server outside of the cluster.

The Witness server and two cluster servers

Fig. 8.3 Witness server service

8.4.2. Notes on Witness server service

8.4.3. How to install Witness server service

Install the Witness server service by using npm command for Node.js environment. Store the Witness server service module in an arbitrary folder, and execute the following command.

> npm install --global clpwitnessd-<version>.tgz

Obtain the Witness server service module in the following path of the DVD-ROM for installation:

Common\<version>\common\tools\witnessd\clpwitnessd-<version>.tgz

8.4.4. How to configure Witness server service

To change the settings of Witness server service, edit the configuration file directly. Open the folder indicated in the first row of the execution results of the command below.

> npm list --global clpwitnessd

Example of execution results:

C:\Users\Administrator\AppData\Roaming\npm

`-- clpwitnessd@4.1.0

Edit clpwitnessd.conf.js that is stored in node_modules\clpwitnessd under the opened folder, with a text editor such as notepad.

Setting items are as follows.

Item

Default

Description

http.enable

True
Specify whether to execute HTTP server or not.
true: execute
false: not execute

http.port

80

Specify the wait port number for HTTP server.

http.keepalive

10000

Specify the keep alive time for HTTP server in milliseconds.

https.enable

False
Specify whether to execute HTTPS server or not.
true: execute
false: not execute

https.port

443

Specify the wait port number for HTTPS server.

https.keepalive

10000

Specify the keep alive time for HTTPS server in milliseconds.

https.ssl.key

server_key.pem

Specify a secret key file to be used for HTTPS server.

https.ssl.crt

server_crt.pem

Specify a certification file to be used for HTTPS server.

log.directory

.

Specify the log output destination folder.

log.level

info
Specify the log output level.
error: Only error logs are output.
warn: Error logs and warning logs are output.
info: Warning logs and information logs are output.
debug: Information logs and detailed logs are output.

log.size

1024 * 1024 * 512

Specify the log rotation size in bytes.

data.available

10000

Specify the default time limit for the communication status information of the cluster server in milliseconds.

8.4.5. How to execute Witness server service

Execute the following command to start up Witness server service in the fore ground. For how to execute the Witness server service as Windows service or Linux daemon, refer to the following section, "Using Witness server service as the OS service".

> clpwitnessd

8.4.6. Using Witness server service as the OS service

If you want to start Witness server service at the OS startup, the Witness server service requires to be registered as the OS service.

The following exemplifies how to register Witness server service as the OS service (in case of Windows service control manager and Linux systemd). The method of registration for the OS service differs depending on the environment. Configure the registration to suit your environment by referring to the explanation below.

Registration for Windows service control manager

The following exemplifies the procedure to register by using npm package winser.

  1. Install winser by npm command. Use the following command so that winser package is downloaded from npm repository and then installed.

    > npm install --global winser

  2. Create a folder to execute the service in any location. By default, this folder stores log files, SSL secret key file and SSL certificate file.

  3. Create package.json file for the service registration with winser, under the folder created in the above step 2. Enter "\\" to separate the characters of the path. The path specified for "start" is line-fed for the convenience of character numbers but actually is in one row.

    {
  "name": "clpwitnessd-service",
  "version": "1.0.0",
  "license": "UNLICENSED",
  "private": true,
  "scripts": {
    "start": "C:\\Users\\Administrator\\AppData\\Roaming\\npm\\clpwitnessd.cmd"
  }
}

  4. Execute winser command to register and start the Witness server service.

    > winser -i -a

  5. Select Control Panel -> Administration Tools -> Service, and confirm that the service (ex. clpwitnessd-service) with the name specified for "name" of package.json has been registered..

Registration for Linux systemd

The following exemplifies the procedure to register by creating the unit file of systemd.

  1. Create a directory to execute the service in any location. By default, this folder stores log files, SSL secret key file and SSL certificate file.
    (ex. /opt/clpwitnessd)
  2. Create the unit file of the Witness server service in /etc/systemd/system.
    (ex. clpwitnessd.service)
    [Unit]
Description=EXPRESSCLUSTER Witness Server
After=syslog.target network.target

[Service]
Type=simple
ExecStart=/usr/bin/clpwitnessd
WorkingDirectory=/opt/clpwitnessd
KillMode=process
Restart= always

[Install]
WantedBy=multi-user.target

  3. Execute systemctl command to register and start the Witness server service.

    # systemctl enable clpwitnessd
# systemctl start clpwitnessd