7. Information on other settings

This chapter provides the information on the other monitor or notification settings.

This chapter covers:

7.1. Shutdown monitoring

7.1.1. Shutdown monitoring

In shutdown monitoring, it is monitored if the OS is stalled when cluster or server shutdown is performed by an EXPRESSCLUSTER command.
If the cluster daemon assumes the OS is stalled, forced reset is executed.

7.1.2. Displaying and changing the shutdown monitoring

  • Performs consistently
    Shutdown is monitored. The heartbeat (see "5. Heartbeat resources details") timeout must be longer than the time required for the OS to shut down, including the applications exiting.
  • Performs only upon the occurrence of a group deactivation failure
    Shutdowns are monitored only upon the occurrence of a group deactivation failure. The heartbeat timeout (see "5. Heartbeat resources details") must be longer than the time required for the OS to shut down, including that needed for the applications to quit.
    It is recommended that you set Performs only upon the occurrence of a group deactivation failure if you are using shared disks, mirror disks or hybrid disks.
  • Disable
    Shutdown is not monitored.

7.1.3. Shutdown monitoring method

You can select how to monitor shutdown from:

7.1.4. Setting of SIGTERM

SIGTERM is issued when shutting down the OS. The range of shutdown stall monitoring and what will be performed at successful OS shutdown are determined by the setting, "Enable SIGTERM handler." When the monitoring method is set to keepalive, what will be performed is the same as when softdog is set.

  • Monitoring method: softdog

    Successful shutdown (when softdog is selected and SIGTERM is enabled)

    When SIGTERM is enabled, the stalled status cannot be detected because monitoring of the shutdown ends if the OS issues SIGTERM during shutdown.

    Successful shutdown (when softdog is selected and SIGTERM is disabled)

    It is recommended to disable SIGTERM if softdog is selected as a method of monitoring.

  • Monitoring method: ipmi / ipmi(High-End Server Option)

    Successful shutdown (when ipmi is selected and SIGTERM is enabled)

    When SIGTERM is enabled, the stalled status cannot be detected because monitoring of the shutdown ends if the OS issues SIGTERM during shutdown.

    Successful shutdown (when ipmi is selected and SIGTERM is disabled)

    • Even if the shutdown is successful without any stalled status, a server is reset by ipmi.

    • On a server that can be powered off by software, reset is not performed.

    It is recommended to enable SIGTERM if ipmi is selected as a method of monitoring.

  • When a stalled status occurs in OS shutdown.

    When a stalled status in shutdown is detected

7.1.5. Using heartbeat timeout

Use the timeout value for shutdown monitoring with the heartbeat timeout value.

7.1.6. Timeout

Specify the timeout value when the heartbeat timeout value is not used as the timeout value for shutdown monitoring.
A value of less than the heartbeat timeout value must be specified to prevent both systems from activating when a failover occurs upon detection of a server down.

7.2. Bonding

7.2.1. Floating IP resource

Notes

If you specify "active-backup" to bonding mode, the communication may be temporarily lost when switching slave interfaces.

Bonding setting example

When you configure the settings for FIP resource by the Cluster WebUI, separate the IP address and bonding device with "%" in Details tab of Properties as described below.

Example: Setting "bond0" as device name, "192.168.1.3" as IP address

Note

For interconnection IP address, specify IP addresses only.

The following shows example settings to use FIP resource on bonding:

Device

Slave

Mode

bond0
eth0
eth1
active-backup(1)
balance-tlb(5)

bond0
eth0
eth1
active-backup(1)
balance-tlb(5)

When FIP resource is enabled on srv1 by ifconfig: (bonding mode is set to "balance-tlb(5).")

  1. Device where eth0 and eth1 are bonding.
    Used the public LAN, and 2nd interconnect LAN

  2. FIP enabled on bond0

  3. Used for the 1st interconnect LAN

7.2.2. Mirror disk connect

Notes

It is not recommended to use a mirror disk connect on bonding because communication may be interrupted temporarily when switching slave interfaces. Depending on the timing of mirroring, mirror recovery may be performed after switching bonding has completed.

An example of bonding setup

The following is an example of setting up bonding on a mirror disk connect:

Cluster Server

Device

Slave

Mode

srv1

bond0
eth1
eth2
balance-rr(0)
active-backup(1)
balance-tlb(5)

srv2

bond0
eth1
eth2
balance-rr(0)
active-backup(1)
balance-tlb(5)

7.3. Forced stop

7.3.1. What is Forced stop?

This function forcibly stops the failing server by the another normal server when it is recognized that the server is failing.

This function stops a physical machine by using the IPMI function.

It stops the guest OS on a virtual machine by using the VMware vCenter Server function.

In addition to the functions above, you can execute a script in which the procedure for stopping the failing server is written. For details, refer to "Script for forced stop" in "7. Information on other settings" in this guide.

7.3.2. Conditions for performing forced stop

  • Forced stop is not performed in the following cases:

    • When the failover group successfully stops before the server fails

    • When the server is shut down by the clpdown command, the OS shutdown command or Cluster WebUI and the failover group successfully stops

    • When the cluster is stopped by the clpcl command or Cluster WebUI and the failover group successfully stops

    • When the server fails and there is no failover group to perform failover from the failing server to another server
      (including when the failover group is not activated in the failing server)

  • Forced stop is performed in the following case:

    • When the server is failing and there is a failover group to perform failover from the failing server to another server

7.3.3. Commands to be used for forced stop

The ipmitool command is used.

Configure the following option values for the command execution on the BMC tab of the server properties.

Options for the ipmitool command

Information configured on the BMC tab of the server properties

-H ip_address

IP address

-U username

User name

-P password

Password

When a command line is not specified for Forced Stop Action in the BMC tab of the server properties, the following commands are executed.

Forced Stop Action

Parameters

BMC Power Off

ipmitool -H ip_address -U username -P password power off

BMC Reset

ipmitool -H ip_address -U username -P password power reset

BMC Power Cycle

ipmitool -H ip_address -U username -P password power cycle

BMC NMI

ipmitool -H ip_address -U username -P password power diag

If the above commands fail, execute the following commands:

Forced Stop Action

Parameters

BMC Power Off

ipmitool -H ip_address -I lanplus -U username -P password power off

BMC Reset

ipmitool -H ip_address -I lanplus -U username -P password power reset

BMC Power Cycle

ipmitool -H ip_address -I lanplus -U username -P password power cycle

BMC NMI

ipmitool -H ip_address -I lanplus -U username -P password power diag

See "IPMI command" in "4. Monitor resource details" of "Monitor resource " for the options used for the actions.

The vmcontrol command of the VMware vSphere Command Line Interface (vCLI) is used to forcibly stop the guest OS on a virtual machine. This function cannot be used if VMware vSphere Command Line Interface (vCLI) is not installed.

Specify the following option values for the command execution.

Option for the vmcontrol command

Information configured for Virtual Machine Forced Stop Setting on the Extension tab of the cluster properties

Information configured for Input for Virtual Machine name on the Info tab of the server properties

--server ip_address

IP address

-

--username username

user name

-

--password password

password

-

--vmname virtualmachine

-

Virtual machine name

The following option is used for action.

Command

Option

Description

vmcontrol

--operation poweroff

Powers off the guest OS on a virtual machine

7.3.4. Specifying the command to be used for forced stop

It is also possible to forcibly stop a physical machine server by specifying an arbitrary command line to be used for the forced stop in Forced Stop Action in the BMC tab of the server properties.

To specify the command line, use the following replacement strings so that the setting values of the server properties are applied on the command line.

Replacement string name
Replacement target
(Setting item in the BMC tab of the server properties)
Replacement target
(Setting item in the forced stop action in the extension tab of the cluster properties)

CLP_BMC_HOST

IP address

-

CLP_BMC_USER

User name

-

CLP_BMC_PASSWORD

Password

-

CLP_BMC_ACTION

-

Forced Stop Action

Characters to be replaced by the replacement string (CLP_BMC_ACTION) for the forced stop action are as follows.

Forced Stop Action

Characters to be replaced by replacement string

BMC Power Off

off

BMC Reset

reset

BMC Power Cycle

cycle

BMC NMI

diag

Note

In the forced stop action, the action to be executed differs depending on whether the replacement string, CLP_BMC_ACTION is specified or not.

  • When CLP_BMC_ACTION is included in the command line:
    The action selected in the forced stop action of the cluster properties is executed.
  • When CLP_BMC_ACTION is not included in the command line:
    The action selected in the forced stop action of the cluster properties is not applied.

Example of the command specified for the forced stop action by using the replacement strings:

ipmitool -H CLP_BMC_HOST -U CLP_BMC_USER -P CLP_BMC_PASSWORD power CLP_BMC_ACTION

7.3.5. Displaying and changing the details of forced stop

For the forced stop settings, refer to "Cluster properties - Extension tab", "Server properties - Info tab" , and "Server properties - BMC tab" in "2. Parameter details" in this guide.

7.3.6. Notes on the forced stop

  • Forcibly stopping the guest OS on a virtual machine
    Only power off operation can be performed. This function cannot be used if communication with VMWare vCenter Server cannot be performed.
  • Notes on ipmitool
    See "IPMI command" in "4. Monitor resource details" of "Monitor resource".
  • Impacts of forced stop
    When you use the forced stop function, the following functions are influenced because power off, reset, power cycle or NMI is forcibly performed regardless of the OS or server status.
    • Dump collection
      Since it is not recognized that dump files are being collected, power off, reset or power cycle is performed even though dump collection is being performed, so dump collection does not complete.
    • Power on within the heartbeat timeout
      When the server is powered on again for the purpose of maintenance etc. within heartbeat timeout, power off, reset, power cycle or NMI may occur after heartbeat timeout has elapsed.
  • BMC network settings
    Configure the settings so that the IP address of the LAN port for BMC management and the IP address which OS uses can communicate with each other. This function cannot be used in the environment where the network for the BMC management is blocked.
    Set the same IP address that is configured in the LAN port for the BMC management to the BMC tab of the server properties.
    See the server's manuals etc. for information on how to configure the IP address of the LAN port for the BMC management etc.

7.4. Script for forced stop

7.4.1. What is the script for forced stop?

When it is recognized that the server is failing, any script created by the user can be executed on one of the rest of servers working normally.
The failing server can be stopped forcibly by using the script.
Moreover, using the script enables to check whether the forced stop is successful or unsuccessful and to control whether to execute the failover or not.

7.4.2. Conditions for executing the script for forced stop

  • The script for forced stop is not executed when:

    • The failover group successfully stops before the server fails

    • The server is shut down by the clpdown command, the OS shutdown command or Cluster WebUI and the failover group successfully stops

    • The cluster is stopped by the clpcl command or Cluster WebUI and the failover group successfully stops

    • The server fails and there is no failover group to perform failover from the failing server to another server
      (including when the failover group is not activated in the failing server)

  • The script for forced stop is executed when the server is failing and there is a failover group to perform failover from the failing server to another server.

7.4.3. Features of the script for forced stop

Environment variables used in the script for forced stop

EXPRESSCLUSTER stores the data such as the information of a failing server to environment variables.

You can use the following environment variables for branch conditions in the script to describe the procedure tailored to the operations of your system.

Environment variable

Setting value

Description
CLP_SERVER_DOWN
...Down server name

Server name

Specifies the name of the failing server
CLP_SERVER_LOCAL
...Local server name

Server name

Specifies the name of the server where the script is executed.
CLP_VMNAME
...Virtual machine name

Virtual machine name

Specifies the virtual machine name set in the server properties.
CLP_DATACENTER_NAME
...Data center name

Data center name

Specifies the data center name set in the server properties.
CLP_VCENTER_HOST
...Host name for vCenter

Host name

Specifies the host name set in the virtual machine forced stop setting.
CLP_VCENTER_USER
...User name for vCenter

User name

Specifies the user name set in the virtual machine forced stop setting.
CLP_VCENTER_PASSWORD
...Password for vCenter

Password

Specifies the password set in the virtual machine forced stop setting.
CLP_BMC_HOST
...IP address for BMC

IP Address

Specifies the IP address set in the server properties.
CLP_BMC_USER
...User name for BMC

User name

Specifies the user name set in the server properties.
CLP_BMC_PASSWORD
...Password for BMC

Password

Specifies the password set in the server properties.

Return value of the script for forced stop

Return 0 when the script terminates normally.

7.4.4. Displaying and changing the details of the script for forced stop

For the settings of the script for forced stop, refer to "Extension tab" in "Cluster properties" in "2. Parameter details" in this guide.

7.4.5. Notes on the script for forced stop

  • Describe the customer-defined process in the script to stop the server.

  • When using the script for forced stop, refer to "Notes on the forced stop - Impacts on forced stop" of "Forced stop" in "7. Information on other settings" in this guide.

  • When the forced stop function and the script for forced stop is used together, they are executed in the following order.

    1. The forced stop function

    2. The script for forced stop

7.5. Chassis Identify

7.5.1. Chassis identify

This function allows for the other normal server to report the server failure by blinking the chassis ID lamp of a failing server by using the IPMI function when it recognizes that the server is failing

7.5.3. Behavior of the chassis ID lamp blinking when the cluster stops

If the chassis ID lamp of a server in the cluster is in the blinking status when the cluster stops, the chassis ID lamp may keep blinking for 250 seconds at the maximum.

7.5.4. Commands to be used for chassis identify

The ipmitool command is used.

If the commands are not installed, this function cannot be used.

Specify the following option values for the command execution in the BMC tab of Server Properties.

The alarms/ialarms command option

Configured in the BMC tab of the server properties

-N ip_address

IP address

-U username

Use name

-P password

Password

When the command lines are not specified for Flash and Turn off of the chassis identify lamp in the BMC tab of the server properties, the following command is executed.

In case of alarms:

Chassis Identify

Parameters

Flash

ipmitool -H ip_address -U username -P password chassis identify 250

Turn off

ipmitool -H ip_address -U username -P password chassis identify 0

In case of ialarms:

Chassis Identify

Parameters

Flash

ipmitool -H ip_address -I lanplus -U username -P password chassis identify 250

Turn off

ipmitool -H ip_address -I lanplus -U username -P password chassis identify 0

7.5.5. Specifying the command to be used for the chassis identify function

It is also possible to execute the chassis identify function by specifying an arbitrary command line used for the chassis identify function in Flash and Turn off of the chassis identify lamp in the BMC tab of the server properties.

To specify the command line, use the following replacement strings so that the setting values of the server properties are applied to the command line.

replacement strings name

Replacement target (Setting item in the BMC tab of the server properties)

CLP_BMC_HOST

IP address

CLP_BMC_USER

Use name

CLP_BMC_PASSWORD

Password

Example of the chassis identify command specified by using the replacement strings:

ipmitool -H CLP_BMC_HOST -U CLP_BMC_USER -P CLP_BMC_PASSWORD chassis identify 250

7.5.6. Displaying and changing the chassis identify details

For the chassis identify settings, refer to "Cluster properties - Alert Service tab" and "Server properties - BMC tab"in "2. Parameter details" in this guide.

7.5.7. Notes on Chassis identify

  • Notes on ipmitool
    See "IPMI command" in "Monitor resource" in "4. Monitor resource details".
  • BMC network settings
    Configure the settings so that the IP address of the LAN port for BMC management and the IP address which OS uses can communicate with each other. This function cannot be used in the environment where the network for the BMC management is blocked.
    Set the same IP address that is configured in the LAN port for the BMC management to the BMC tab of the server properties.
    See the server's manuals etc. for information on how to configure the IP address of the LAN port for the BMC management etc.

7.6. Alert Service

7.6.1. What is Alert Service?

EXPRESSCLUSTER X Alert Service (hereafter Alert Service) is a function to report failures mentioned above found in EXPRESSCLUSTER-installed cluster systems to system administrators in remote locations.

Failures are reported in three ways, each serving a different purpose.

  1. E-mail report
    Alert messages in the Cluster WebUI are sent by e-mail to administrators.
  2. Network Warning light
    The network warning light is a visual display of the status of the server. When the server shuts down successfully, the network warning light goes off.
    The e-mail report and the network warning light function work independently of each other.
  3. SNMP trap sending
    When a Cluster WebUI alert message is displayed, the contents of the alert are sent with an SNMP trap.

Alert Service allows you to:

  • Receive information about failures while not physically located in the same place as the management PC. This is achieved via e-mail reporting function.

  • Receive e-mail messages on your mobile phone.

  • Visually be alerted of failures by viewing the network warning light.

  • Recognize a failure audibly by reproducing the audio file for the network warning light.

  • Notify the servers that are configured as the destination of the details of errors by SNMP trap sending.

Mail Report notifies the content of the alert in the following format by e-mail.

Subject:

EXPRESSCLUSTER

Body:

Message: Server [down server] has been stopped.
Type: nm
ID: 2
Host: [mail sending source server name]
Date: [send time stamp]

7.6.2. Notes on Alert Service

  • To use the mail report and network warning light functions, EXPRESSCLUSTER X Alert Service 4.2 for Linux is required.

  • The task of Alert Service is to send the first report of failure but not to examine or find the cause of failure. When a failure occurs, instead of using the Alert Service, try other methods, such as viewing EXPRESSCLUSTER logs or syslog, to find out the cause of the error.

  • If you use the Linux network warning light function, it may prove necessary to install the rsh package

7.6.3. Mail report actions

7.6.4. Network Warning Light status

The network warning light performs the following operations.

  1. When the server is started
    When the server starts up successfully, warning light changes to green.
  2. When the server shuts down
    When the server shuts down successfully, warning light goes off.
  3. When the server fails
    When the server fails, its warning light flashes in red. If all servers in the cluster fail, the warning light of the server that failed last will not work because the network warning light is controlled by a normal server that monitors other servers.

Once the network warning light is lit or starts flashing, it will not go off until the cluster shuts down. Run the clplamp command introduced in the following section to put the light out. For more information on the clplamp command, see "Turning off warning light (clplamp command)" in "8.2. EXPRESSCLUSTER commands" in this guide.

For a network warning light (specified by NEC) that supports playback of an audio file, the setting also enables audio file reproduction to link to On/Off.

7.6.5. Operations of SNMP trap sending

7.7. SNMP linkage

7.7.1. SNMP linkage

SNMP linkage enables SNMP trap sending from EXPRESSCLUSTER and information acquisition by SNMP from an SNMP manager according to the EXPRESSCLUSTER MIB definitions.

7.7.2. EXPRESSCLUSTER MIB definitions

The information sent/acquired with SNMP linkage is configured by the MIB definition files.

To use the functions of SNMP trap sending and information acquisition by SNMP, described later, MIB definition files are required.

To receive SNMP traps from EXPRESSCLUSTER by using an SNMP manager, or to acquire cluster statuses from an SNMP manager, set the EXPRESSCLUSTER MIB definition files in the SNMP manager.

For how to set the MIB definition files in an SNMP manager, refer to the manual for the SNMP manager.

The EXPRESSCLUSTER MIB definition files are placed in the following directory on the EXPRESSCLUSTER X CD-ROM.

<EXPRESSCLUSTER_X_CD-ROM>\Common\<version number>\common\mib

The MIB definition files provide the functions described below.

No.

MIB definition file

Description

NEC-CLUSTER-SMI.mib

Configures the EXPRESSCLUSTER MIB tree root path.

NEC-CLUSTER-EVENT-MIB.mib

Configures the trap and MIB definitions for the EXPRESSCLUSTER SNMP trap sending function.

NEC-CLUSTER-MANAGEMENT-MIB.mib
Configures MIB definitions for the following EXPRESSCLUSTER information:
- Cluster information
- Server information
- Group information

The available functions depend on the files set in the SNMP manager.

To receive SNMP traps from EXPRESSCLUSTER:

1. NEC-CLUSTER-SMI.mib
2. NEC-CLUSTER-EVENT-MIB.mib

To acquire information by SNMP:

1. NEC-CLUSTER-SMI.mib
3. NEC-CLUSTER-MANAGEMENT-MIB.mib

7.7.3. SNMP trap sending

SNMP trap sending serves to send the contents of Cluster WebUI alert messages to the SNMP manager.

To send a trap, the SNMP trap sending destination is required to be configured. Configure it by referring to Destination Settings of SNMP Trap in "Alert Service tab" in "Cluster properties" in "2. Parameter details" in this guide.

The traps to be sent are defined by NEC-CLUSTER-EVENT-MIB.

NEC-CLUSTER-EVENT-MIB defines the following MIB objects.

clusterEventNotifications group

This group defines the traps to be sent. The MIB objects defined for the group function as described below.

No.

SNMP TRAP OID

Description

clusterEventInformation
Trap for information level alerts.
A clusterEvent group MIB object is attached.

clusterEventWarning
Trap for warning level alerts.
A clusterEvent group MIB object is attached.

clusterEventError
Trap for error level alerts.
A clusterEvent group MIB object is attached.

clusterEvent group

This group defines the information appended to the traps. The MIB objects defined for the group function as described below.

No.

SNMP OID

Description

clusterEventMessage

Indicates the alert message.

clusterEventID

Indicates the event ID.

clusterEventDateTime

Indicates the time at which the alert originated.

clusterEventServerName

Indicates the server from which the alert originated.

clusterEventModuleName

Indicates the module from which the alert originated.

7.7.4. Information acquisition by SNMP

By using the SNMP protocol, some information about the EXPRESSCLUSTER configuration and status can be acquired. However, EXPRESSCLUSTER does not include SNMP agent functions. For an SNMP agent, the Net-SNMP snmpd daemon needs to be implemented separately.

SNMP agent

The SNMP agent serves to return a response about the configuration information or status information (GetResponse) to information acquisition requests (GetRequest, GetNextRequest) from an SNMP manager (network management software).

Note

To use information acquisition by SNMP, you must take the steps described in "Setting up the SNMP linkage function" in the "Installation and Configuration Guide".

7.7.5. MIB objects acquirable with SNMP linkage

The MIB objects that can be acquired with the SNMP linkage function are defined by NEC-CLUSTER-MANAGEMENT-MIB.

NEC-CLUSTER-MANAGEMENT-MIB defines the following MIB objects.

clusterGeneral group

This group is used to acquire cluster information. The MIB objects defined for the group function as described below.

No.

SNMP OID

Description

clusterName

Indicates the name of the cluster.

clusterComment

Indicates the comment of the cluster.

clusterStatus

Indicates the current statust of the cluster.

The correspondence between the MIB value and the Cluster WebUI status is as described below.

MIB value status
---------------------
normal    Normal
caution   Caution
error     Error
unknown    -

clusterServer group

This group is used to acquire server information. Indexes on acquisition of clusterServerTable are sorted by server priority. The MIB objects defined for the group function as described below.

No.

SNMP OID

Description

clusterServerLocalServerIndex

Indicates the index of the server receiving the present SNMP information acquisition request (clusterServerIndex).

clusterServerTable

Indicates the information table for the server.

clusterServerEntry

Indicates the server information list.

The index for the list is clusterServerIndex.

clusterServerIndex

Indicates the index for uniquely identifying the server.

clusterServerName

Indicates the name of the server.

clusterServerComment

Indicates a comment for the server.

clusterServerStatus

Indicates the current status of the server.

The correspondence between the MIB value and the Cluster WebUI status is as described below.

MIB value      Group type
------------------------------
failover       Failover group
cluster        Management group
virtualMachine Virtual machine group

Values other than those indicated above may be acquired depending on the status of the server.

clusterServerPriority

Indicates the priority of the server.

clusterServerProductName

Indicates the name of the EXPRESSCLUSTER product installed on the server.

clusterServerProductVersion

Indicates the version of the EXPRESSCLUSTER product installed on the server.

clusterServerProductInstallPath

Indicates the installation path of EXPRESSCLUSTER on the server.

clusterServerPlatformName

Indicates the name of the platform on the server.

clusterGroup group

This group is used to acquire group information. The MIB objects defined for the group function as described below.

No.

SNMP OID

Description

clusterGroupTable

Indicates the information table for the group.

clusterGroupEntry

Indicates the group information list.

The index for the list is clusterGroupIndex.

clusterGroupIndex

Indicates the index for uniquely identifying the group.

clusterGroupName

Indicates the name of the group.

clusterGroupComment

Indicates a comment for the group.

clusterGroupType

Indicates the type of the group.

The correspondence between the MIB value and the group type is as described below.

MIB value         Group type
---------------------------------------
failover          Failover group
cluster           Management group
virtualMachine    Virtual machine group

clusterGroupStatus

Indicates the current status of the group.

The correspondence between the MIB value and the Cluster WebUI status is as described below.

MIB value        status
---------------------------
online           Online
onlineFailure    Online Failure
offlineFailure   Offline Failure
offline          Offline
unknown          Unknown
onlinePending    Online Pending
offlinePending   Offline Pending

clusterGroupCurrentServerIndex

Indicates the index of the server on which the group is currently active (clusterServerIndex).

The return value of a deactivated group is -1.

7.8. Cluster service automatic startup prohibition after improper stop

7.8.1. Cluster service automatic startup prohibition

This function prohibits the EXPRESSCLUSTER service from automatically starting up at the next OS activation after the cluster has been shut down, reboot, or stopped by Cluster WebUI or the EXPRESSCLUSTER service has been stopped by using a command other than the clpstdn command and the clpcl -t -a command.

When the automatic startup prohibition setting is enabled, the EXPRESSCLUSTER service will not automatically start at the next server activation after the cluster has been shut down, reboot, or stopped by Cluster WebUI or the EXPRESSCLUSTER service has been stopped by using a command other than the clpstdn command and the clpcl -t -a command.

Even in cases where cluster shutdown or cluster stop is executed, if an error occurs in the EXPRESSCLUSTER service stop sequence, or if the stop sequence is not executed due to the likes of an OS reset or a power interruption, the EXPRESSCLUSTER service will not automatically start at the next OS activation.

7.8.2. Displaying and changing the automatic startup prohibition setting

  • Cluster service's not stop normal, prohibit automatic startup
    Prohibits cluster service automatic startup at the next OS activation if the servers are stopped by a means other than cluster shutdown or cluster stop, or if the cluster shutdown or stop sequence does not finish successfully.
  • Not prohibit cluster service automatic startup after improper stop
    Does not prohibit cluster service automatic startup.

7.8.3. Conditions for automatic startup prohibition

The conditions for automatic startup prohibition are as described below.

  • The cluster is stopped by a means other than cluster shutdown or cluster stop.

  • The cluster service stop sequence is not executed due to a reason such as an OS reset, panic, or power interruption.

  • Group deactivation fails in the cluster service stop sequence as a result of cluster shutdown or stop.

  • The cluster is stopped on one of the servers comprising the cluster.

7.8.4. Notes on automatic startup prohibition

  • At OS activation, if the EXPRESSCLUSTER service does not start automatically, activate the EXPRESSCLUSTER service by using Cluster WebUI or the clpcl command.

  • At OS activation, if the EXPRESSCLUSTER service does not start automatically, Cluster WebUI alert messages and syslog messages are output.

7.9. Grace period dependence at the automatic failover between server groups

7.9.1. What is the grace period dependence?

One server group waits specified time for the other server group to start failover when the automatic failover is executed between server groups. When the grace period elapsed after the server down was detected, the failover is executed.

7.9.2. Condition for the grace period dependence

One server group waits for the other server group with any of the following configurations to start the failover.

  • Use Server Group settings in the Info tab is selected.

  • Multiple server groups are specified for Server Groups that can run the Group in the Startup Server tab.

  • Prioritize server group failover policy is selected and Enable only manual failover among the server group is not selected for Auto Failover of Failover Attribute in the Attribute tab.

In the following cases, one server group does not wait specified time for the other server group to start failover:

  • One server executes the failover to another server within the same server group.

  • The server down is detected by the server down notification.

  • The script for forced stop is successfully executed while Execute Script for Forced Stop is selected, or the condition not to execute the script for forced stop is met.

  • The forced stop is successfully executed while Execute Script for Forced Stop is not selected and Use Forced Stop is selected, or the condition not to execute the forced stop is met.

  • The NP resolution resource is configured.

7.9.3. Displaying and changing the grace period dependence

Specify the waiting time for Grace period of server group failover policy.

If 0 is specified, one server group does not wait for the other server group to start failover.

7.9.4. Notes on the grace period dependence

If any operation is done for the failover target group while the other server group waits during the grace period, the settings to wait during the grace period is cancelled and the other server group does not failover.

If the once-failed server is detected to be alive while the other server waits during the grace period, the settings to wait during the grace period is cancelled and the failover is not executed.

If the failover target server goes down, the failover may start later than when the grace period ends.

7.10. Witness server service

7.10.1. What is Witness server service?

Witness service is the service to receive Witness heartbeat from each server in the cluster and send back the status information of receiving the heartbeat from each server as a response. It is installed in a server outside of the cluster.

7.10.2. Notes on Witness server service

  • Witness server service operates in Node.js environment. Therefore, Node.js needs to be installed before the installation of the Witness server service.

7.10.3. Operation verified environment for Witness server service

Its operation has been verified in the following environments.

OS

Requirement

Version

Windows Server 2012 R2

Node.js 10.13.0

4.1.0

Red Hat Enterprise Linux 7 update4

Node.js 8.12.0

4.1.0

7.10.4. How to install Witness server service

Install the Witness server service by using npm command for Node.js environment. Store the Witness server service module in an arbitrary folder, and execute the following command.

> npm install --global clpwitnessd-<version>.tgz

7.10.5. How to configure Witness server service

To change the settings of Witness server service, edit the configuration file directly. Open the folder indicated in the first row of the execution results of the command below.

> npm list --global clpwitnessd

Example of execution results:

C:\Users\Administrator\AppData\Roaming\npm
`-- clpwitnessd@4.1.0

Edit clpwitnessd.conf.js that is stored in node_modules\clpwitnessd under the opened folder, with a text editor such as notepad.

Setting items are as follows.

Item

Default

Description

http.enable

True
Specify whether to execute HTTP server or not.
true: execute
false: not execute

http.port

80

Specify the wait port number for HTTP server.

http.keepalive

10000

Specify the keep alive time for HTTP server in milliseconds.

https.enable

False
Specify whether to execute HTTPS server or not.
true: execute
false: not execute

https.port

443

Specify the wait port number for HTTPS server.

https.keepalive

10000

Specify the keep alive time for HTTPS server in milliseconds.

https.ssl.key

server_key.pem

Specify a secret key file to be used for HTTPS server.

https.ssl.crt

server_crt.pem

Specify a certification file to be used for HTTPS server.

log.directory

-

Specify the log output destination folder.

log.level

info
Specify the log output level.
error: Only error logs are output.
warn: Error logs and warning logs are output.
info: Warning logs and information logs are output.
debug: Information logs and detailed logs are output.

log.size

1024 * 1024 * 512

Specify the log rotation size in bytes.

data.available

10000

Specify the default time limit for the communication status information of the cluster server in milliseconds.

7.10.6. How to execute Witness server service

Excute the following command to start up Witness server service in the fore ground. For how to execute the Witness server service as Windows service or Linux daemon, refer to the following section, "Using Witness server service as the OS service".

> clpwitnessd

7.10.7. Using Witness server service as the OS service

If you want to start Witness server service at the OS startup, the Witness server service requires to be registered as the OS service.

The following exemplifies how to register Witness server service as the OS service (in case of Windows service control manager and Linux systemd). The method of registration for the OS service differs depending on the environment. Configure the registration to suit your environment by referring to the explanation below.

Registration for Windows service control manger

The following exemplifies the procedure to register by using npm package winser.

  1. Install winser by npm command. Use the following command so that winser package is downloaded from npm repository and then installed.

    > npm install --global winser

  2. Create a folder to execute the service in any location. By default, this folder stores log files, SSL secret key file and SSL certificate file.

  3. Create package.json file for the service registration with winser, under the folder created in the above step 2. Enter " \\ " to separate the characters of the path. The path specified for "start" is line-fed for the convenience of character numbers but actually is in one row.

  4. Execute winser command to register and start the Witness server service.

> winser -i -a

  1. Select Control Panel -> Administration Tools -> Service, and confirm that the service (ex. clpwitnessd-service) with the name specified for "name" of pacage.json has been registered.

Registration for Linux systemd

The following exemplifies the procedure to register by creating the unit file of systemd.

  1. Create a directory to execute the service in any location. By default, this folder stores log files, SSL secret key file and SSL certificate file.
    (ex. /opt/clpwitnessd) (ex. /opt/clpwitnessd)
  2. Create the unit file of the Witness server service in /etc/systemd/system.
    (ex. clpwitnessd.service) (ex. clpwitnessd.service)

  3. Execute systemctl command to register and start the Witness server service.

    # systemctl enable clpwitnessd

    # systemctl start clpwitnessd