1. Preface¶

1.1. Who Should Use This Guide¶

The EXPRESSCLUSTER® X SingleServerSafe Operation Guide is intended for system administrators who will operate and maintain an introduced system.

1.2. How This Guide Is Organized¶

2. EXPRESSCLUSTER X SingleServerSafe command reference: Provides information on commands available to use in EXPRESSCLUSTER.
3. Notes and restrictions: Provides information on known problems and restrictions.
4. Error messages: Lists and describes error messages you might encounter when operating EXPRESSCLUSTER X SingleServerSafe.

1.3. Terms Used in This Guide¶

EXPRESSCLUSTER X SingleServerSafe, which is described in this guide, uses windows and commands common to those of the clustering software EXPRESSCLUSTER X to ensure high compatibility with EXPRESSCLUSTER X in terms of operation and other aspects. Therefore, cluster-related terms are used in parts of the guide.

The terms used in this guide are defined below.

Cluster, cluster system: A single server system using EXPRESSCLUSTER X SingleServerSafe
Cluster shutdown, reboot: Shutdown or reboot of a system using EXPRESSCLUSTER X SingleServerSafe
Cluster resource: A resource used in EXPRESSCLUSTER X SingleServerSafe
Cluster object: A resource object used in EXPRESSCLUSTER X SingleServerSafe
Failover group: A group of group resources (such as applications and services) used in EXPRESSCLUSTER X SingleServerSafe

1.4. EXPRESSCLUSTER X SingleServerSafe Documentation Set¶

The EXPRESSCLUSTER X SingleServerSafe documentation consists of the three guides below. The title and purpose of each guide is described below:

EXPRESSCLUSTER X SingleServerSafe Installation Guide

This guide is intended for system engineers who intend to introduce a system using EXPRESSCLUSTER X SingleServerSafe and describes how to install EXPRESSCLUSTER X SingleServerSafe.

EXPRESSCLUSTER X SingleServerSafe Configuration Guide

This guide is intended for system engineers who intend to introduce a system using EXPRESSCLUSTER X SingleServerSafe and system administrators who will operate and maintain the introduced system. It describes how to set up EXPRESSCLUSTER X SingleServerSafe.

EXPRESSCLUSTER X SingleServerSafe Operation Guide

This guide is intended for system administrators who will operate and maintain an introduced system that uses EXPRESSCLUSTER X SingleServerSafe. It describes how to operate EXPRESSCLUSTER X SingleServerSafe.

1.5. Conventions¶

In this guide, Note, Important, and See also are used as follows:

Note

Used when the information given is important, but not related to the data loss and damage to the system and machine.

Important

Used when the information given is necessary to avoid the data loss and damage to the system and machine.

1.6. Contacting NEC¶

For the latest product information, visit our website below:

https://www.nec.com/global/prod/expresscluster/

2. EXPRESSCLUSTER X SingleServerSafe command reference¶

This chapter describes the commands available with EXPRESSCLUSTER X SingleServerSafe.

EXPRESSCLUSTER X SingleServerSafe uses commands common to those of the clustering software EXPRESSCLUSTER X to ensure high compatibility with EXPRESSCLUSTER X in terms of operation and other aspects.

This chapter covers:

2.1. Operating the cluster from the command line
2.2. EXPRESSCLUSTER commands
2.3. Displaying the status (clpstat command)
2.4. Operating the EXPRESSCLUSTER daemon (clpcl command)
2.5. Shutting down the server (clpstdn command)
2.6. Operating groups (clpgrp command)
2.7. Collecting logs (clplogcc command)
2.8. Applying and backing up configuration data (clpcfctrl command)
2.9. Adjusting time-out temporarily (clptoratio command)
2.10. Modifying the log level and size (clplogcf command)
2.11. Managing licenses (clplcnsc command)
2.12. Outputting messages (clplogcmd command)
2.13. Controlling monitor resources (clpmonctrl command)
2.14. Controlling group resources (clprsc command)
2.15. Requesting processing to cluster servers (clprexec command)
2.16. Controlling reboot count (clpregctrl command)
2.17. Checking the process health (clphealthchk command)
2.18. Displaying the cluster statistics information (clpperfc command)
2.19. Checking the cluster configuration information (clpcfchk command)
2.20. Adding a firewall rule (clpfwctrl command)

2.1. Operating the cluster from the command line¶

EXPRESSCLUSTER X SingleServerSafe provides various commands for performing operations from the command prompt. These commands are useful in such cases as when you are setting up a cluster or cannot use the Cluster WebUI. You can perform a greater number of operations by using the command line than by using the Cluster WebUI.

Note

If the monitor resource detects an error when you have specified a group resource (such as an application resource) as a recovery target in the settings for error detection by a monitor resource, do not perform the following control operations for any service or group by using a command or the Cluster WebUI during recovery (reactivation -> final action).

Stopping or suspending a service
Starting or stopping a group

If you perform the above-mentioned operations while recovery caused by detection of an error by a monitor resource is in progress, other group resources of the group with an error may not stop.

However, you can perform them when the final action is completed.

Important

The installation directory contains executable-format files and script files that are not listed in this guide. Do not execute these files by programs or applications other than EXPRESSCLUSTER. Any problems caused by not using EXPRESSCLUSTER will not be supported.

2.2. EXPRESSCLUSTER commands¶

Commands for construction

command	Explanation	Refer to
clpcfctrl	Delivers the configuration data created by the Cluster WebUI to servers. Backs up the configuration data to be used by the Cluster WebUI.	2.8. Applying and backing up configuration data (clpcfctrl command)
clplcnsc	Manages the product or trial version license of this product.	2.11. Managing licenses (clplcnsc command)
clpcfchk	Checks cluster configuration data.	2.19. Checking the cluster configuration information (clpcfchk command)
clpfwctrl.sh	Adds a firewall rule.	2.20. Adding a firewall rule (clpfwctrl command)

Commands for showing status

command	Explanation	Refer to
clpstat	Displays the status and configuration data of EXPRESSCLUSTER X SingleServerSafe.	2.3. Displaying the status (clpstat command)
clphealthchk	Check the process health.	2.17. Checking the process health (clphealthchk command)

Commands for operation

command	Explanation	Refer to
clpcl	Starts, stops, suspends, or resumes the daemon.	2.4. Operating the EXPRESSCLUSTER daemon (clpcl command)
clpstdn	Stops and shuts down the EXPRESSCLUSTER daemon.	2.5. Shutting down the server (clpstdn command)
clpgrp	Starts and stops groups.	2.6. Operating groups (clpgrp command)
clptoratio	Extends or displays thetimeout values.	2.9. Adjusting time-out temporarily (clptoratio command)
clpmonctrl	Suspends and/or resumes monitor resources on a server.	2.13. Controlling monitor resources (clpmonctrl command)
clpregctrl	Displays and/or initializes reboot count on a single server.	2.16. Controlling reboot count (clpregctrl command)
clprsc	Suspends or resumes group resources.	2.14. Controlling group resources (clprsc command)
clprexec	Requests that an EXPRESSCLUSTER server execute a process from external monitoring.	2.15. Requesting processing to cluster servers (clprexec command)

Commands for logs

command	Explanation	Refer to
clplogcc	Collects logs and OS information.	2.7. Collecting logs (clplogcc command)
clplogcf	Modifies and displays log level and log output file size.	2.10. Modifying the log level and size (clplogcf command)
clpperfc	Displays cluster statistical information on a group or a monitor resource.	2.18. Displaying the cluster statistics information (clpperfc command)

Script-related commands

command

Explanation

Refer to

clplogcmd

Write this command in the EXEC resource script to output messages to any destination.

2.12. Outputting messages (clplogcmd command)

Important

The installation directory contains executable files and script files that are not listed in this guide. Do not execute these files by using any program other than EXPRESSCLUSTER X SingleServerSafe. Any problems caused by not using EXPRESSCLUSTER will not be supported.

2.3. Displaying the status (clpstat command)¶

Displays the status and configuration data of EXPRESSCLUSTER X SingleServerSafe.

Command line: clpstat: -s [--long]

clpstat: -g

clpstat: -m

clpstat: -i [--detail]

clpstat: --cl [--detail]

clpstat: --sv [--detail]

clpstat: --grp [group_name] [--detail]

clpstat: --rsc [resource_name] [--detail]

clpstat: --mon [monitor_name] [--detail]

Description: Displays the server status and configuration information.

Option

-s¶

None¶: Displays the status.

--long¶: Displays a name of the cluster name and resource name until the end.

-g¶: Displays a group map.

-m¶: Displays the status of each monitor resource.

-i¶: Displays the configuration data.

--cl¶: Displays the configuration data.

--sv¶: Displays the server configuration information.

--grp [group_name]¶: Displays server group configuration information. By specifying the name of a server group, you can display only the information on the specified server group.

--rsc [resource_name]¶: Displays group resource configuration information. By specifying the name of a group resource, you can display only the information on the specified group resource.

--mon [monitor_name]¶: Displays monitor resource configuration information. By specifying the name of a monitor resource, you can display only the information on the specified monitor resource.

--detail¶: Displays more detailed information on the setting.

Return Value

0	Success
Other than the above	Failure

Notes: This command must be executed by a user with the root privilege.

For the language used for this command output, see "Info tab" of "Cluster properties" in "Details of other settings" in "EXPRESSCLUSTER X SingleServerSafe Configuration Guide".

When you run the clpstat command with the -s option or without any option, names such as a server name and a resource name are displayed only partway.

Error Messages

Message	Cause/Solution
Log in as root.	Log on as root user.
Invalid configuration file. Create valid cluster configuration data.	Create valid cluster configuration data by using the Cluster WebUI.
Invalid option.	Specify a valid option.
Could not connect to the server. Check if the cluster daemon is active.	Check if the cluster daemon is started.
Invalid server status.	Check if the cluster daemon is started.
Server is not active. Check if the cluster daemon is active.	Check if the cluster daemon is started.
Invalid server name. Specify a valid server name in the cluster.	Specify the valid name of a server in the cluster.
Invalid heartbeat resource name. Specify a valid heartbeat resource name in the cluster.	Specify the valid name of a heartbeat resource in the cluster.
Invalid network partition resource name. Specify a valid network partition resource name in the cluster.	Specify the valid name of a network partition resolution resource in the cluster.
Invalid group name. Specify a valid group name in the cluster.	Specify the valid name of a group in the cluster.
Invalid group resource name. Specify a valid group resource name in the cluster.	Specify the valid name of a group resource in the cluster.
Invalid monitor resource name. Specify a valid monitor resource name in the cluster.	Specify the valid name of a monitor resource in the cluster.
Connection was lost. Check if there is a server where the cluster daemon is stopped in the cluster.	Check if there is any server on which the cluster daemon has stopped in the cluster.
Invalid parameter.	The value specified as a command parameter may be invalid.
Internal communication timeout has occurred in the cluster server. If it occurs frequently, set the longer timeout.	A time-out occurred in the EXPRESSCLUSTER internal communication. If time-out keeps occurring, set the internal communication time-out longer.
Internal error. Check if memory or OS resources are sufficient.	Check to see if the memory or OS resource is sufficient.
Invalid server group name. Specify a valid server group name in the cluster.	Specify the correct server group name in the cluster.
The cluster is not created.	Create and apply the cluster configuration data.
Could not connect to the server. Internal error. Check if memory or OS resources are sufficient.	Check to see if the memory or OS resource is sufficient.
Cluster is stopped. Check if the cluster daemon is active.	Check if the cluster daemon is started.
Cluster is suspended. To display the cluster status, use --local option.	Cluster is suspended. To display the cluster status, use --local option.

2.4. Operating the EXPRESSCLUSTER daemon (clpcl command)¶

Operates the EXPRESSCLUSTER daemon.

Command line: clpcl -s

clpcl -t [-w timeout] [--apito timeout]

clpcl -r [-w timeout] [--apito timeout]

clpcl --suspend [--force] [-w timeout] [--apito timeout]

clpcl --resume

Description: This command starts, stops, suspends, or resumes the EXPRESSCLUSTER daemon.

Option

-s¶: Starts the EXPRESSCLUSTER daemon.

-t¶: Stops the EXPRESSCLUSTER daemon.

-r¶: Restarts the EXPRESSCLUSTER daemon.

--suspend¶: Suspends the EXPRESSCLUSTER daemon.

--resume¶: Resumes the EXPRESSCLUSTER daemon.

-w timeout¶: Specifies the wait time to stop or suspend the cluster daemon to be completed when -t, -r, or --suspend option is used.

The unit of time is second.

When a timeout is not specified, it waits for unlimited time.

When "0" is specified in timeout, it does not wait for the completion of stop or suspension of the EXPRESSCLUSTER daemon.

When the -w option is not specified, it waits for the completion of stop or suspension of the EXPRESSCLUSTER daemon for (heartbeat timeout x 2) (seconds).

--force¶: When used with the --suspend option, this option forcefully suspends the service regardless of the server status.

--apito timeout¶: Specify the interval (internal communication timeout) to wait for the EXPRESSCLUSTER daemon start or stop in seconds. A value from 1 to 9999 can be specified.

If the --apito option is not specified, waiting for the EXPRESSCLUSTER daemon start or stop is performed according to the value set to the internal communication timeout of the cluster properties.

Return Value

0	Success
Other than 0	Failure

Remarks: When this command is executed with the -s or --resume option specified, it returns control when processing starts on the target server.

When this command is executed with the -t or --suspend option specified, it returns control after waiting for the processing to complete.

When this command is executed with the -r option specified, it returns control when the EXPRESSCLUSTER daemon restarts on the target server after stopping once.

Run the clpstat command to display the started or resumed status of the EXPRESSCLUSTER daemon.

Notes: This command must be executed by a user with the root privilege.

This command cannot be executed while a group is being started or stopped.

Execute the --suspend option when the EXPRESSCLUSTER daemon is active. The --force option forcibly suspends the EXPRESSCLUSTER daemon.

When executing the --resume option, make sure that the EXPRESSCLUSTER daemon is not running by the clpstat command.

Example

Example 1: Activating the EXPRESSCLUSTER daemon in the server

# clpcl -s

Suspend and Resume

When you want to update configuration data or EXPRESSCLUSTER, you can stop the EXPRESSCLUSTER daemon while continuing the operation. This status is called "suspend." Returning from the suspended status to normal status is called "resume."

The suspend and resume operations request processing of the server. The EXPRESSCLUSTER daemon of the server must be active when you execute a suspend operation.

The following functions stop when the cluster is suspended because the cluster daemon stops while active resources stay active.
- All monitor resources stop.
- You cannot perform operations on groups or group resources (start/stop).
- The following commands are disabled;
  - clpcl options other than --resume
  - clpstdn
  - clpgrp
  - clptoratio
  - clpmonctrl

Error Messages

Message	Cause/Solution
Log in as root.	Log on as root user.
Invalid configuration file. Create valid cluster configuration data .	Create valid cluster configuration data using the Cluster WebUI.
Invalid option.	Specify a valid option.
Performed stop processing to the stopped cluster daemon.	The stopping process has been executed on the stopped cluster daemon.
Performed startup processing to the active cluster daemon.	The startup process has been executed on the activated cluster daemon.
Could not connect to the server. Check if the cluster daemon is active.	Check if the cluster daemon is started.
Could not connect to the data transfer server. Check if the server has started up.	Check if the server is running.
Failed to obtain the list of nodes. Specify a valid server name in the cluster.	Specify the valid name of a server in the cluster.
Failed to obtain the daemon name.	Failed to obtain the cluster name.
Failed to operate the daemon.	Failed to control the cluster.
Resumed the daemon that is not suspended.	Performed the resume process for the HA Cluster daemon that is not suspended.
Invalid server status.	Check that the cluster daemon is started.
Server is busy. Check if this command is already run.	This command may have already been run.
Server is not active. Check if the cluster daemon is active.	Check if the cluster daemon is started.
There is one or more servers of which cluster daemon is active. If you want to perform resume, check if there is any server whose cluster daemon is active in the cluster.	When you execute the command to resume, check if there is no server in the cluster on which the cluster daemon is started.
All servers must be activated. When suspending the server, the cluster daemon need to be active on all servers in the cluster.	When you execute the command to suspend, the cluster daemon must be started in all servers in the cluster.
Resume the server because there is one or more suspended servers in the cluster.	Execute the command to resume because some server(s) in the cluster is in the suspend status.
Invalid server name. Specify a valid server name in the cluster.	Specify the valid name of a sever in the cluster.
Connection was lost. Check if there is a server where the cluster daemon is stopped in the cluster.	Check if there is any server on which the cluster daemon is stopped in the cluster.
Invalid parameter.	The value specified as a command parameter may be invalid.
Internal communication timeout has occurred in the cluster server. If it occurs frequently, set the longer timeout.	A time-out occurred in the HA Cluster internal communication. If time-out keeps occurring, set the internal communication time-out longer.
Processing failed on some servers. Check the status of failed servers.	If stopping has been executed with all the servers specified, there is one of more server on which the stopping process has failed. Check the status of the server(s) on which the stopping process has failed.
Internal error. Check if memory or OS resources are sufficient.	Check to see if the memory or OS resource is sufficient.
There is a server that is not suspended in cluster. Check the status of each server.	There is a server that is not suspended in the cluster. Check the status of each server.
Suspend %s : Could not suspend in time.	The server failed to complete the suspending process of the cluster daemon within the time-out period. Check the status of the server.
Stop %s : Could not stop in time.	The server failed to complete the stopping process of the cluster daemon within the time-out period. Check the status of the server.
Stop %s : Server was suspended. Could not connect to the server. Check if the cluster daemon is active.	The request to stop the cluster daemon was made. However the server was suspended.
Could not connect to the server. Check if the cluster daemon is active.	The request to stop the cluster daemon was made. However connecting to the server failed. Check the status of the server.
Suspend %s : Server already suspended. Could not connect to the server. Check if the cluster daemon is active.	The request to suspend the cluster daemon was made. However the server was suspended.
Event service is not started.	Event service is not started. Check it.
Mirror Agent is not started.	Mirror Agent is not started. Check it.
Event service and Mirror Agent are not started.	Event service and Mirror Agent are not started. Check them.
Some invalid status. Check the status of cluster.	The status of a group may be changing. Try again after the status change of the group is complete.

2.5. Shutting down the server (clpstdn command)¶

Shuts down the server.

Command line: clpstdn [-r]

Description: Stops and shuts down the EXPRESSCLUSTER daemon of the server.

Option

None¶: Shuts down the server.

-r¶: Shuts down and reboots the server.

Return Value

0	Success
Other than 0	Failure

Remarks: This command returns control when the group stop processing is completed.

Notes: This command must be executed by a user with the root privilege.

This command cannot be executed while a group is being started or stopped.

Examples

Example 1: Shutting down the server

# clpstdn

Example 2: Shutting down and rebooting the server

# clpstdn -r

2.6. Operating groups (clpgrp command)¶

Operates groups.

Command line: clpgrp -s [group_name] [--apito timeout]

clpgrp -t [group_name] [--apito timeout]

Description: Starts and stops groups.

Option

-s [group_name]¶: Starts a group. When you specify the name of a group, only the specified group starts up. If no group name is specified, all groups start up.

-t [group_name]¶: Stops a group. When you specify the name of a group, only the specified group stops. If no group name is specified, all groups stop.

--apito timeout¶: Specify the interval (internal communication timeout) to wait for the group resource start or stop in seconds. A value from 1 to 9999 can be specified.

If the --apito option is not specified, waiting for the group resource start or stop is performed according to the value set to the internal communication timeout of the cluster properties.

Return Value

0	Success
Other than 0	Failure

Notes: This command must be executed by a user with the root privilege.

The EXPRESSCLUSTER daemon must be started on the server that runs this command.

Examples

The following is a simple example of group operation.

The server has groupA.

Run the following command on the server. Then groupA starts.
# clpgrp -s groupA
Fig. 2.1 Group startup¶

Run the following command on the server. Then groupA stops.
# clpgrp -t groupA
Fig. 2.2 Group stop¶

Error message

Message	Cause/Solution
Log in as root.	Log on as root user.
Invalid configuration file. Create valid cluster configuration data.	Create valid cluster configuration data using the Cluster WebUI.
Invalid option.	Specify a valid option.
Could not connect to the server. Check if the cluster daemon is active.	Check if the cluster daemon is started.
Invalid server status.	Check if the cluster daemon is started.
Server is not active. Check if the cluster daemon is active.	Check if the cluster daemon is started.
Invalid server name. Specify a valid server name in the cluster.	Specify the valid name of sever in the cluster.
Connection was lost. Check if there is a server where the cluster daemon is stopped in the cluster.	Check if there is any server on which the cluster daemon has stopped in the cluster.
Invalid parameter.	The value specified as a command parameter may be invalid.
Internal communication timeout has occurred in the cluster server. If it occurs frequently, set the longer timeout.	A time-out occurred in the EXPRESSCLUSTER internal communication. If time-out keeps occurring, set the internal communication time-out longer.
Invalid server. Specify a server that can run and stop the group, or a server that can be a target when you move the group.	The server that starts/stops the group or to which the group is moved is invalid. Specify a valid server.
Could not start the group. Try it again after the other server is started, or after the Wait Synchronization time is timed out.	Start up the group after waiting for the remote server to start up, or after waiting for the time-out of the start-up wait time.
No operable group exists in the server.	Check if there is any group that is operable in the server which requested the process.
The group has already been started on the local server.	Check the status of the group by using the Cluster WebUI or the clpstat command.
The group has already been started on the other server. To start/stop the group on the local server, use -f option.	Check the status of the group by using the Cluster WebUI or the clpstat command. If you want to start up or stop a group which was started in a remote server from the local server, move the group or run the command with the -f option.
The group has already been started on the other server. To move the group, use "-h <hostname>" option.	Check the status of the group by using the Cluster WebUI or clpstat command. If you want to move a group which was started on a remote server, run the command with the "-h <hostname>" option.
The group has already been stopped.	Check the status of the group by using the Cluster WebUI or the clpstat command.
Failed to start one or more group resources. Check the status of group	Check the status of group by using Cluster WebUI or the clpstat command.
Failed to stop one or more group resources. Check the status of group	Check the status of group by using the Cluster WebUI or the clpstat command.
The group is busy. Try again later.	Wait for a while and then try again because the group is now being started up or stopped.
An error occurred on one or more groups. Check the status of group	Check the status of the group by using the Cluster WebUI or the clpstat command.
Invalid group name. Specify a valid group name in the cluster.	Specify the valid name of a group in the cluster.
Server is not in a condition to start group or any critical monitor error is detected.	Check the status of the server by using the Cluster WebUI or clpstat command. An error is detected in a critical monitor on the server on which an attempt was made to start a group.
There is no appropriate destination for the group. Other servers are not in a condition to start group or any critical monitor error is detected.	Check the status of the server by using the Cluster WebUI or clpstat command. An error is detected in a critical monitor on all other servers.
The group has been started on the other server. To migrate the group, use "-h <hostname>" option.	Check the status of the group by using the Cluster WebUI or clpstat command. If you want to move a group which was started on a remote server, run the command with the "-h <hostname>" option.
Some invalid status. Check the status of cluster.	Invalid status for some sort of reason. Check the status of the cluster.
Internal error. Check if memory or OS resources are sufficient.	Check to see if the memory or OS resource is sufficient.

2.7. Collecting logs (clplogcc command)¶

Collects logs.

Command line: clplogcc [-t collect_type] [-r syslog_rotate_number] [-o path]

Description: Collects information including logs and the OS information by accessing the data transfer server.

Option

None¶: Logs are collected.

-t collect_type¶: Specifies a log collection pattern. When this option is omitted, a log collection pattern will be type1.

-r syslog_rotate_number¶: Specifies how many generations of syslog will be collected. When this option is omitted, two generations will be collected.

-o path¶: Specifies the output destination of collector files. When this option is skipped, logs are output under tmp of the installation path.

Return Value

0	Success
Other than 0	Failure

Remarks: Since log files are compressed by tar.gz, add the xzf option to the tar command to decompress them.

Notes: This command must be executed by a user with the root privilege.

Examples

Example 1: Collecting logs from the server

# clplogcc
Collect Log server1 : Success

Log collection results (server status) of servers on which log collection is executed are displayed.

Process servername: Result (server status)

Execution Result

For this command, the following processes are displayed.

Steps in Process	Explanation
Connect	Displayed when the access fails.
Get Filesize	Displayed when acquiring the file size fails.
Collect Log	Displayed with the file acquisition result.

The following results (server status) are displayed:

Result (server status)	Explanation
Success	Log collection succeeded.
Timeout	Timeout occurred.
Busy	The server is busy.
Not Exist File	The file does not exist.
No Freespace	No free space on the disk.
Failed	Failure caused by other errors.

Error Message

Message

Cause/Solution

Log in as root.

Log in as a root user.

Invalid configuration file. Create valid cluster configuration data.

Create valid configuration data by using the Cluster WebUI.

Invalid option.

Specify the correct option.

Specify a number in a valid range.

Specify a number within a valid range.

Specify a correct number.

Specify a valid number.

Specify correct generation number of syslog.

Specify a valid number for the syslog generation.

Collect type must be specified 'type1' or 'type2' or 'type3' or 'type4' or 'type5' or 'type6'. Incorrect collection type is specified.

Invalid collection type is specified.

Specify an absolute path as the destination of the files to be collected.

Specify an absolute path for the output destination of collected files.

Specifiable number of servers are the max number of servers that can constitute a cluster.

The number of servers that can be specified is the maximum number of servers that can be set up.

Could not connect to the server. Check if the cluster daemon is active.

Check if the cluster daemon is started.

Invalid server status.

Check if the cluster daemon is started.

Server is busy. Check if this command is already run.

This command may be run already. Check them.

Internal error. Check if memory or OS resources are sufficient.

Memory or OS resources may not be sufficient. Check them.

2.7.1. Collecting logs by specifying a type (-t option)¶

To collect only the specified types of logs, run the clplogcc command with the -t option.

Specify a type from 1 thorough 6 for the log collection.

type1

type2

type3

type4

Type 5

Type 6

Default collection information

✓

✓

✓

✓

n/a

n/a

syslog

✓

✓

✓

n/a

n/a

n/a

core

✓

✓

n/a

✓

n/a

n/a

OS information

✓

✓

✓

✓

n/a

n/a

script

✓

✓

n/a

n/a

n/a

n/a

ESMPRO/AC

✓

✓

n/a

n/a

n/a

n/a

HA Logs

n/a

✓

n/a

n/a

n/a

n/a

Mirror Statistics

n/a

n/a

n/a

n/a

n/a

n/a

Cluster Statistics

n/a

n/a

n/a

n/a

n/a

✓

System resource statistical information

✓

✓

✓

✓

n/a

✓

Run this command from the command line as follows.

Example: When collecting logs using type2

# clplogcc -t type2

When no option is specified, a log type will be type 1.

Information to be collected by default
- Logs of each module in the EXPRESSCLUSTER Server
- Alert logs
- Attribute of each module (ls -l) in the EXPRESSCLUSTER Server
  - In bin, lib
  - In alert/bin, webmgr/bin
  - In ha/jra/bin, ha/sra/bin, ha/jra/lib, ha/sra/lib
  - In drivers/md
  - In drivers/khb
  - In drivers/ka
- All installed packages (rpm -qa execution result)
- EXPRESSCLUSTER X SingleServerSafe version (rpm -qi expresscls execution result)
- distribution (/etc/*-release)
- License Information
- Configuration data file
- Policy file
- Dump files in the shared memory used by EXPRESSCLUSTER X SingleServerSafe
- Local node status of EXPRESSCLUSTER (clpstat --local execution results)
- Process and thread information (ps execution result)
- PCI device information (lspci execution result)
- Service information (execution results of the commands such as systemctl, chkconfig, and ls)
- Output result of kernel parameter (sysctl -a execution results)
- glibc version (rpm -qi glibc execution result)
- Kernel loadable module configuration (/etc/modules.conf. /etc/modprobe.conf)
- Kernel ring buffer (dmesg execution result)
- File system (/etc/fstab)
- IPC resource (ipcs execution result)
- System (uname -a execution result)
- Network statistics (netstat and ss execution result IPv4/IPv6)
- ip (execution results of the command ip addr, link, maddr, route or -s l)
- All network interfaces (ethtool execution result)
- Information collected upon emergency OS shutdown
- libxml2 version (rpm -qi libxml2 execution result)
- Static host table (/etc/hosts)
- File system export table (exportfs -v execution result)
- User resource limitations (ulimit -a execution result)
- File system exported by kernel-based NFS (/etc/exports)
- OS locale
- Terminal session environment value (export execution result)
- Language locale (/etc/sysconfig/i18n)
- Time zone (env - date execution result)
- Work area of EXPRESSCLUSTER server
- Monitoring options
  
  This information is collected if options are installed.
- Collected dump information when the monitor resource timeout occurred
- Collected Oracle detailed information when Oracle monitor resource abnormity was detected
syslog
- syslog (/var/log/messages)
- syslog (/var/log/syslog)
- Syslogs for the number of generations specified (/var/log/messages.x)
- journal log (such as files in /var/run/log/journal/)
core file
- core file of EXPRESSCLUSTER module
  
  Stored in /opt/nec/clusterpro/log by the following archive names.
Alert related:
```
altyyyymmdd_x.tar
```
Directory for the WebManager server related:
```
wmyyyymmdd_x.tar
```
EXPRESSCLUSTER core related:
```
clsyyyymmdd_x.tar
```
yyyymmdd indicates the date when the logs are collected. x is a sequence number.
OS information
- Kernel mode LAN heartbeat, keep alive
  - /proc/khb_moninfo
  - /proc/ka_moninfo
- /proc/devices
- /proc/mdstat
- /proc/modules
- /proc/mounts
- /proc/meminfo
- /proc/cpuinfo
- /proc/partitions
- /proc/pci
- /proc/version
- /proc/ksyms
- /proc/net/bond*
- all files of /proc/scsi/ all files in the directory
- all files of /proc/ide/ all files in the directory
- /etc/fstab
- /etc/syslog.conf
- /etc/syslog-ng/syslog-ng.conf
- /proc/sys/kernel/core_pattern
- /proc/sys/kernel/core_uses_pid
- /etc/snmp/snmpd.conf
- Kernel ring buffer (dmesg execution result)
- ifconfig (ifconfig execution result)
- iptables (iptables -L execution result)
- ipchains (ipchains -L execution result)
- df (df execution result)
- raw device information (raw -qa execution result)
- kernel module load information (lsmod execution result)
- host name, domain information (hostname, domainname execution result)
- dmidecode (dmidecode execution result)
- LVM device information (vgdisplay -v execution result)
- snmpd version information (snmpd -v execution result)
- Virtual Infrastructure information (the result of running virt-what)
When you collect logs, you may find the following message on the console. This does not mean failure. The logs are collected normally.
```
hd#: bad special flag: 0x03
ip_tables: (C) 2000-2002 Netfilter core team
```
(Where hd# is the name of the IDE device that exists on the server)
Script

Start/stop script for a group that was created with the Cluster WebUI.

If you specify a user-defined script other than the above (/opt/nec/clusterpro/scripts), it is not included in the log collection information. It must be collected separately.
ESMPRO/AC Related logs

Files that are collected by running the acupslog command.
HA logs
- System resource information
- JVM monitor log
- System monitor log
Mirror Statistics

This version does no collect.
Cluster Statistics
- Cluster Statistics
- In perf/cluster
System resource statistics

System resource statistics

In perf/system

2.7.2. syslog generations (-r option)¶

To collect syslogs for the number of generations specified, run the following command.

Example: Collecting logs for the 3 generations

# clplogcc -r 3

The following syslogs are included in the collected logs.

/var/log/messages
/var/log/messages.1
/var/log/messages.2

When no option is specified, two generations are collected.
You can collect logs for 0 to 99 generations.
When 0 is specified, all syslogs are collected.

Number of Generation	Number of generations to be acquired
0	All Generations
1	Current
2	Current + Generation 1
3	Current + Generation 1 to 2
:
x	Current + Generation 1 to (x - 1)

2.7.3. Output paths of log files (-o option)¶

Log file is named and be saved as "server name-log.tar.gz".
Since log files are compressed by tar.gz, add the xzf option to the tar command to decompress them.

If not specifying -o option

Logs are output in tmp of installation path.
# clplogcc
Collect Log server-name: Success
# ls /opt/nec/clusterpro/tmp
server-name-log.tar.gz

When the -o option is specified:

If you run the command as follows, logs are located in the specified /home/log directory.
# clplogcc -o /home/log
Collect Log server-name: Success
# ls /home/log
server-name-log.tar.gz

2.7.4. Collecting information when a failure occurs¶

When the following failure occurs, the information for analyzing the failure is collected.

When a server daemon configuring the server abnormally terminates due to interruption by a signal (core dump), an internal status error, or another cause
When a group resource activation error or deactivation error occurs
When monitoring error occurs in a monitor resource

Information to be collected is as follows:

Server information
- Some module logs in EXPRESSCLUSTER servers
- Dump files in the shared memory used by EXPRESSCLUSTER X SingleServerSafe
- Configuration data file
- Core files of EXPRESSCLUSTER module
OS information (/proc/*)
- /proc/devices
- /proc/partitions
- /proc/mdstat
- /proc/modules
- /proc/mounts
- /proc/meminfo
- /proc/net/bond*
Information created by running a command
- Results of the sysctl -a
- Results of the ps
- Results of the top
- Results of the ipcs
- Results of the netstat -i
- Results of the ifconfig
- Results of the df
- Results of the raw -qa
- journalctl -e execution result

These are collected by default in the log collection. You do not need to collect them separately.

2.8. Applying and backing up configuration data (clpcfctrl command)¶

2.8.1. Applying configuration data (clpcfctrl --push)¶

Applies the configuration data to servers.

Command line: clpcfctrl --push [-p portnumber] [-x directory] [--nocheck]

Description: Applies the configuration data created by the Cluster WebUI to servers.

Option

--push¶: Specify this option when applying the data.

This option cannot be omitted.

-p¶: portnumber Specifies a port number of data transfer port.

When this option is omitted, the default value is used. In general, it is not necessary to specify this option.

-x directory¶: Specify this option to apply the configuration data in the specified directory.

Use this option with either -l or -w.

When -l is specified, configuration data saved on the file system by the Cluster WebUI on Linux is used.

When -w is specified, configuration data saved by the Cluster WebUI on Windows is used.

--nocheck¶: Configuration data is not checked. Use this option only when deleting a server.

Return Value

0	Success
Other than 0	Failure

Notes: This command must be executed by a user with the root privilege.

Examples

Example 1: Delivering configuration data that was saved on the file system using the Cluster WebUI on Linux

# clpcfctrl --push -x /mnt/config
file delivery to server 127.0.0.1 success.
The upload is completed successfully.(cfmgr:0)
Command succeeded.(code:0)

Error Message

Message	Cause/Solution
Log in as root.	Log on as a root user.
This command is already run.	This command has already been run.
Invalid option.	The option is invalid. Check the option.
Invalid mode. Check if --push or --pull option is specified.	Check if the --push is specified.
The target directory does not exist.	The specified directory does not exist.
Invalid host name. Server specified by -h option is not included in the configuration data.	The server specified with -h is not included in configuration data. Check if the specified server name or IP address is correct.
Canceled.	This message is displayed when you enter a character other than "y" in response to the command.
Failed to initialize the xml library. Check if memory or OS resources are sufficient.	Memory or OS resources may not be sufficient. Check them.
Failed to load the configuration file. Check if memory or OS resources are sufficient.	Memory or OS resources may not be sufficient. Check them.
Failed to change the configuration file. Check if memory or OS resources are sufficient.	Memory or OS resources may not be sufficient. Check them.
Failed to load the all.pol file. Reinstall the RPM.	Reinstall the EXPRESSCLUSTER Server RPM.
Failed to load the cfctrl.pol file. Reinstall the RPM.	Reinstall the EXPRESSCLUSTER Server RPM.
Failed to get the install path. Reinstall the RPM.	Reinstall the EXPRESSCLUSTER Server RPM.
Failed to get the cfctrl path. Reinstall the RPM.	Reinstall the EXPRESSCLUSTER Server RPM.
Failed to get the list of group.	Failed to acquire the list of group.
Failed to get the list of resource.	Failed to acquire the list of resource.
Failed to initialize the trncl library. Check if memory or OS resources are sufficient.	Memory or OS resources may not be sufficient. Check them.
Failed to connect to server %1. Check if the other server is active and then run the command again.	Accessing the server has failed. Check if other server(s) has been started. Run the command again after the server has started up.
Failed to connect to trnsv. Check if the other server is active.	Accessing the server has failed. Check if other server(s) has been started.
File delivery failed. Failed to deliver the configuration data. Check if the other server is active and run the command again.	Delivering configuration data has failed. Check if other server(s) has been started. Run the command again after the server has started up.
Multi file delivery failed. Failed to deliver the configuration data. Check if the other server is active and run the command again.	Delivering configuration data has failed. Check if other server(s) has been started. Run the command again after the server has started up.
Failed to deliver the configuration data. Check if the other server is active and run the command again.	Delivering configuration data has failed. Check if other server(s) has been started. Run the command again after the server has started up.
The directory "/work" is not found. Reinstall the RPM.	Reinstall the EXPRESSCLUSTER Server RPM.
Failed to make a working directory.	Memory or OS resources may not be sufficient. Check them.
The directory does not exist.	Memory or OS resources may not be sufficient. Check them.
This is not a directory.	Memory or OS resources may not be sufficient. Check them.
The source file does not exist.	Memory or OS resources may not be sufficient. Check them.
The source file is a directory.	Memory or OS resources may not be sufficient. Check them.
The source directory does not exist.	Memory or OS resources may not be sufficient. Check them.
The source file is not a directory.	Memory or OS resources may not be sufficient. Check them.
Failed to change the character code set (EUC to SJIS).	Memory or OS resources may not be sufficient. Check them.
Failed to change the character code set (SJIS to EUC).	Memory or OS resources may not be sufficient. Check them.
Command error.	Memory or OS resources may not be sufficient. Check them.
Failed to initialize the cfmgr library. Check if memory or OS resources are sufficient.	Memory or OS resources may not be sufficient. Check them.
Failed to get size from the cfmgr library. Check if memory or OS resources are sufficient.	Memory or OS resources may not be sufficient. Check them.
Failed to allocate memory.	Memory or OS resources may not be sufficient. Check them.
Failed to change the directory.	Memory or OS resources may not be sufficient. Check them.
Failed to run the command.	Memory or OS resources may not be sufficient. Check them.
Failed to make a directory.	Memory or OS resources may not be sufficient. Check them.
Failed to remove the directory.	Memory or OS resources may not be sufficient. Check them.
Failed to remove the file.	Memory or OS resources may not be sufficient. Check them.
Failed to open the file.	Memory or OS resources may not be sufficient. Check them.
Failed to read the file.	Memory or OS resources may not be sufficient. Check them.
Failed to write the file.	Memory or OS resources may not be sufficient. Check them.
Internal error. Check if memory or OS resources are sufficient.	Memory or OS resources may not be sufficient. Check them.
The upload is completed successfully. To apply the changes you made, shutdown and reboot the cluster.	The upload is completed successfully. To apply the changes you made, shut down the server, and then reboot it.
The upload was stopped. To upload the cluster configuration data, stop the cluster.	The upload was stopped. To upload the configuration data, stop the server.
The upload was stopped. To upload the cluster configuration data, stop the Mirror Agent.	The upload was stopped. To upload the configuration data, stop MirrorAgent.
The upload was stopped. To upload the cluster configuration data, stop the resources to which you made changes.	The upload was stopped. To upload the configuration data, stop the resources you changed.
The upload was stopped. To upload the cluster configuration data, stop the groups to which you made changes.	The upload was stopped. To upload the configuration data, suspend the server. To upload, stop the group to which you made changes.
The upload was stopped. To upload the cluster configuration data, suspend the cluster.	The upload was stopped. To upload the configuration data, suspend the server.
The upload is completed successfully. To apply the changes you made, restart the Alert Sync service. To apply the changes you made, restart the WebManager service.	The upload is completed successfully. To apply the changes you made, restart the AlertSync service. To apply the changes you made, restart the WebManager service.
The upload is completed successfully. To apply the changes you made, restart the Information Base service.	The upload is completed successfully. To apply the changes you made, restart the Information Base service.
The upload is completed successfully. To apply the changes you made, restart the API service.	The upload is completed successfully. To apply the changes you made, restart the API service.
The upload is completed successfully. To apply the changes you made, restart the Node Manager service.	The upload is completed successfully. To apply the changes you made, restart the Node Manager service.
Internal error. Check if memory or OS resources are sufficient.	Memory or OS resources may not be sufficient. Check them.
The upload is completed successfully.	The upload is completed successfully.
The upload was stopped. Failed to deliver the configuration data. Check if the other server is active and run the command again.	The upload was stopped. Delivering configuration data has failed. Check if the other server is active and run the command again.
The upload was stopped. There is one or more servers that cannot be connected to. To apply cluster configuration information forcibly, run the command again with "--force" option.	The upload was stopped. The server that cannot connect exists. To forcibly upload the configuration data, run the command again with the --force option.

2.8.2. Backing up the configuration data (clpcfctrl --pull)¶

Backs up the configuration data.

Command line: clpcfctrl --pull -l|-w [-p portnumber] [-x directory]

Description: Backs up the configuration data to be used by the Cluster WebUI.

Option

--pull¶: Specify this option when performing backup.

This option cannot be omitted.

-l¶: Specify this option when backing up data as the configuration data that is used for the Cluster WebUI on Linux.

You cannot specify both -l and -w together.

-w¶: Specify this option when backing up data as the configuration data that is used for the Cluster WebUI on Windows.

You cannot specify both -l and -w together.

-p¶: portnumber Specifies a port number of data transfer port.

When this option is omitted, the default value is used. In general, it is not necessary to specify this option.

-x directory¶: Specify this option when backing up configuration data in the specified directory.

Use this option with either -l or -w.

When -l is specified, configuration data is backed up in the format which can be loaded by the Cluster WebUI on Linux.

When -w is specified, configuration data is saved in the format which can be loaded by the Cluster WebUI on Windows.

Return Value

0	Success
Other than 0	Failure

Notes: This command must be executed by a user with the root privilege.

Examples

Example 1: Backing up configuration data to the specified directory so that the data can be loaded by the Cluster WebUI on Linux

# clpcfctrl --pull -l -x /mnt/config
Command succeeded.(code:0)

Error Message

Message	Cause/Solution
Log in as root.	Log on as a root user.
This command is already run.	This command has already been run.
Invalid option.	The option is invalid. Check the option.
Invalid mode. Check if --push or --pull option is specified.	Check if the --pull is specified.
The target directory does not exist.	The specified directory does not exist.
Canceled.	This message is displayed when you enter a character other than "y" in response to the command.
Failed to initialize the xml library. Check if memory or OS resources are sufficient.	Memory or OS resources may not be sufficient. Check them.
Failed to load the configuration file. Check if memory or OS resources are sufficient.	Memory or OS resources may not be sufficient. Check them.
Failed to change the configuration file. Check if memory or OS resources are sufficient.	Memory or OS resources may not be sufficient. Check them.
Failed to load the all.pol file. Reinstall the RPM.	Reinstall the EXPRESSCLUSTER Server RPM.
Failed to load the cfctrl.pol file. Reinstall the RPM.	Reinstall the EXPRESSCLUSTER Server RPM.
Failed to get the install path. Reinstall the RPM.	Reinstall the EXPRESSCLUSTER Server RPM.
Failed to get the cfctrl path. Reinstall the RPM.	Reinstall the EXPRESSCLUSTER Server RPM.
Failed to initialize the trncl library. Check if memory or OS resources are sufficient.	Memory or OS resources may not be sufficient. Check them.
Failed to connect to server %1. Check if the other server is active and then run the command again.	Accessing the server has failed. Check if other server(s) has been started. Run the command again after the server has started up.
Failed to connect to trnsv. Check if the other server is active.	Accessing the server has failed. Check if other server(s) has been started.
Failed to get configuration data. Check if the other server is active.	Acquiring configuration data has failed. Check if other server(s) has been started.
The directory "/work" is not found. Reinstall the RPM.	Reinstall the EXPRESSCLUSTER Server RPM.
Failed to make a working directory.	Memory or OS resources may not be sufficient. Check them.
The directory does not exist.	Memory or OS resources may not be sufficient. Check them.
This is not a drirectory.	Memory or OS resources may not be sufficient. Check them.
The source file does not exist.	Memory or OS resources may not be sufficient. Check them.
The source file is a directory.	Memory or OS resources may not be sufficient. Check them.
The source directory does not exist.	Memory or OS resources may not be sufficient. Check them.
The source file is not a directory.	Memory or OS resources may not be sufficient. Check them.
Failed to change the character code set (EUC to SJIS).	Memory or OS resources may not be sufficient. Check them.
Failed to change the character code set (SJIS to EUC).	Memory or OS resources may not be sufficient. Check them.
Command error.	Memory or OS resources may not be sufficient. Check them.
Failed to initialize the cfmgr library. Check if memory or OS resources are sufficient.	Memory or OS resources may not be sufficient. Check them.
Failed to get size from the cfmgr library. Check if memory or OS resources are sufficient.	Memory or OS resources may not be sufficient. Check them.
Failed to allocate memory.	Memory or OS resources may not be sufficient. Check them.
Failed to change the directory.	Memory or OS resources may not be sufficient. Check them.
Failed to run the command.	Memory or OS resources may not be sufficient. Check them.
Failed to make a directory.	Memory or OS resources may not be sufficient. Check them.
Failed to remove the directory.	Memory or OS resources may not be sufficient. Check them.
Failed to remove the file.	Memory or OS resources may not be sufficient. Check them.
Failed to open the file.	Memory or OS resources may not be sufficient. Check them.
Failed to read the file.	Memory or OS resources may not be sufficient. Check them.
Failed to write the file.	Memory or OS resources may not be sufficient. Check them.
Internal error. Check if memory or OS resources are sufficient.	Memory or OS resources may not be sufficient. Check them.

2.9. Adjusting time-out temporarily (clptoratio command)¶

Extends or displays the current timeout ratio.

Command line: clptoratio -r ratio -t time

clptoratio -i

clptoratio -s

Description

Temporarily extends the following timeout values:

Monitor resource
Heartbeat resource
Alert synchronous service
WebManager service

The current timeout ratio is displayed.

Option

-r ratio¶: Specifies the timeout ratio. Use 1 or larger integer. The maxim timeout ratio is 10,000.

If you specify "1," you can return the modified timeout ratio to the original as you can do so when you are using the -i option.

-t time¶: Specifies the extension period.

You can specify minutes for m, hours for h, and days for d. The maximum period of time is 30 days.

Example:

2m, 3h, 4d

-i¶: Sets back the modified timeout ratio.

-s¶: Refers to the current timeout ratio.

Return Value

0	Success
Other than 0	Failure

Remarks

When the server is shut down, the timeout ratio you have set will become ineffective. However, if the server is not shut down, the timeout ratio and the extension period that you have set will be maintained.
With the -s option, you can only refer to the current timeout ratio. You cannot see other information such as remaining time of extended period.
You can see the original timeout value by using the status display command.

Heartbeat timeout

# clpstat --cl --detail

Monitor resource timeout

# clpstat --mon monitor resource name --detail

Notes: This command must be executed by a user with the root privilege.

Execute this command when the EXPRESSCLUSTER daemon of the server is active.

When you set the timeout ratio, make sure to specify the extension period. However, if you set "1" for the timeout ratio, you cannot specify the extension period.

You cannot specify a combination such as "2m3h," for the extension period.

Examples

Example 1: Doubling the timeout ratio for three days

# clptoratio -r 2 -t 3d

Example 2: Setting back the timeout ratio to original

# clptoratio -i

Example 3: Referring to the current timeout ratio

# clptoratio -s
present toratio : 2

The current timeout ratio is set to 2.

Error Message

Message	Cause/Solution
Log in as root.	Log on as root user.
Invalid configuration file. Create valid cluster configuration data.	Create valid cluster configuration data by using the Cluster WebUI.
Invalid option.	Specify a valid option.
Specify a number in a valid range.	Specify a number within a valid range.
Specify a correct number.	Specify a valid number.
Scale factor must be specified by integer value of 1 or more.	Specify 1 or larger integer for ratio.
Specify scale factor in a range less than the maximum scale factor.	Specify a ratio that is not larger than the maximum ratio.
Set the correct extension period.	Set a valid extension period.
Ex) 2m, 3h, 4d	Set the extension period which does not exceed the maximum ratio.
Set the extension period in a range less than the maximum extension period.	Check if the cluster daemon is started.
Could not connect to the server. Check if the cluster daemon is active.	Check if the cluster daemon is started.
Server is not active. Check if the cluster daemon is active.	Check if there is any server in the cluster with the cluster daemon stopped.
Connection was lost. Check if there is a server where the cluster daemon is stopped in the cluster.	Check if there is any server in the cluster with the cluster daemon stopped.
Invalid parameter.	The value specified as a parameter of the command may be invalid.
Internal communication timeout has occurred in the cluster server. If it occurs frequently, set the longer timeout.	Time-out has occurred in the internal communication of EXPRESSCLUSTER. If it occurs frequently, set the internal communication time-out longer.
Processing failed on some servers. Check the status of failed servers.	There are servers that failed in processing. Check the status of server in the cluster. Operate it while all the servers in the cluster are up and running.
Internal error. Check if memory or OS resources are sufficient.	Check to see if the memory or OS resource is sufficient.

2.10. Modifying the log level and size (clplogcf command)¶

Modifies and displays log level and log output file size.

Command line: clplogcf -t type -l level -s size

Description: Modifies the settings of the log level and log output file size.

Displays the currently specified values.

Option

-t type¶: Specifies a module type whose settings will be changed.

If both -l and -s are omitted, the information set to the specified module will be displayed. See the list of "Types that can be specified to the -t option" for types which can be specified.

-l level¶: Specifies a log level.

You can specify one of the following for a log level.

1, 2, 4, 8, 16, 32

You can see more detailed information as the log level increases.

-s¶: size Specifies the size of a file for log output.

The unit is byte.

None¶: Displays the entire configuration information currently set.

Return Value

0	Success
Other than 0	Failure

Remarks: Each type of log output by EXPRESSCLUSTER X SingleServerSafe uses four log files. Therefore, it is necessary to have the disk space that is four times larger than what is specified by -s.

Notes: This command must be executed by a user with the root privilege.

To run this command, the EXPRESSCLUSTER event service must be started.

The settings revert to the default values when the server restarts.

Examples

Example 1: Modifying the pm log level

# clplogcf -t pm -l 8

Example 2: Seeing the pm log level and log file size

# clplogcf -t pm
TYPE, LEVEL, SIZE
pm, 8, 1000000

Example 3: Displaying the values currently configured

# clplogcf
TYPE, LEVEL, SIZE
trnsv, 4, 1000000
xml, 4, 1000000
logcf, 4, 1000000

Error Message

Message	Cause/Solution
Log in as root.	Log on as a root user.
Invalid option.	The option is invalid. Check the option.
Failed to change the configuration. Check if clpevent is running.	clpevent may not be started yet.
Invalid level	The specified level is invalid.
Invalid size	The specified size is invalid.
Failed to load the configuration file. Check if memory or OS resources are sufficient.	The server has not been created.
Failed to initialize the xml library. Check if memory or OS resources are sufficient.	Memory or OS resources may not be sufficient. Check them.
Failed to print the configuration. Check if clpevent is running.	clpevent may not be started yet.

Types that can be specified for the -t option (y=yes, n=no)

Type:	Module Type	Explanation	The EXPRESSCLUSTER Server
apicl	libclpapicl.so.1.0	API client library	✓
apisv	libclpapisv.so.1.0	API server	✓
cl	clpcl	Server startup and stop command	✓
cfctrl	clpcfctrl	Server generation and server information backup command	✓
cfmgr	libclpcfmgr.so.1.0	Configuration data operation library	✓
grp	clpgrp	Group startup and stop command	✓
rsc	clprsc	Group resource startup and stop command	✓
haltp	clpuserw	Shutdown stalling monitoring	✓
healthchk	clphealthchk	Process health check command	✓
ibsv	clpibsv	Information Base server	✓
lcns	libclplcns.so.1.0	License library	✓
lcnsc	clplcnsc	License registration command	✓
logcc	clplogcc	Collect logs command	✓
logcf	clplogcf	Log level and size modification command	✓
logcmd	clplogcmd	Alert producing command	✓
mail	clpmail	Mail Report	✓
mgtmib	libclpmgtmib.so.1.0	SNMP coordination library	✓
monctrl	clpmonctrl	Monitoring control command	✓
nm	clpnm	node map management	✓
pm	clppm	Process management	✓
rc/rc_ex	clprc	Group and group resource management	✓
reg	libclpreg.so.1.0	Reboot count control library	✓
regctrl	clpregctrl	Reboot count control command	✓
rm	clprm	Monitor management	✓
roset	clproset	Disk control	✓
relpath	clprelpath	Process kill command	✓
scrpc	clpscrpc	Script log rotation command	✓
stat	clpstat:	Status display command	✓
stdn	clpstdn	Server shutdown command	✓
toratio	clptoratio	Timeout ratio modification command	✓
trap	clptrap	SNMP trap command	✓
trncl	libclptrncl.so.1.0	Transaction library	✓
rexec	clprexec	External monitoring link processing request command	✓
trnsv	clptrnsv	Transaction server	✓
alert	clpaltinsert	Alert	✓
webmgr	clpwebmc	WebManager service	✓
webalert	clpaltd	Alert synchronization	✓
exec	clpexec	Exec resource	✓
diskw	clpdiskw	Disk monitor resource	✓
ipw	clpipw	IP monitor resource	✓
miiw	clpmiiw	NIC Link Up/Down monitor resource	✓
mtw	clpmtw	Multi target monitor resource	✓
pidw	clppidw	PID monitor resource	✓
volmgrw	clpvolmgrw	Volume manager monitor resource	✓
userw	clpuserw	User mode monitor resource	✓
mrw	clpmrw	Message reception monitor resource	✓
snmpmgr	libclp snmpmgr	SNMP trap reception library	✓
lanhb	clplanhb	LAN heartbeat	✓
oraclew	clp_oraclew	Oracle monitor resource	✓
db2w	clp_db2w	DB2 monitor resource	✓
psqlw	clp_psqlw	PostgreSQL monitor resource	✓
mysqlw	clp_mysqlw	MySQL monitor resource	✓
odbcw	clp_odbcw	ODBC monitor resource	✓
sqlserverw	clp_sqlserverw	SQL Server monitor resource	✓
sambaw	clp_sambaw	Samba monitor resource	✓
nfsw	clp_nfsw	NFS monitor resource	✓
httpw	clp_httpw	HTTP monitor resource	✓
ftpw	clp_ftpw	FTP monitor resource	✓
smtpw	clp_smtpw	SMTP monitor resource	✓
pop3w	clp_pop3w	POP3 monitor resource	✓
imap4w	clp_imap4w	IMAP4 monitor resource	✓
tuxw	clp_tuxw	Tuxedo monitor resource	✓
wlsw	clp_wlsw	WebLogic monitor resource	✓
wasw	clp_wasw	WebSphere monitor resource	✓
otxw	clp_otxw	WebOTX monitor resource	✓
jraw	clp_jraw	JVM monitor resource	✓
sraw	clp_sraw	System monitor resource	✓
psrw	clp_psrw	Process resource monitor resource	✓
psw	clppsw	Process name monitor resource	✓
vmctrl	libclpvmctrl.so.1.0	VMCtrl library	✓
perfc	clpperfc	Command to display cluster statistical information	✓
cfchk	clpcfchk	Command to check cluster configuration data	✓

2.11. Managing licenses (clplcnsc command)¶

the clplcnsc command manages licenses.

Command line: clplcnsc -i [licensefile...]

clplcnsc -l [-a]

clplcnsc -d serialno [-q]

clplcnsc -d -t [-q]

clplcnsc -d -a [-q]

clplcnsc --reregister licensefile...

Description: This command registers, refers to and remove the licenses of the product version and trial version of this product.

Option

-i [licensefile...]¶: When a license file is specified, license information is acquired from the file for registration. You can specify multiple licenses. You can also specify a wildcard. If nothing is specified, you need to enter license information interactively.

-l [-a]¶

References the registered license.

The name of displayed items are as follows.

Item	Explanation
Serial No	Serial number (product version only)
User name	User name (trial version only)
Key	License key
Licensed Number of CPU	The number of license (per CPU)
Licensed Number of Computers	The number of license (per node)
Start date	Start date of valid period 1 2
End date	End date of valid period 1 2
Status	Status of the license

Status	Explanation
valid	valid
invalid	invalid
unknown	unknown
inactive	Before valid period 1 2
expired	After valid period 1 2

1(1,2,3,4): Displayed in the case of the fixed term license
2(1,2,3,4): Displayed in the case of the license of trial version

When -a option not specifed, the license status of "invalid", "unknown" and "expired" are not displayed.

When specifying -a option, all the licenses are displayed regardless of the license status.

-d <param>¶

param

serialno
Deletes the license with the specified serial number.

-t
Deletes all the registered licenses of the trial version.

-a
Deletes all the registered licenses.

-q¶: Deletes licenses without displaying a warning message. This is used with -d option.

--reregister licensefile...¶: Reregisters a fixed-term license. Usually, it is unnecessary to execute the command with this option.

Return Value

0	Normal termination
1	Cancel
3	Initialization error
5	The option is invalid
8	Other internal error

Example of a command entry:

for registration

Registering the license interactively

# clplcnsc -i

Product Version/Product Version (Fixed Term)

Select a product division.
Selection of License Version
1.  Product Version
2.  Trial Version
e. Exit
Select License Version. [1, 2, or e (default:1)]...
Enter a serial number.

Enter serial number [ Ex. XXXXXXXX000000]...

Enter a license key.

Enter license key [ Ex. XXXXXXXX-XXXXXXXX-XXXXXXXX-XXXXXXXX]...

Trial Version

Select a product division.

Selection of License Version
1.  Product Version
2.  Trial Version
e. Exit
Select License Version. [1, 2, or e (default:1)]...

Enter a user name.

Enter user name [ 1 to 63byte ]...

Enter a license key.

Enter license key
[Ex. XXXXX-XXXXXXXX-XXXXXXXX-XXXXXXXX]...

Specify a license file

# clplcnsc -i /tmp/cpulcns.key

for referring to the license
```
# clplcnsc -l
```

Product version

< EXPRESSCLUSTER X SingleServerSafe <PRODUCT> >

Seq... 1
Key..... A1234567-B1234567-C1234567-D1234567
Licensed Number of CPU... 2
Status... valid

Seq... 2
Serial No..... AAAAAAAA000002
Key..... E1234567-F1234567-G1234567-H1234567
Licensed Number of Computers... 1
Status... valid

Product version (fixed term)

< EXPRESSCLUSTER X SingleServerSafe <PRODUCT> >

Seq... 1
Serial No..... AAAAAAAA000001
Key..... A1234567-B1234567-C1234567-D1234567
Start date..... 2018/01/01
End date...... 2018/01/31
Status........... valid

Seq... 2
Serial No..... AAAAAAAA000002
Key..... E1234567-F1234567-G1234567-H1234567
Status........... inactive

Trial version

< EXPRESSCLUSTER X SingleServerSafe <TRIAL> >

Seq... 1
Key..... A1234567-B1234567-C1234567-D1234567
User name... NEC
Start date..... 2018/01/01
End date...... 2018/02/28
Status........... valid

for deleting the license
```
# clplcnsc -d AAAAAAAA000001 -q
```
for deleting the license
```
# clplcnsc -d -t -q
```
for deleting the license
```
# clplcnsc -d -a
```

Deletion confirmation
Are you sure to remove the license? [y/n] ...

Notes: Run this command as the root user.

Furthermore, when you use -d option and -a option together, all the trial version licenses and product version licenses will be deleted. To delete only the trial license, also specify the -t option. If the licenses including the product license have been deleted, register the product license again.

When you refer to a license which includes multiple licenses, all included licenses information are displayed.

Error Message

Message	Cause/Solution
Processed license num (success : %d, error : %d).	The number of processed licenses (success:%d, error:%d) If error is not 0, check if the license information is correct.
Command succeeded.	The command ran successfully.
Command failed.	The command did not run successfully.
Log in as root.	You are not authorized to run this command. Log on as the root user.
Invalid cluster configuration data. Check the cluster configuration information.	The cluster configuration data is invalid. Check the cluster configuration data by using the Cluster WebUI.
Initialization error. Check if memory or OS resources are sufficient.	Check to see if the memory or OS resource is sufficient.
The command is already run.	The command is already running. Check the running status by using a command such as the ps command.
The license is not registered.	The license has not been registered yet.
Could not open the license file. Check if the license file exists on the specified path.	Input/Output cannot be done to the license file. Check to see if the license file exists in the specified path.
Could not read the license file. Check if the license file exists on the specified path.	Input/Output cannot be done to the license file. Check to see if the license file exists in the specified path.
The field format of the license file is invalid. The license file may be corrupted. Check the destination from where the file is sent.	The field format of the license file is invalid. The license file may be corrupted. Check it with the file sender.
The cluster configuration data may be invalid or not registered.	The cluster configuration data may be invalid or not registered. Check the configuration data.
Failed to terminate the library. Check if memory or OS resources are sufficient.	Check to see if the memory or OS resource is sufficient.
Failed to register the license. Check if the entered license information is correct.	Check to see if the entered license information is correct.
Failed to open the license. Check if the entered license information is correct.	Check to see if the entered license information is correct.
Failed to remove the license.	License deletion failed. Parameter error may have occurred or resources (memory or OS) may not be sufficient.
This license is already registered.	This license has already been registered. Check the registered license.
This license is already activated.	This license has already been used. Check the registered license.
This license is unavailable for this product.	This license cannot be used for this product. Check the license.
The maximum number of licenses was reached.	The maximum number of registered licenses has been reached. Delete invalid licenses.
Internal error. Check if memory or OS resources are sufficient.	Check to see if the memory or OS resource is sufficient.

2.12. Outputting messages (clplogcmd command)¶

Registers the specified message with syslog and alert logs, or reports the message by mail.

Command line: clplogcmd -m message [--syslog] [--alert] [--mail] [-i eventID] [-l level]

Note

Generally, it is not necessary to run this command to set up or operate a server. You need to write the command in the EXEC resource script.

Description: Write this command in the EXEC resource script and output messages you want to send to the destination.

Option

-m message¶: Specifies a message. This option cannot be omitted. The maximum size of message is 511 bytes. (When syslog is specified as an output destination, the maximum size is 485 bytes.) The message exceeding the maximum size will not be shown.

You may use alphabets, numbers, and symbols 3 .

--syslog¶

--alert¶

--mail¶

--trap¶: Specify the output destination from syslog, alert, mail and trap. (You can specify multiple destinations.)

This parameter can be omitted. The syslog and alert will be the output destinations when the parameter is omitted.

-i eventID¶: Specify an event ID. Specify event ID. The maximum value of event ID is 10,000.

This parameter can be omitted. The default value 1 is set when the parameter is omitted.

-l level¶: Level of alert to output.

Select a level of alert output from ERR, WARN, or INFO. The icon on the alert logs of the Cluster WebUI is determined according to the level you select here.

This parameter can be omitted. The default value INFO is set to level when the parameter is omitted.

For details, see the online manual.

Return Value

0	Success
Other than 0	Failure

Notes: This command must be executed by a user with the root privilege.

When mail is specified as the output destination, you need to make the settings to send mails by using the mail command.

Examples

Example 1: When specifying only message (output destinations are syslog and alert):

When the following is written in the EXEC resource script, the message is produced in syslog and alert.

clplogcmd -m test1

The following log is the log output in syslog:

Sep 1 14:00:00 server1 clusterpro: <type: logcmd><event: 1> test1

Example 2: When specifying message, output destination, event ID, and level (output destination is mail):

When the following is written in the EXEC resource script, the message is sent to the mail address set in the Cluster Properties of the Cluster WebUI.

clplogcmd -m test2 --mail -i 100 -l ERR

The following information is sent to the mail destination:

Message:test2
Type: logcmd
ID: 100
Host: server1
Date: 2018/09/01 14:00:00

Example 3: When specifying a message, output destination, event ID, and level (output destination is trap):

When the following is written in the exec resource script, the message is set to the SNMP trap destination set in Cluster Properties of the Cluster WebUI. For more information on the SNMP trap destination settings, see "Cluster properties - Alert Service tab" in "Details of other settings" in "EXPRESSCLUSTER X SingleServerSafe Configuration Guide".

clplogcmd -m test3 --trap -i 200 -l ERR

The following information is sent to the SNMP trap destination:

Trap OID: clusterEventError

Attached data 1: clusterEventMessage = test3

Attached data 2: clusterEventID = 200

Attached data 3: clusterEventDateTime = 2011/08/01 09:00:00

Attached data 4: clusterEventServerName = server1

Attached data 5: clusterEventModuleName = logcmd

3

Notes on using symbols in the message:

The symbols below must be enclosed in double quotes (""):

# & ' ( ) ~ | ; : * < > , .

(For example, if you specify "#" in the message, # is produced.)
The symbols below must have a backslash \ in the beginning:

\ ! " & ' ( ) ~ | ; : * < > , .

(For example, if you specify \\ in the message, \ is produced.)
The symbol that must be enclosed in double quotes ("") and have a backslash \ in the beginning:

`

(For example, if you specify "\" in the message, ` is produced.)
When there is a space in the message, it must be placed in enclosed in double quotes ("").
The symbol % cannot be used in the message.

2.13. Controlling monitor resources (clpmonctrl command)¶

Controls the monitor resources.

Command line: clpmonctrl -s [-m resource_name] [-w wait_time]

clpmonctrl -r [-m resource_name] [-w wait_time]

clpmonctrl -c [-m resource_name]

clpmonctrl -v [-m resource_name]

clpmonctrl -e -m resource_name

clpmonctrl -n [-m resource_name]

Description: Suspends and/or resumes monitor resources.

Option

-s¶: Suspends monitoring.

-r¶: Resumes monitoring.

-c¶: Resets the times counter of the recovery action.

-v¶: Displays the times counter of the recovery action.

-e¶: Enables the Dummy Failure. Be sure to specify a monitor resource name with the -m option.

-n¶: Disables the Dummy Failure. When a monitor resource name is specified with the -m option, the function is disabled only for the resource. When the -m option is omitted, the function is disabled for all monitor resources.

-m resource_name¶: Specifies a monitor resource to be controlled.

This option can be omitted. All monitor resources are controlled when the option is omitted.

-w wait_time¶: Waits for control monitoring on a monitor resource basis. (in seconds)

This option can be omitted. The default value 5 is set when the option is omitted.

Return Value

0	Completed successfully.
1	Privilege for execution is invalid
2	The option is invalid
3	Initialization error
4	The configuration data is invalid.
5	Monitor resource is not registered.
6	The specified monitor resource is invalid
10	EXPRESSCLUSTER is not running.
11	The cluster daemon is suspended
90	Monitoring control wait timeout
128	Duplicated activation
255	Other internal error

Examples

Example 1: When suspending all monitor resources:

# clpmonctrl -s
Command succeeded.

Example 2: When resuming all monitor resources:

# clpmonctrl -r
Command succeeded.

Remarks: If you suspend an already suspended monitor resource or resume an already started one, this command abends without changing the status of the monitor resource.

Notes

This command must be executed by a user with the root privilege.
Check the status of monitor resource by using the status display command or Cluster WebUI.
Before you run this command, use the clpstat command or Cluster WebUI to verify that the status of monitor resources is in either "Online" or "Suspend."

When the recovery action of monitor resource uses one of the following settings, "Final Action Count" (which is displayed in the -v option) indicates the number of times to execute a script before the final action.

Execute Script Before Final Action: Enable
Final action: No Operation

Error Messages

Message	Causes/Solution
Command succeeded.	The command ran successfully.
Log in as root.	You are not authorized to run this command. Log on as root user.
Initialization error. Check if memory or OS resources are sufficient.	Check to see if the memory or OS resource is sufficient.
Invalid cluster configuration data. Check the cluster configuration information.	The cluster configuration data is invalid. Check the cluster configuration data by using the Cluster WebUI.
Monitor resource is not registered.	The monitor resource is not registered.
Specified monitor resource is not registered. Check the cluster configuration information.	The specified monitor resource is not registered. Check the cluster configuration data by using the Cluster WebUI.
The cluster has been stopped. Check the active status of the cluster daemon by using the command such as ps command.	The cluster has been stopped. Check the activation status of the cluster daemon by using a command such as ps command.
The cluster has been suspended. The cluster daemon has been suspended. Check activation status of the cluster daemon by using a command such as the ps command.	The cluster daemon has been suspended. Check the activation status of the cluster daemon by using a command such as ps command.
Waiting for synchronization of the cluster. The cluster is waiting for synchronization. Wait for a while and try again.	Synchronization of the cluster is awaited. Try again after cluster synchronization is completed.
Monitor %1 was unregistered, ignored. The specified monitor resources %1 is not registered, but continue processing. Check the cluster configuration data.	There is an unregistered monitor resource in the specified monitor resources but it is ignored and the process is continued. Check the cluster configuration data by using the Cluster WebUI. %1: Monitor resource name
Monitor %1 denied control permission, ignored. but continue processing.	The specified monitor resources contain the monitor resource which cannot be controlled, but it does not affect the process. %1: Monitor resource name
This command is already run.	The command is already running. Check the running status by using a command such as ps command.
Internal error. Check if memory or OS resources are sufficient.	Check to see if the memory or OS resource is sufficient.

Monitor resource types that can be specified for the -m option

Type	Suspending/resuming monitoring	Resetting the times counter of the recovery action	Enabling/disabling Dummy Failure
diskw	✓	✓	✓
ipw	✓	✓	✓
miiw	✓	✓	✓
mtw	✓	✓	✓
pidw	✓	✓	✓
volmgrw	✓	✓	✓
userw	✓	✓	n/a
mrw	✓	✓	n/a
genw	✓	✓	✓
oraclew	✓	✓	✓
db2w	✓	✓	✓
psqlw	✓	✓	✓
mysqlw	✓	✓	✓
odbcw	✓	✓	✓
sqlserverw	✓	✓	✓
sambaw	✓	✓	✓
nfsw	✓	✓	✓
httpw	✓	✓	✓
ftpw	✓	✓	✓
smtpw	✓	✓	✓
pop3w	✓	✓	✓
imap4w	✓	✓	✓
tuxw	✓	✓	✓
wlsw	✓	✓	✓
wasw	✓	✓	✓
otxw	✓	✓	✓
jraw	✓	✓	✓
sraw	✓	✓	✓
psrw	✓	✓	✓
psw	✓	✓	✓

2.14. Controlling group resources (clprsc command)¶

Controls group resources.

Command line: clprsc -s resource_name [-f] [--apito timeout]

clprsc -t resource_name [-f] [--apito timeout]

Description: Starts and stops group resources.

Option

-s¶: Starts group resources.

-t¶: Stops group resources.

-f¶: When the group resource is online, all group resources that the specified group resource depends starts up.

When the group resource is offline, all group resources that the specified group resource depends stop.

--apito timeout¶: Specify the interval (internal communication timeout) to wait for the group resource start or stop in seconds. A value from 1 to 9999 can be specified.

If the --apito option is not specified, waiting for the group resource start or stop is performed according to the value set to the internal communication timeout of the cluster properties.

Return Value

0	Completed successfully.
Other than 0	Terminated due to a failure.

Examples

Group resource configuration

# clpstat
========== CLUSTER STATUS  ==========
Cluster : cluster
<server>
    server1     : Online
      lanhb1    : Normal
      lanhb2    : Normal
<group>
    ManagementGroup : Online
      current   : server1
      ManagementIP : Online
    failover1   : Online
      current   : server1
      exec1     : Online
<monitor>
    ipw1        : Normal
==================================

Example 1: When stopping the resource (exec1) of the group (failover1)

# clprsc -t exec1
Command succeeded.

# clpstat

========== CLUSTER STATUS ==========
<Abbreviation>
<group>
ManagementGroup: Online
Current: server1
ManagementIP: Online
failover1: Online
current: server1
exec1: Offline
<Abbreviation>

Example 2: When starting the resource (fip1) of the group(failover 1)

# clprsc -s exec1
Command succeeded.

# clpstat
========== CLUSTER STATUS ==========
<Abbreviation>
<group>
ManagementGroup: Online
Current: server1
ManagementIP: Online
failover1: Online
current: server1
exec1: Online
<Abbreviation>

Notes: This command must be executed by a user with the root privilege.

Check the status of the group resources by the status display or the Cluster WebUI.

Error Messages

Message	Causes/Solution
Log in as Administrator.	Run this command as a user with Administrator privileges.
Invalid cluster configuration data. Check the cluster configuration information.	The cluster construction information is not correct. Check the cluster construction information by Cluster WebUI.
Invalid option.	Specify a correct option.
Could not connect server. Check if the cluster service is active.	Check if the EXPRESSCLUSTER is activated.
Invalid server status. Check if the cluster service is active.	Check if the EXPRESSCLUSTER is activated.
Server is not active. Check if the cluster service is active.	Check if the EXPRESSCLUSTER is activated.
Invalid server name. Specify a valid server name in the cluster.	Specify a correct server name in the cluster.
Connection was lost. Check if there is a server where the cluster service is stopped in the cluster.	Check if there is any server with EXPRESSCLUSTER service stopped in the cluster.
Internal communication timeout has occurred in the cluster server. If it occurs frequently, set the longer timeout. The group resource is busy. Try again later.	Timeout has occurred in internal communication in the EXPRESSCLUSTER. Set the internal communication timeout longer if this error occurs frequently. Because the group resource is in the process of starting or stopping, wait for a while and try again.
An error occurred on group resource. Check the status of group resource.	Check the group resource status by using the Cluster WebUI or the clpstat command.
Could not start the group resource. Try it again after the other server is started, or after the Wait Synchronization time is timed out.	Wait until the other server starts or the wait time times out, and then start the group resources.
No operable group resource exists in the server.	Check there is a processable group resource on the specified server.
The group resource has already been started on the local server.	Check the group resource status by using the Cluster WebUI or clpstat command.
The group resource has already been started on the other server.	Check the group resource status by using the Cluster WebUI or clpstat command. Stop the group to start the group resources on the local server.
The group resource has already been stopped.	Check the group resource status by using the Cluster WebUI or clpstat command.
Failed to start group resource. Check the status of group resource.	Check the group resource status by using the Cluster WebUI or clpstat command.
Failed to stop resource. Check the status of group resource.	Check the group resource status by using the Cluster WebUI or clpstat command.
Depended resource is not offline. Check the status of resource.	Because the status of the depended group resource is not offline, the group resource cannot be stopped. Stop the depended group resource or specify the -f option.
Depending resource is not online. Check the status of resource.	Because the status of the depended group is not online, the group resource cannot be started. Start the depended group resource or specify the -f option.
Invalid group resource name. Specify a valid group resource name in the cluster.	The group resource is not registered.
Server is not in a condition to start resource or any critical monitor error is detected.	Check the group resource status by using the Cluster WebUI or clpstat command. An error is detected in a critical monitor on the server on which an attempt to start a group resource was made.
Internal error. Check if memory or OS resources are sufficient.	Memory or OS resources may be insufficient. Check them.

2.15. Requesting processing to cluster servers (clprexec command)¶

Issues a processing execution request to another server on which EXPRESSCLUSTER is installed.

Command line: clprexec --failover [group_name] -h IP [-r resource_name] [-w timeout] [-p port_number] [-o logfile_path]

clprexec --script script_file -h IP [-p port_number] [-w timeout] [-o logfile_path]

clprexec --notice [mrw_name] -h IP [-k category[.keyword]] [-p port_number] [-w timeout] [-o logfile_path]

clprexec --clear [mrw_name] -h IP [-k category[.keyword]] [-p port_number] [-w timeout] [-o logfile_path]

Description: The command issues the request to execute specified process to the server in another cluster.

Option

--failover¶: Requests group failover. Specify a group name for group_name.

When not specifying the group name, specify the name of a resource that belongs to the group by using the -r option.

--script script_name¶: Requests script execution.

For script_name, specify the file name of the script to execute (such as a shell script or executable file).

The script must be created in the work/rexec folder, which is in the folder where EXPRESSCLUSTER is installed, on each server specified using -h.

--notice¶: Sends an error message to the EXPRESSCLUSTER server.

Specify a message reception monitor resource name for mrw_name.

When not specifying the monitor resource name, specify the category and keyword of the message reception monitor resource by using the -k option.

--clear¶: Requests changing the status of the message reception monitor resource from "Abnormal" to "Normal."

Specify a message reception monitor resource name for mrw_name.

When not specifying the monitor resource name, specify the category and keyword of the message reception monitor resource by using the -k option.

-h IP Address¶: Specify the IP addresses of EXPRESSCLUSTER servers that receive the processing request.

Up to 32 IP addresses can be specified by separating them with commas.

* If this option is omitted, the processing request is issued to the local server.

-r resource_name¶: Specify the name of a resource that belongs to the target group for the processing request when the --failover option is specified.

-k category[.keyword]¶: For category, specify the category specified for the message receive monitor when the --notice or --clear option is specified.

To specify the keyword of the message receive monitor resource, specify them by separating them with dot after category.

-p port_number¶: Specify the port number.

For port_number, specify the data transfer port number specified for the server that receives the processing request.

The default value, 29002, is used if this option is omitted.

-o logfile_path¶: In logfile_path, specify the path of the file to which to output the detailed log of this command.

The file contains the log of one command execution.

* If this option is not specified on a server where EXPRESSCLUSTER is not installed, the log is always output to the standard output.

-w timeout¶: Specify the command timeout time. The default, 180 seconds, is used if this option is not specified.

A value from 5 to MAXINT can be specified.

Return Value

0	Completed successfully.
Other than 0	Terminated due to a failure.

Notes: When issuing error messages by using the clprexec command, the message reception monitor resources for which executing an action when an error occurs is specified in EXPRESSCLUSTER server must be registered and started.

The server that has the IP address specified for the -h option must satisfy the following conditions:

= EXPRESSCLUSTER X 3.0 or later must be installed.

= EXPRESSCLUSTER must be running.

(When an option other than --script is used)

= mrw must be set up and running.

(When the --notice or --clear option is used)

When using the Limiting the access by using client IP addresses function, add the IP address of the device in which the clprexec command is executed to the IP Addresses of the Accessible Clients list.

For details of the Limiting the access by using client IP addresses function, see "WebManager tab" of "Cluster properties" in "Details of other settings" in the EXPRESSCLUSTER X SingleServerSafe Configuration Guide.

Examples

Example 1: This example shows how to issue a request to fail over the group failover1 to EXPRESSCLUSTER server 1 (10.0.0.1):

# clprexec --failover failover1 -h 10.0.0.1 -p 29002

Example 2: This example shows how to issue a request to fail over the group to which the group resource (exec1) belongs to EXPRESSCLUSTER server 1 (10.0.0.1):

# clprexec --failover -r exec1 -h 10.0.0.1

Example 3: This example shows how to issue a request to execute the script (script1.sh) on EXPRESSCLUSTER server 1 (10.0.0.1):

# clprexec --script script1.sh -h 10.0.0.1

Example 4: This example shows how to issue an error message to EXPRESSCLUSTER server 1 (10.0.0.1):

* mrw1 set, category: earthquake, keyword: scale3
This example shows how to specify a message receive monitor resource name:
# clprexec --notice mrw1 -h 10.0.0.1 -w 30 -p /tmp/clprexec/ lprexec.log
This example shows how to specify the category and keyword specified for the message receive monitor resource:
# clprexec --notice -h 10.0.0.1 -k earthquake.scale3 -w 30 -p /tmp/clprexec/clprexec.log

Example 5: This example shows how to issue a request to change the monitor status of mrw1 to EXPRESSCLUSTER server 1 (10.0.0.1):

* mrw1 set, category: earthquake, keyword: scale3
This example shows how to specify a message receive monitor resource name:
# clprexec --clear mrw1 -h 10.0.0.1
This example shows how to specify the category and keyword specified for the message receive monitor resource:
# clprexec --clear -h 10.0.0.1 -k earthquake.scale3

Error Messages

Message	Cause/solution
rexec_ver:%s	-
%s %s : %s succeeded.	-
%s %s : %s will be executed from now.	Check the processing result on the server that received the request.
%s %s : Group Failover did not execute because Group(%s) is offline.	-
%s %s : Group migration did not execute because Group(%s) is offline.	-
Invalid option.	Check the command argument.
Could not connect to the data transfer servers. Check if the servers have started up.	Check whether the specified IP address is correct and whether the server that has the IP address is running.
Command timeout.	Check whether the processing is complete on the server that has the specified IP address.
All servers are busy.Check if this command is already run.	This command might already be running. Check whether this is so.
%s %s : This server is not permitted to execute clprexec.	Check whether the IP address of the server that executes the command is registered in the list of client IP addresses that are not allowed to connect to the Cluster WebUI.
%s %s : Specified monitor resource(%s) does not exist.	Check the command argument.
%s failed in execute.	Check the status of the EXPRESSCLUSTER server that received the request.

2.16. Controlling reboot count (clpregctrl command)¶

Controls reboot count limitation.

Command line: clpregctrl --get

clpregctrl -g

clpregctrl --clear -t type -r registry

clpregctrl -c -t type -r registry

Description: Displays or initializes the reboot count on a server.

Option

-g, --get¶: Displays reboot count information.

-c, --clear¶: Initializes reboot count.

-t type¶: Specifies the type to initialize the reboot count. The type that can be specified is rc or rm.

-r registry¶: Specifies the registry name. The registry name that can be specified is haltcount.

Return Value

0	Completed successfully.
1	Privilege for execution is invalid
2	Duplicated activation
3	The option is invalid
4	The configuration data is invalid.
10 to 17	Internal Error
20 to 22	Obtaining reboot count information has failed.
90	Allocating memory has failed.
91	Changing the work directory as failed.

Examples

Display of reboot count information

# clpregctrl -g
******************************
-------------------------
type : rc
registry : haltcount
comment : halt count
kind : int
value : 0
default : 0
-------------------------
type : rm
registry : haltcount
comment : halt count
kind : int
value : 3
default : 0
******************************
Command succeeded.(code:0)
#

The reboot count is initialized in the following examples.

Example1: When initializing the count of reboots caused by group resource error:

# clpregctrl -c -t rc -r haltcount
Command succeeded.(code:0)
#

Example2: When initializing the count of reboots caused by monitor resource error:

# clpregctrl -c -t rm -r haltcount
Command succeeded.(code:0)
#

Notes: This command must be executed by a user with the root privilege.

Error Messages

Message	Causes/Solution
Command succeeded.	The command ran successfully.
Log in as root.	You are not authorized to run this command. Log on as root user.
The command is already executed. Check the execution state by using the "ps" command or some other command.	The command is already running. Check the running status by using a command such as ps command.
Invalid option.	Specify a valid option.
Internal error. Check if memory or OS resources are sufficient.	Check to see if the memory or OS resource is sufficient.

2.17. Checking the process health (clphealthchk command)¶

Checks the process health.

Command line: clphealthchk [ -t pm | -t rc | -t rm | -t nm | -h]

Note

This command must be run on the server whose process health is to be checked because this command checks the process health of a single server.

Description: This command checks the process health of a single server.

Option

None¶: Checks the health of all of pm, rc, rm, and nm.

-t <process>¶

process

pm
Checks the health of pm.

rc
Checks the health of rc.

rm
Checks the health of rm.

nm
Checks the health of nm.

-h¶: Displays the usage.

Return Value

0	Normal termination
1	Privilege for execution is invalid
2	Duplicated activation
3	Initialization error
4	The option is invalid
10	The process stall monitoring function has not been enabled.
11	The cluster is not activated (waiting for the cluster to start or the cluster has been stopped.)
12	The cluster daemon is suspended
100	There is a process whose health information has not been updated within a certain period. If the -t option is specified, the health information of the specified process is not updated within a certain period.
255	Other internal error

Examples

Example 1: When the processes are healthy

# clphealthchk
pm OK
rc OK
rm OK
nm OK

Example 2: When clprc is stalled

# clphealthchk
pm OK
rc NG
rm OK
nm OK

# clphealthchk -t rc
rc NG

Example 3: When the cluster has been stopped

# clphealthchk
The cluster has been stopped

Remarks: If the cluster has been stopped or suspended, the process is also stopped.

Notes: Run this command as the root user.

Error Messages

Message	Cause/Solution
Log in as root.	You are not authorized to run this command. Log on as the root user.
Initialization error. Check if memory or OS resources are sufficient.	Check to see if the memory or OS resource is sufficient.
Invalid option.	Specify a valid option.
The function of process stall monitor is disabled.	The process stall monitoring function has not been enabled.
The cluster has been stopped.	The cluster has been stopped.
The cluster has been suspended.	The cluster has been suspended.
This command is already run.	The command has already been started. Check the running status by using a command such as ps command.
Internal error. Check if memory or OS resources are sufficient.	Check to see if the memory or OS resource is sufficient.

2.18. Displaying the cluster statistics information (clpperfc command)¶

the clpperfc command displays the cluster statistics information.

Command line: clpperfc --starttime -g group_name

clpperfc --stoptime -g group_name

clpperfc -g [group_name]

clpperfc -m monitor_name

Description

This command displays the median values (millisecond) of the group start time and group stop time.

This command displays the monitoring processing time (millisecond) of the monitor resource.

Option

--starttime -g group_name¶: Displays the median value of the group start time.

--stoptime -g group_name¶: Displays the median value of the group stop time.

-g [group_name]¶

Displays the each median value of the group start time and group stop time.

If groupname is omitted, it displays the each median value of the start time and stop time of all the groups.

-m monitor_name¶: Displays the last monitor processing time of the monitor resource.

Return value

0	Normal termination
1	Invalid command option
2	User authentication error
3	Configuration information load error
4	Configuration information load error
5	Initialization error
6	Internal error
7	Internal communication initialization error
8	Internal communication connection error
9	Internal communication processing error
10	Target group check error
12	Timeout error

Example of Execution

(when displaying the median value of the group start time)

# clpperfc --starttime -g failover1
200

Example of Execution

(when displaying each median value of the start time and stop time of the specific group)

# clpperfc -g failover1
            start time    stop time
failover1          200          150

Example of Execution

(when displaying the monitor processing time of the monitor resource)

# clpperfc -m monitor1
100

Remarks

The time is output in millisecond by this commands.

If the valid start time or stop time of the group was not obtained, - is displayed.

If the valid monitoring time of the monitor resource was not obtained, 0 is displayed.

Notes: Execute this command as a root user.

Error Messages

Message	Cause/Solution
Log in as root.	Run this command as the root user.
Invalid option.	The command option is invalid. Check the command option.
Command timeout.	Command execution timed out .
Internal error.	Check if memory or OS resources are sufficient.

2.19. Checking the cluster configuration information (clpcfchk command)¶

This command checks the cluster configuration information.

Command line: clpcfchk -o path [-i conf_path]

Description: This command checks the validness of the setting values based on the cluster configuration information.

Option

-o path¶: Specifies the directory to store the check results.

-i conf_path¶

Specifies the directory which stored the configuration information to check.

If this option is omitted, the applied configuration information is checked.

Return Value

0	Normal termination
Other	than 0 Termination with an error

Example of Execution (when checking the applied configuration information)

# clpcfchk -o /tmp
server1 : PASS

Example of Execution (when checking the stored configuration information)

# clpcfchk -o /tmp -i /tmp/config
server1 : PASS

Execution Result

For this command, the following check results (total results) are displayed.

Check Results (Total Results)	Description
PASS	No error found.
FAIL	An error found. Check the check results.

Remarks: Only the total results of each server are displayed.

Notes

Run this command as a root user.

When checking the configuration information exported through Cluster WebUI, decompress it in advance.

Error Messages

Message	Cause/Solution
Log in as root.	Log in as a root user.
Invalid option.	Specify a valid option.
Could not opened the configuration file. Check if the configuration file exists on the specified path.	The specified path does not exist. Specify a valid path.
Server is busy. Check if this command is already run.	This command has been already activated.
Failed to obtain properties.	Failed to obtain the properties.
Failed to check validation.	Failed to check the cluster configuration.
Internal error. Check if memory or OS resources are sufficient.	The amount of memory or OS resources may be insufficient. Check for any insufficiency.

2.20. Adding a firewall rule (clpfwctrl command)¶

Adds or deletes a firewall rule on servers for EXPRESSCLUSTER.

Command line: clpfwctrl --add [--zone=<ZONE>]

clpfwctrl --remove

clpfwctrl --help

Description

Note

Before executing this command, enable the server firewall.

Note

This command adds a rule to or deletes it from a firewall zone on a single server.

Note

Execute this command immediately after installing EXPRESSCLUSTER and directly after applying configuration data.

Note

This command supports only environments where the firewall-cmd and firewall-offline-cmd commands can be used.

A rule can be added to a firewall zone for accessing port numbers for EXPRESSCLUSTER, and the added rule can be deleted from the zone.
For more information on port numbers to be specified with this command, and for that on protocols, see "EXPRESSCLUSTER X SingleServerSafe Installation Guide" -> "About EXPRESSCLUSTER X SingleServerSafe" -> "Preparing and verifying the server environment before installation" -> "Verifying the firewall settings (Required)".
Add a rule with the following name to a firewall zone. If the rule name is already used, first delete it, then add it again. Do not change the rule name.

Rule name
- clusterpro

Option

--add [--zone=<ZONE>]¶: Adds a firewall rule, to a zone (if specified). If no zone is specified, the rule is added to the default zone.

--remove¶: Deletes the added firewall rule.

--help¶: Displays the usage.

Return value

0	Success
Other than 0	Failure

Notes: Execute this command as root.

This command does not add an outbound firewall rule. Adding it requires a separate procedure.

Once a JVM monitor resource is registered, this command always allows the port number for managing the resource.

Executing this command discards the firewall configuration that is temporarily set on the memory.

Example of Execution

Adding a rule to the default zone:

# clpfwctrl.sh --add
Command succeeded.

Example of Execution

Adding a rule to the home zone:

# clpfwctrl.sh --add --zone=home
Command succeeded.

Example of Execution

Deleting the added rule:

# clpfwctrl.sh --remove
Command succeeded.

Error Messages

Message	Cause/Solution
Log in as root.	Log in as a user with root privileges.
Invalid option.	Specify the right option.
Failed to register rule(CLUSTERPRO). Invalid port.	Check the configuration data, which includes an invalid port number.
Failed to register rule(CLUSTERPRO). Invalid zone.	Check the zone name, which is invalid.
Unsupported environment.	The OS is unsupported.
Could not read xmlpath. Check if xmlpath exists on the specified path. (%1)	Check if the xml path exists in the configuration data. %1: xml path
Could not opened the configuration file. Check if the configuration file exists on the specified path. (%1)	Check if the policy file exists. %1: xml path
Could not read type. Check if type exists on the policy file. (%1)	Check if the policy file exists. %1: xml path
not exist xmlpath. (%1)	Check if the xml path exists in the configuration data. %1: xml path
Failed to obtain properties. (%1)	Check if the xml path exists in the configuration data. %1: xml path
Not exist java install path. (%1)	Check if the Java installation path exists. %1: Java installation path
Internal error. Check if memory or OS resources are sufficient. (%1)	The possible cause is insufficient memory or insufficient OS resources. Check if these two are sufficient. %1: xml path

3. Notes and restrictions¶

This chapter provides information on known problems and how to troubleshoot the problems.

This chapter covers:

3.1. After starting operating EXPRESSCLUSTER X SingleServerSafe

3.1. After starting operating EXPRESSCLUSTER X SingleServerSafe¶

This section provides notes on situations you might encounter after starting to operate EXPRESSCLUSTER.

3.1.1. Messages displayed when the driver is loaded¶

When the clpka driver is loaded, the following message may be displayed in the console or syslog. This symptom is not an error.

kernel: clpka: no version for "struct_module" found: kernel tainted.
kernel: clpka: module license 'unspecified' taints kernel.

3.1.2. ipmi messages¶

If IPMI is used as the user-mode monitor resource, the following many kernel module warning logs are output to syslog:

modprobe: modprobe: Can't locate module char-major-10-173

To avoid this log output, rename /dev/ipmikcs.

3.1.3. Restrictions during recovery operation¶

When you have configured a group resource (EXEC resource) as a recovery target in the settings of error detection by a monitor resource, and the monitor resource detects an error, do not perform the following commands or the controls of servers or groups by the Cluster WebUI while recovery (reactivation -> final action) is ongoing.

Stopping or suspending a server
Starting or stopping a group

If you perform the above-mentioned operations while recovery caused by detection of an error by a monitor resource is in progress, other group resources of the group with an error may not stop.

However, you can perform them when the final action is completed.

3.1.4. Executable files and script files not described in the Command Reference¶

The installation directory contains executable files and script files that are not described in "2. EXPRESSCLUSTER X SingleServerSafe command reference" in this guide. Do not execute these files by using any program other than EXPRESSCLUSTER X SingleServerSafe.

Any problems caused by not using EXPRESSCLUSTER will not be supported.

3.1.5. Messages displayed when logs are collected¶

When you collect logs, you may find the following message on the console. This does not mean failure. The logs are collected normally.

hd#: bad special flag: 0x03
ip_tables: (C) 2000-2002 Netfilter core team

(The name of the IDE device that exists on the server is stored in hd#.)

kernel: Warning: /proc/ide/hd?/settings interface is obsolete, and will be removed soon!

3.1.6. Service start/stop scripts¶

For an init.d environment, the service start and stop scripts output an error in the following cases. An error is not output for a systemd environment.

Immediately after the EXPRESSCLUSTER Server is installed (for SUSE Linux)

When the OS is shut down, the service stop scripts below output an error. This error is output because services are not running and does not indicate an actual problem.
- clusterpro_alertsync
- clusterpro_webmgr
- clusterpro
- clusterpro_api
- clusterpro_ib
- clusterpro_trn
- clusterpro_evt
OS shutdown after manually stopping a service (for SUSE Linux)

After a service is stopped using the clpcl command or the Cluster WebUI, the stop script for the service that stopped when the OS shut down outputs an error. This error is output because the service stopped and does not indicate an actual problem.
- clusterpro

In the following case, the service stop scripts are executed in the incorrect order:

OS shutdown after all services are disabled by executing chkconfig --del name

After the EXPRESSCLUSTER services are disabled, they are stopped in the incorrect order when the OS shuts down. This occurs because the EXPRESSCLUSTER services disabled when the OS shut down are not stopped.

If the server is shut down by the Cluster WebUI or by an EXPRESSCLUSTER command such as the clpstdn command, the EXPRESSCLUSTER services stopping in the incorrect order does not cause a problem.

3.1.7. Checking the service status when systemd is used¶

For a systemd environment, the status of services output by the systemctl command may not reflect the actual status of the cluster.

Use the clpstat command and Cluster WebUI to check the cluster status.

3.1.8. Script files used in EXEC resources¶

The script files used in the EXEC resources are stored in the following directory on the server:

/installation path/scripts/group-name/EXEC resource-name/

If the following changes are made in configuration change, the pre-change script files are not deleted from the server.

When the EXEC resource is deleted or renamed
When a group that belongs to the EXEC resource is deleted or renamed

Old EXEC resource scripts can be deleted when unnecessary.

3.1.9. Monitor resources that monitor active resources¶

When monitor resources that monitoring timing is "Active" have suspended and resumed, the following restriction apply:

In case stopping target resource after suspending monitor resource, monitor resource becomes suspended. As a result, monitoring restart cannot be executed.
In case stopping or starting target resource after suspending monitor resource, monitoring by monitor resource starts when target resource starts.

3.1.10. Notes on the Cluster WebUI¶

If the Cluster WebUI is operated in the state that it cannot communicate with the connection destination, it may take a while until the control returns.
When going through the proxy server, make the settings for the proxy server be able to relay the port number of the Cluster WebUI.
When going through the reverse proxy server, the Cluster WebUI will not operate properly.
When updating EXPRESSCLUSTER X SingleServerSafe, close all running browsers. Clear the browser cache and restart the browser.
Cluster configuration data created using a later version of this product cannot be used with this product.
Closing the Web browser (by clicking Exit from the menu), the dialog box to confirm to save may be displayed.

When you continue to edit, click the Stay on this page button.
Reloading the Web browser (by selecting Refresh button from the menu or tool bar), the dialog box to confirm to save may be displayed.

When you continue to edit, click the Stay on this page button.
For notes and restrictions of Cluster WebUI other than the above, see the online manual.

3.1.11. System monitor resources,Process resource monitor resource¶

To change a setting, the cluster must be suspended.
System monitor resources do not support a delay warning for monitor resources.
For the SELinux setting, set permissive or disabled.

The enforcing setting may disable the communication needed by EXPRESSCLUSTER.
If the date or time of the OS has been changed while System Resource Agent is running, resource monitoring may operate incorrectly as described below since the timing of analysis which is normally done at 10 minute intervals may differ the first time after the date or time is changed. If either of the following occur, suspend and resume cluster.
- No error is detected even after the specified duration for detecting errors has passed.
- An error is detected before the specified duration for detecting errors has elapsed.
Up to 64 disks that can be monitored by the disk resource monitoring function of System monitor resources.

3.1.12. JVM monitor resources¶

When restarting the monitoring-target Java VM, suspend or shut down the cluster before restarting the Java VM.
To change a setting, the cluster must be suspended.
JVM monitor resources do not support a delay warning for monitor resources.
When changing the language (for example, from Japanese to Chinese) set to the Cluster WebUI (Cluster Properties - Info tab - Language) after JVM monitor resource registration, delete the registered JVM monitor resource, and then register it again.

3.1.13. HTTP monitor resource¶

The HTTP monitor resource uses any of the following OpenSSL shared library symbolic links:
- libssl.so
- libssl.so.1.1 (OpenSSL 1.1.1 shared libraly)
- libssl.so.10 (OpenSSL 1.0 shared libraly)
- libssl.so.6 (OpenSSL 0.9 shared libraly)
The above symbolic links may not exist depending on the OS distribution or version, or the package installation status.

If the above symbolic links cannot be found, the following error occurs in the HTTP monitor resource.
```
Detected an error in monitoring<Module Resource Name>. (1 :Can not found library. (libpath=libssl.so, errno=2))
```
For this reason, if the above error occurred, be sure to check whether the above symbolic links exit in /usr/lib or /usr/lib64.

If the above symbolic links do not exit, create the symbolic link libssl.so, as in the command example below.

Command example:
```
cd /usr/lib64 # Move to /usr/lib64.
ln -s libssl.so.1.0.1e libssl.so # Create a symbolic link.
```

Convention	Usage	Example
Bold	Indicates graphical objects, such as fields, list boxes, menu selections, buttons, labels, icons, etc.	In User Name, type your name. On the File menu, click Open Database.
Angled bracket within the command line	Indicates that the value specified inside of the angled bracket can be omitted.	`clpstat -s [-h host_name]`
#	Prompt to indicate that a Linux user has logged in as root user.	`# clpcl -s -a`
Monospace	Indicates path names, commands, system output (message, prompt, etc), directory, file names, functions and parameters.	`/Linux/5.0/en/server/`
bold	Indicates the value that a user actually enters from a command line.	Enter the following: clpcl -s -a
`italic`	Indicates that users should replace italicized part with values that they are actually working with.	`rpm -i expressclssss-<version_number> -<release_number>.x86_64.rpm`