1. Preface¶
This document "EXPRESSCLUSTER X for Linux SAP NetWeaver System Configuration Guide" describes how to create and start a cluster for SAP NetWeaver.
1.1. Who Should Use This Guide¶
1.2. How This Guide is Organized¶
This guide consist of the following two documents:
"EXPRESSCLUSTER X for Linux SAP NetWeaver System Configuration Guide"
"EXPRESSCLUSTER X for Linux SAP NetWeaver Configuration Example"
1.3. Conventions¶
In this guide, Note, Important, See also are used as follows:
Note
Used when the information given is important, but not related to the data loss and damage to the system and machine.
Important
Used when the information given is necessary to avoid the data loss and damage to the system and machine.
See also
Used to describe the location of the information given at the reference destination.
The following conventions are used in this guide.
Convention |
Usage |
Example |
---|---|---|
Bold |
Indicates graphical objects, such as text boxes, list boxes, menu selections, buttons, labels, icons, etc. |
Click Start.
Properties dialog box
|
Angled bracket within the command line |
Indicates that the value specified inside of the angled bracket can be omitted. |
|
# |
Prompt to indicate that a Linux user has logged on as root user. |
|
Monospace |
Indicates path names, commands, system output (message, prompt, etc.), directory, file names, functions and parameters. |
|
bold |
Indicates the value that a user actually enters from a command line. |
Enter the following:
# clpcl -s -a
|
|
Indicates that users should replace italicized part with values that they are actually working with. |
|
In the figures of this guide, this icon represents EXPRESSCLUSTER.
1.4. EXPRESSCLUSTER X Documentation Set (for Internal Version 3.3.x/4.0.x)¶
The EXPRESSCLUSTER X manuals consist of the following four guides. The title and purpose of each guide is described below:
EXPRESSCLUSTER X Getting Started Guide
This guide is intended for all users. The guide covers topics such as product overview, system requirements, and known problems.
EXPRESSCLUSTER X Installation and Configuration Guide
This guide is intended for system engineers and administrators who want to build, operate, and maintain a cluster system. Instructions for designing, installing, and configuring a cluster system with EXPRESSCLUSTER are covered in this guide.
EXPRESSCLUSTER X Reference Guide
This guide is intended for system administrators. The guide covers topics such as how to operate EXPRESSCLUSTER, function of each module, maintenance-related information, and troubleshooting. The guide is supplement to the Installation and Configuration Guide.
EXPRESSCLUSTER X Integrated WebManager Administrator's Guide
This guide is intended for system administrators who manage cluster systems using EXPRESSCLUSTER with Integrated WebManager, and also intended for system engineers who introduce Integrated WebManager. This guide describes detailed issues necessary for introducing Integrated WebManager in the actual procedures.
1.5. EXPRESSCLUSTER X Documentation Set (for Internal Version 4.1.xor later)¶
The EXPRESSCLUSTER X manuals consist of the following five guides. The title and purpose of each guide is described below:
EXPRESSCLUSTER X Getting Started Guide
This guide is intended for all users. The guide covers topics such as product overview, system requirements, and known problems.
EXPRESSCLUSTER X Installation and Configuration Guide
This guide is intended for system engineers and administrators who want to build, operate, and maintain a cluster system. Instructions for designing, installing, and configuring a cluster system with EXPRESSCLUSTER are covered in this guide.
EXPRESSCLUSTER X Reference Guide
This guide is intended for system administrators. The guide covers topics such as how to operate EXPRESSCLUSTER, function of each module and troubleshooting. The guide is supplement to the Installation and Configuration Guide.
EXPRESSCLUSTER X Maintenance Guide
This guide is intended for administrators and for system administrators who want to build, operate, and maintain EXPRESSCLUSTER-based cluster systems. The guide describes maintenance-related topics for EXPRESSCLUSTER.
EXPRESSCLUSTER X Hardware Feature Guide
This guide is intended for administrators and for system engineers who want to build EXPRESSCLUSTER-based cluster systems. The guide describes features to work with specific hardware, serving as a supplement to the Installation and Configuration Guide.
1.7. Terminology in this guide¶
Terminology used in this guide
- This product
- For EXPRESSCLUSTER X 3.3EXPRESSCLUSTER X for Linux SAP NetWeaver
- For EXPRESSCLUSTER X 4.0 or laterEXPRESSCLUSTER X for Linux SAP NetWeaver/SAP HANA
- Configuration Guide
EXPRESSCLUSTER X for Linux SAP NetWeaver System Configuration Guide
- Configuration Example
EXPRESSCLUSTER X for Linux SAP NetWeaver Configuration Example
- Connecter for Sap
The connecter which links with SAP included in this product.
- SAP NW
SAP NetWeaver
- ASCS
ABAP SAP Central Services Instance
- ERS
Enqueue Replication Server
- PAS
Primary Application Server
- AAS
Additional Application Server
- DA
Diagnostics Agent
- ENSA
Standalone Enqueue Server
- ENSA2
Standalone Enqueue Server 2
2. Overview of the cluster system¶
2.1. Overview of SAP NW Cluster¶
2.1.1. Functional Overview¶
A cluster with the following configuration can be built by combining SAP NW and EXPRESSCLUSTER.
2.1.1.1. SAP NW cluster configuration using EXPRESSCLUSTER¶
ABAP SAP Central Services Instance (hereafter ASCS)
(With ENSA2 used) ERS
Configure the following components as failover groups for a single server configuration in which failover groups operate on each node.
(With ENSA used) ERS
Primary Application Server Instance (hereafter PAS)
Additional Application Server Instance (hereafter AAS)
Diagnostics Agent (hereafter DA)
saphostexec
The diagram below shows the configuration with ENSA used.
The diagram below shows the configuration with ENSA2 used.
2.1.1.2. Dependency between failover groups¶
2.1.1.3. SAP NW monitoring using EXPRESSCLUSTER¶
In addition to the monitoring functions provided by EXPRESSCLUSTER the SAP NW cluster system uses a monitoring package that supports the SAP system and an SAP NW-specific monitoring to monitor the SAP NW components for response errors and hang-ups.
2.1.1.4. Instance number configuration of the SAP NW components¶
2.1.1.5. Integration between SAP NW and EXPRESSCLUSTER¶
User requests to SAP NW are sent to EXPRESSCLUSTER via the Connecter for SAP (clp_shi_connector). The EXPRESSCLUSTER cluster is operated by SAP NW.
2.1.1.6. Illustration of exclusive control of ASCS/ERS instance by EXPRESSCLUSTER¶
Start both ASCS and ERS instances on different nodes. Start ERS instance on only one node. If ENSA is used, start the failover group for exclusive control on all nodes except the node which ERS instance starts.
EXPRESSCLUSTER handles failover process of the ASCS instance as follows.
If ENSA is used, fail over the ASCS instance to the node where the ERS instance was started before. If ENSA2 is used, fail over the ASCS instance to the node determined by the startup priority set in the failover group for ASCS.
If ENSA is used, ERS instance stops based on the SAP NW specifications automatically by ASCS instance, after the failover of it is executed. If ENSA2 is used and the ERS instance has already been started on the failover target node of ASCS instance, the custom monitor resource of EXPRESSCLUSTER will execute the failover of ERS instance to another node.
If ENSA is used, EXPRESSCLUSTER works as follows once the ERS instance got stopped based on the SAP NW specifications.
Starts of the failover group for exclusive control on the node where ASCS instance was moved to .
Starts the ERS instance on the node that the ASCS instance is currently not working.
Stops the failover group for exclusive control on the node where the ERS instance just started.
The above mechanism of exclusive control of both ASCS/ERS instances by EXPRESSCLUSTER works similarly in the case of more than 3 nodes.
2.1.1.7. Note on manual operation of the ERS instance¶
The ERS instance is used for the replication of the lock table from the ASCS instance. To ensure its redundancy the ERS instance must work on the node where the ASCS instance is not running. The ERS instance should not even manually be launched on the node where the ASCS instance is running. Additionally the ERS instance should not be launched on more than two nodes at same time.
The failover group of the ERS instance is not restarted automatically, when the node where the ERS instance was working recovers from a failure. After validating the health of the node a manual restart of the ERS instance failover group is required.
2.1.1.8. ENSA2 (Internal Version 4.1.0-1 or later)¶
In order to use ENSA2, SAP NetWeaver Application Server for ABAP 7.52 or later is required.
Also, for SAP NetWeaver Application Server for ABAP 7.52, ENSA can be used.
Differences in failover groups for the ERS instance are as follows:
Item |
ENSA |
ENSA2 |
---|---|---|
Number of failover groups |
One for each node |
One for each cluster |
Active node in a failover group |
Only the nodes belonging to the failover group |
Possible to start for all nodes. However, set the startup priority in reverse order of the startup priority of the failover group for ASCS. |
Startup attribute |
Manual startup |
Automatic startup |
Resources composing the failover group
|
The number in <> is the depth of dependency.
<1> EXEC resource for ERS instance service
<1> EXEC resource for ERS instance
|
The number in <> is the depth of dependency.
<0> Floating IP resource for ERS instance
<1> EXEC resource for ASCS startup check
<2> EXEC resource for ERS instance service
<2> EXEC resource for ERS instance
|
2.1.2. Operating Environment¶
This section describes the OS and SAP NW versions on which the operation of the Connector for SAP has been verified.
x86_64
NW Version / ABAP Platform
SAP Kernel Version
EXPRESSCLUSTERVersionOS
Cluster Configuration
7.4 742745 3.3.2-1 or later Red Hat Enterprise Linux 7.0Red Hat Enterprise Linux 7.1 NAS connection, SAN connection, shared disk type7.5
745749 3.3.3-1 or later Red Hat Enterprise Linux 7.2Red Hat Enterprise Linux 7.3SUSE Linux Enterprise Server 12 SP1 NAS connection, SAN connection, shared disk type7.5
7457493.3.5-1 or later
Red Hat Enterprise Linux 7.4
NAS connection, SAN connection, shared disk type
NW Version / ABAP Platform
SAP Kernel Version
EXPRESSCLUSTERVersionOS
Cluster Configuration
7.5
745749753 4.0.0-1 or later Red Hat Enterprise Linux 7.3Red Hat Enterprise Linux 7.4SUSE LINUX Enterprise Server 12 SP1 NAS connection, SAN connection, shared disk type7.52
753
4.1.0-1 or later
Red Hat Enterprise Linux 7.5
NAS connection, SAN connection, shared disk type
7.52
753
4.2.0-1 or later
Red Hat Enterprise Linux 7.6Red Hat Enterprise Linux 7.7NAS connection, SAN connection, shared disk type
1809
773
4.1.0-1 or later
Red Hat Enterprise Linux 7.6Red Hat Enterprise Linux 7.7NAS connection, SAN connection, shared disk type
2020
781
4.3.0-1 or later
Red Hat Enterprise Linux 8.2
NAS connection, SAN connection, shared disk type
2021
785
5.0.0-1 or later
Red Hat Enterprise Linux 8.2
NAS connection, SAN connection, shared disk type
IBM POWER
NW Version / ABAP Platform
SAP Kernel Version
EXPRESSCLUSTERVersionOS
Cluster Configuration
7.5
745749753 3.3.5-1 or later4.0.0-1 or later SUSE Linux Enterprise Server 11 SP4 NAS connection, SAN connection, shared disk type7.52
753
4.1.0-1 or later
SUSE Linux Enterprise Server 11 SP4
NAS connection, SAN connection, shared disk type
Note the following:
When using a LAN heartbeat, be sure to use a LAN heartbeat resource. Do not use a kernel mode LAN heartbeat resource.
When using a user space monitor resource, specify softdog for Method.
When performing shutdown monitoring, specify softdog for Method.
For the hardware and software requirements of SAP NW, see the SAP NW manuals.
2.1.3. Building Procedure¶
The sequence of building the SAP NW cluster is shown below.
Installation and basic setup of Linux OS
Setup of shared disk and network
Installation of EXPRESSCLUSTER
Building of cluster nodes with NAS resource (for EXPRESSCLUSTER internal version 3.x/4.x), EXEC resource (for EXPRESSCLUSTER internal version 5.x) and floating IP
Installation of SAP NW
Setup of SAP NW cluster in EXPRESSCLUSTER
2.1.4. HA Database for SAP NW¶
Since SAP NW can run on several database technologies, e.g. SAP HANA, SAP MaxDB, IBM DB2, Oracle, Microsoft SQLSERVER, this guide assumes there is already a high available database setup in place. If you need help how create an HA setup for your database scenario please follow related EXPRESSCLUSTER documents on https://www.nec.com/en/global/prod/expresscluster/.
Throughout this document the HA database setup will be referred to as "database".
2.1.5. New features and improvements¶
The following features and improvements have been released.
EXPRESSCLUSTER Internal Version 3.3.x
InternalVersion Contents3.3.4-1
SAP Connector for SAP NetWeaver supports the following OS:- Red Hat Enterprise Linux 7.3- SUSE Linux Enterprise Server 12 SP13.3.5-1
SAP Connector for SAP NetWeaver supports the following OS:- Red Hat Enterprise Linux 7.4- Novell SUSE LINUX Enterprise Server 11 SP4 (IBM POWER)-
Verified the cluster configuration with two NFS servers. Please refer to "3.1.2. When using two NFS servers"
EXPRESSCLUSTER Internal Version 4.0.xor later
InternalVersion Contents4.1.0-1
SAP Connector for SAP NetWeaver supports the following OS:- Red Hat Enterprise Linux 7.54.1.0-1
The Connector for SAP for SAP NetWeaver supports the following SAP NetWeaver:- SAP NetWeaver Application Server for ABAP 7.524.1.0-1
The Connector for SAP/the bundled scripts for SAP NetWeaver supports the following:- Maintenance mode- Standalone Enqueue Server 24.1.0-1
Improved the bundled scripts for SAP NW as follows:- Added parameters to clp_shi_connector.conf and some bundled scripts in order to facilitate the customization- Improved the quality of process alive monitoring of SAP Host Agent4.3.0-1
SAP Connector for SAP NetWeaver now supports the following SAP NetWeaver:- ABAP Platform 2020
EXPRESSCLUSTER Internal Version 5.0.xor later
InternalVersion Contents5.0.0-1
SAP Connector supports the following environment:- ABAP Platform 2021
2.1.6. Corrected information¶
Modification has been performed on the following minor versions.
EXPRESSCLUSTER Internal Version 3.3.x
Version in which the problem has been solved/ Version in which the problem occurred Phenomenon Level Occurrence condition/Occurrence frequency 3.3.5-1/ 3.3.2-1 to 3.3.4-2 A failover group for the ERS instance of SAP NetWeaver may not link with a failover group for exclusive control in the same node.* To deal with this problem, it is required to replace the script manually. The sample script can be obtained from the support portal (content ID: 9510100151). M Rarely occurs when starting up a failover group for ASCS instances.
EXPRESSCLUSTER Internal Version 4.0.xor later
Version in which the problem has been solved/ Version in which the problem occurred Phenomenon Level Occurrence condition/Occurrence frequency 4.1.0-1/ 3.3.0-1, 4.0.0-1 At the time of failure detection of a custom monitor resource using the sample script for SAP NW, start processing of SAP instance service is performed during stop processing of SAP instance service. S Occurs when it takes time to stop the SAP instance service. 4.1.0-1/ 3.3.0-1, 4.0.0-1 If any languages other than English is selected in the language settings of EXPRESSCLUSTER, SAP Connector for SAP NetWeaver does not operate normally. S If any languages other than English is selected, this problem inevitably occurs. 4.3.0-1/ 4.1.0-1 to 4.2.2-1 Regarding SAP Connector for SAP NetWeaver, an unnecessary error message is outputted to syslog. S This problem occurs with the following conditions: The SMM_PATH parameter is set in clp_shi_connector.conf, and the SAP instance is started/stopped.
2.2. OS Installation and basic settings¶
Refer to the following SAP NOTEs for SAP NW installation and needed software on Node#1 and Node#2.
SAP NOTEs
#0171356 : SAP software on Linux: General information
#0784391 : SAP support terms and 3rd-party Linux kernel drivers
#1391070 : Linux UUID solutions
#0146003 : Application servers cannot be started
For examples of settings used in this manual please refer to "An example of setting OS" in the supplied SAP NetWeaver Configuration Example document.
3. Installation of EXPRESSCLUSTER and SAP NW¶
3.1. Configuration Consisting of a SAP NW Cluster and NFS Server¶
In this guide, a SAP NW cluster consists of an active node (Node#1) and standby node (Node#2). In addition, an NFS server is used to store SAP NW shared data and so on. Therefore, two nodes for a SAP NW cluster and one or more NFS server are required. If you want to make the NFS server redundant, configure a cluster with two or more NFS servers.
3.1.1. When using a single NFS server¶
The following figure shows a configuration using a single NFS server (Node#3).
In this configuration, NW shared data and so on are provided from one NFS server. Therefore, this NFS server is a single point of failure of the SAP NW cluster.
3.1.2. When using two NFS servers¶
The following figure shows a configuration using two nodes (Node#3, Node#4) as an NFS server.
It is also necessary to configure two nodes used as an NFS server as a cluster in a unidirectional standby configuration by using EXPRESSCLUSTER. SAP NW shared data and so on (ASCS data, and commands and files in Figure 3.3 System Configuration Using Two Nodes as an NFS Server ) are stored in a shared disk or mirror disk to make information consistent between these two nodes. In this configuration, the NFS service can be failed over between two nodes. Therefore, the NFS server is not a single point of failure of the SAP NW cluster.
Please refer to "Sample configuration of EXPRESSCLUSTER in an NFS cluster" in the SAP NetWeaver Configuration Example document for further details.
3.1.3. Measures to be taken when monitoring fails due to NFS disconnection¶
The EXPRESSCLUSTER custom monitor resource that monitors SAP NW components uses the SAP NW commands installed in the NFS server. Therefore, if NFS connection between a SAP NW cluster and NFS server is disconnected, the custom monitor resource cannot access the commands. This causes a monitoring process not to be complete. If this status lasts longer than the time set to Timeout of the custom monitor resource, the monitoring process fails.
To reduce the possibility that a monitoring process fails, set up a SAP NW cluster so that the disk monitor resource checks whether access to the NFS connection destination is available and the custom monitor resource monitors SAP NW components only when no error is detected by the disk monitor resource.
For how to set up the custom monitor resource and disk monitor resource, please refer to "3.5.2. Setup of Monitor Resources"
If monitoring SAP NW components frequently fails due to NFS disconnection, take the following measures:
Improving the network status between a SAP NW cluster and NFS server
Extending Timeout of the custom monitor resource
When monitoring SAP NW components fails due to NFS disconnection, there is a possibility that the SAP NW components are in abnormal state and cannot be restarted. In such a case, restart the node including these SAP NW components from EXPRESSCLUSTER.
3.3. Preparation of EXPRESSCLUSTER¶
Please refer to the "Installation and Configuration Guide" for additional information how to build an EXPRESSCLUSTER environment.
Create a cluster environment with two nodes and a NFS server in the order shown below.
Completely install EXPRESSCLUSTER, build a cluster with a NAS resource (for EXPRESSCLUSTER internal version 3.x/4.x), a EXEC resource (for EXPRESSCLUSTER internal version 5.x) and a floating IP, and start EXPRESSCLUSTER before installing SAP NW.
Preparations before installing SAP NW
Installation of EXPRESSCLUSTER
License registration
Creation of cluster configuration information
Creating a cluster
Creating failover groups
Adding additional group resources
Specifying dependencies between failover groups
For details of settings used in this manual please refer to "Sample configuration of EXPRESSCLUSTER in a SAP NW cluster" in the supplied SAP NetWeaver Configuration Example document.
After completing the above processes continue with "3.4. Setup of SAP NW Environment" and "3.5. Setup of EXPRESSCLUSTER"
3.3.1. Installation of EXPRESSCLUSTER¶
If an older version of this product is already installed, back up the bundled scripts and the configuration file of the Connecter for SAP. The following shows an example to store the bundled scripts installed on /root/sample to /home/backup.
# cp -p /opt/nec/clusterpro/etc/clp_shi_connector.conf /home/backup # cp -rp /root/sample /home/backup # rpm -e expresscls spnw-<Version of EXPRESSCLUSTER>.x86_64.rpm
After the installation of EXRESSCLUSTER has finished please enter the following command and install the Connecter for SAP. The same rpm package is used for both x86_64 and IBM POWER.
# rpm -i expresscls_spnw-<Version of EXPRESSCLUSTER>.x86_64.rpm
3.3.2. License registration¶
The license must be registered before you can use EXPRESSCLUSTER.
For details on the license registration please refer to the following document:
This product contains the following four licenses.
Licensed Product Name |
---|
EXPRESSCLUSTER X for Linux |
EXPRESSCLUSTER X Database Agent for Linux |
EXPRESSCLUSTER X System Resource Agent for Linux |
EXPRESSCLUSTER X File Server Agent for Linux |
3.3.3. Creating a cluster¶
Create a cluster from EXPRESSCLUSTER WebManager (hereafter referred to as WebManager) (for Internal Version 3.3.x/4.0.x)/Cluster WebUI (for Internal Version 4.1.xor later).
For details how to create a cluster please refer to the following document:
Note
3.3.4. Creating failover groups¶
Create the failover groups to which each node belongs from WebManager (for Internal Version 3.3.x/4.0.x)/Cluster WebUI (for Internal Version 4.1.xor later).
For details how to create a failover group please refer to the following document:
Now please create the following failover groups:
ASCS instance
ERS instance (for ENSA2 configuration)
ERS1 instance (for ENSA configuration)
ERS2 instance (for ENSA configuration)
PAS instance
AAS instance
DA1 instance
DA2 instance
hostexec1
hostexec2
Exclusive1 (for ENSA configuration)
Exclusive2 (for ENSA configuration)
The numbers included in the failover group names (e.g. 1 in ERS1) refer to the node on which this failover group is running. This means DA1 is on Node#1, DA2 on Node#2, and so on.
When ENSA2 is used, create only one failover group for the ERS instance.
3.3.4.1. Failover group for exclusive control¶
If ENSA is used, it is necessary to create a failover group for the exclusive control of the ASCS and ERS instances. This section describes how to create a failover group.
<common failover group name><number>
Example in this manual
Exclusive-Group1 (Node#1)
Exclusive-Group2 (Node#2)
Note
The failover group name must not contain any spaces.
Note
If the failover group name does not conform to the naming conventions, exclusive control of ASCS/ERS instance cannot function normally.
For exclusive control of ASCS/ERS instance please refer to "2.1.1.6. Illustration of exclusive control of ASCS/ERS instance by EXPRESSCLUSTER".
3.3.4.2. Startup attribute of a failover group¶
Set the startup attribute depending on the configuration as follows:
failover group |
ENSA |
ENSA2 |
---|---|---|
failover group for ERS instance |
Manual Startup |
Auto Startup |
failover group for others |
Auto Startup |
Auto Startup |
3.3.5. Adding additional group resources¶
Add the floating IP resource, NAS resource (for EXPRESSCLUSTER internal version 3.x/4.x) and EXEC resource (for EXPRESSCLUSTER internal version 5.x) to the failover groups created in the previous section.
For details how to add a group resource please refer to the following document:
Please add the following group resources to each failover group:
ASCS instance group
- Add a floating IP resource and assign the IP address settings from 3.2.2. Network Setting.- Add a NAS resource (for EXPRESSCLUSTER internal version 3.x/4.x) / EXEC resource (for EXPRESSCLUSTER internal version 5.x) and assign the mount point for ASCS.
3.3.6. Specifying dependency between failover groups¶
Specify the dependency between failover groups.
The dependency between each instance in SAP NW (starting order) is shown below.
Database
->
ASCS
->
ERSPASAAS
Each instance must be stopped in the reverse order.
Note
As outlined in 2.1.4. HA Database for SAP NW it is assumed there is a database available. This database is a prerequisite for the above dependencies and needs to be available initially. If this is not the case, then you cannot continue from here.
Note
Do not need to specify any dependency for DA and hostexec.
For details how to specify dependencies in EXPRESSCLUSTER please refer to the following document:
for Internal Version 3.3.x/4.0.x:
"Reference Guide"- "Understanding setting of group start dependence and group stop dependence"- "Understanding the settings of dependency among group resources (Common to group resources)"for Internal Version 4.1.xor later :
3.4. Setup of SAP NW Environment¶
Terminology used in "3.4. Setup of SAP NW Environment" and "3.5. Setup of EXPRESSCLUSTER".
- SID
SAP System ID
- DASID
Diagnostics Agent System ID
- INO
Instance Number
The installation path and installation procedure for the product files for SAP NW may vary depending on your configuration.
For details how to build the SAP NW environment please refer at least to the following documents:
Master Guide
Installation Guide
The installation guide for each database and OS type supported by SAP NW can be downloaded from the following URL:Please make sure to review the appropriate guide for environment in detail.
Please make sure to review at least the following SAP notes applicable to your environment, as well as notes mentioned in SAP's guides.
SAP NOTEs
#0171356 : SAP software on Linux: Essential information
#0784391 : SAP support terms and 3rd-party Linux kernel drivers
#2002167 : Red Hat Enterprise Linux 7.x: Installation und Upgrade
#0174911 : Determining the hardware key (customer key)
#0181543 : License key for high availability environment
#0870871 : License key installation
#1391070 : Linux UUID solutions
#0146003 : Application servers cannot be started
Build the environment for SAP NW in the order shown below.
For details how to install SAP NW please refer SAP's official documentation (see above URL).
Note
Before you start the following procedure please make sure to have the database installed according to SAP's documents and it is available.
Preparing Node#1 and Node#2 for SAP NW installation (3.4.1. Preparing Node#1 and Node#2 for SAP NW installation)
Installation of ASCS/ERS instances on Node#1 (3.4.2. Installation of ASCS and ERS instances (Node#1))
Installation of PAS instance on Node#1 (3.4.3. Installation of PAS instance (Node#1))
Installation of ERS instance on Node#2 (3.4.4. Installation of ERS instance (Node#2))
Installation of AAS instance on Node#2 (3.4.5. Installation of AAS Instance (Node#2))
Activation of the Connector for SAP on Node#1 and Node#2 (3.4.6. Activation of the Connector for SAP)
SAP license registration (3.4.7. SAP license registration)
Changing SAP service settings (3.4.8. Changing SAP Instance Service Setting)
Disabling auto startup for SAP instances (3.4.9. Disabling Auto Startup for SAP Instances)
Enabling auto stop for ERS instance (3.4.10. Enabling Auto Stop for ERS Instance)
"3.4.2. Installation of ASCS and ERS instances (Node#1)" through "3.4.3. Installation of PAS instance (Node#1)" describe how to install SAP NW on Node#1.
"3.4.4. Installation of ERS instance (Node#2)" through "3.4.5. Installation of AAS Instance (Node#2)" describe how to install SAP NW on Node#2.
"3.4.6. Activation of the Connector for SAP" describes how to set up Node#1 and Node#2 to use the Connector for SAP.
"3.4.7. SAP license registration" describes how to register SAP license.
"3.4.8. Changing SAP Instance Service Setting" describes how to disable the automatic startup of SAP service in Node#1 and Node#2.
"3.4.9. Disabling Auto Startup for SAP Instances" describes the procedure for disabling auto startup for SAP Instances.
"3.4.10. Enabling Auto Stop for ERS Instance" describes the procedure for enabling auto stop for ERS Instance.
For an example how instance names and numbers are set in this manual please refer to "Sample settings for SAP NW" in the supplied SAP NetWeaver Configuration Example document.
For how to update SAP NW, refer to "4.1. SAP NW Update".
3.4.1. Preparing Node#1 and Node#2 for SAP NW installation¶
Completely install EXPRESSCLUSTER, specify a floating IP, NAS resource (for EXPRESSCLUSTER internal version 3.x/4.x) and EXEC resource (for EXPRESSCLUSTER internal version 5.x), start EXPRESSCLUSTER, and activate the floating IP, NAS resource (for EXPRESSCLUSTER internal version 3.x/4.x) and EXEC resource (for EXPRESSCLUSTER internal version 5.x) in Node#1 before installing SAP NW.
The location to save the SAP software logistics tool including the sapinst command described later depends on your environment and the installation media used (DVD-ROM or downloaded files). The sapinst command is a command used to install SAP NW.
3.4.2. Installation of ASCS and ERS instances (Node#1)¶
Perform this work on Node#1.
Specify the host name associated with the floating IP of ASCS instance as an environment variable SAPINST_USE_HOSTNAME and execute sapinst.
# env SAPINST_USE_HOSTNAME=ASCS_Hostname ./sapinst
Note
Enter the host name associated with the floating IP of ASCS instance for ASCS_Hostname.
Install ERS after the installation for ASCS is completed.
If ENSA is used, execute sapinst as follows:
# ./sapinst
If ENSA2 is used, execute sapinst with the specification of the host name associated with the floating IP for the ERS instance to the environment variable SAPINST_USE_HOSTNAME:
# env SAPINST_USE_HOSTNAME=ERS_Hostname ./sapinst
Note
For ERS_Hostname, set the host name associated with the floating IP for the ERS instance.
The SIDs (SAP System IDs) and instance numbers for ACSC specified during installation are used in 3.5.1.1. Setting up the ASCS resource (ASCS).
The SIDs and INOs for ERS are used in 3.5.1.2. Setting up the ERS1 (Node#1) resource (ERS1) with ENSA used or in 3.5.1.4. Setting up the ERS resource (ERS) with ENSA2 used.
In this manual, set SID and INO as follows:
Instance |
SID |
INO |
---|---|---|
ASCS |
NEC |
10 |
ERS |
NEC |
20 |
3.4.3. Installation of PAS instance (Node#1)¶
Perform this work on Node#1.
# ./sapinst
The SID (SAP System ID) and instance number for PAS specified during installation are used in 3.5.1.5. Setting up the PAS resource.
The DASID (Diagnostics Agent SAP System ID) and instance number for DA specified during installation are used in 3.5.1.7. Setting up the DA1 (Node#1) resource (DA1).
In this manual, set SID and INO as follows:
Instance |
SID |
INO |
---|---|---|
PAS |
NEC |
30 |
DA1 |
DAA |
97 |
Note
DA (Diagnostics Agent) is the instance installed when installing of a PAS instance.
3.4.4. Installation of ERS instance (Node#2)¶
Perform this work on Node#2.
If ENSA is used, execute sapinst as follows:
# ./sapinst
If ENSA2 is used, execute sapinst with the specification of the host name associated with the floating IP for the ERS instance to the environment variable SAPINST_USE_HOSTNAME:
# env SAPINST_USE_HOSTNAME=ERS_Hostname ./sapinst
Note
For ERS_Hostname, set the host name associated with the floating IP for the ERS instance.
The SID (SAP System ID) and instance number for ERS specified during installation are used in 3.5.1.3. Setting up the ERS2 (Node#2) resource (ERS2). with ENSA used or in 3.5.1.4. Setting up the ERS resource (ERS) with ENSA2 used.
In this manual, set SID and INO as follows:
Instance |
SID |
INO |
---|---|---|
ERS |
NEC |
for ENSA configuration: 21
for ENSA2 configuration: 20
|
3.4.5. Installation of AAS Instance (Node#2)¶
Perform this work on Node#2.
Because the AAS only operates on Node#2, it is not necessary to specify a floating IP. Execute sapinst without specifying an environment variable.
# ./sapinst
The SID (SAP System ID) and instance number for AAS specified during installation are used in 3.5.1.6. Setting up the AAS resource.
The DASID (Diagnostics Agent System ID) and instance number for DA specified during installation are used in 3.5.1.8. Setting up the DA2 (Node#2) resource (DA2).
In this manual, set SID and INO as follows:
Instance |
SID |
INO |
---|---|---|
AAS |
NEC |
40 |
DA2 |
DAA |
96 |
Note
DA (Diagnostics Agent) is the instance installed when installing an AAS instance.
3.4.6. Activation of the Connector for SAP¶
To combine the EXPRESSCLUSTER Connector for SAP with SAP NW every instance needs according entries in their start profiles. Please perform the following steps.
Perform this work on Node#1 and Node#2.
3.4.6.1. Setting up the SAP profile¶
Add the following specifications to every instance profile for SAP instances to activate the SAP HA Connector and combine it with EXPRESSCLUSTER.
A setting example in this manual is shown below. The path may vary according to your installation. In this environment the following settings are used:
/sapmnt/<SID>/profile/<SID>_<INSTANCENAME><INO>_<HOSTNAME>
service/halib = /usr/sap/<SID>/<INSTANCENAME><INO>/exe/saphascriptco.so service/halib_cluster_connector = /opt/nec/clusterpro/bin/clp_shi_connector_wrapper
Please verify to add this information to each instance profile.
Additionally add this information to the instance profile of DA instance on Node#1 and Node#2.
/usr/sap/<DASID>/SYS/profile/<SID>_<INSTANCENAME><INO>_<HOSTNAME>
service/halib = /usr/sap/<DAA SID>/SMDA<DA INO>/exe/saphascriptco.so service/halib_cluster_connector = /opt/nec/clusterpro/bin/clp_shi_connector_wrapper
The location of saphascriptco.so specified in service/halib differs depending on the version of NW. If /usr/sap/<SID>/<INSTANCENAME><INO>/exe/saphascriptco.so does not exist, specify /usr/sap/hostctrl/exe/saphascriptco.so for service/halib.
3.4.6.2. Assigning the sudo privilege to the SAP NW user¶
Assign the sudo privilege to the SAP NW user so that the SAP HA Connector can be executed. Set up the privilege by using the visudo command as the root user. Add the following specification:
Defaults:%sapsys !requiretty %sapsys ALL=(ALL) NOPASSWD: ALLNote
Set up the groups automatically created during installation of SAP NW so that sudo can be executed to normally combine SAP NW and EXPRESSCLUSTER. If the SAP NW user cannot execute sudo, starting and stopping of SAP NW instances cannot be normally controlled.
3.4.7. SAP license registration¶
For details how to register the SAP license refer to the official SAP documentation.
3.4.8. Changing SAP Instance Service Setting¶
Perform this work on Node#1 and Node#2.
Enter the following command for disabling the automatic startup of SAP instance service, because each SAP process is run by EXPRESSCLUSTER.
# systemctl disable sapinit
After entering the above command, check that the automatic startup of SAP instance service is disabled.
# chkconfig --list sapinit sapinit 0:off 1:off 2:off 3:off 4:off 5:off 6:off
3.4.9. Disabling Auto Startup for SAP Instances¶
Perform this work on Nodes #1 and #2.
To disable auto startup by the SAP interface for the ERS and DA instances, change the relevant configuration in the profile of these instances.
The ERS instance profile is placed in the following location.
/sapmnt/<SID>/profile/<SID>_ERS<INO>_<hostname>
The DA instance profile is placed in the following location.
/usr/sap/<DASID>/SYS/profile/<DASID>_SMDA<INO>_<hostname>
Change the Autostart value in each profile to 0.
Autostart=0
3.4.10. Enabling Auto Stop for ERS Instance¶
Perform this work on Nodes #1 and #2.
In case that ASCS had a failover to the node where ERS instance is running, ERS needs to be stopped automatically.
The ERS instance profile is placed in the following location.
/sapmnt/<SID>/profile/<SID>_ERS<INO>_<hostname>
#Restart_Program_00 = local $(_ER) pf=$(_PFL) NR=$(SCSID)
Start_Program_00 = local $(_ER) pf=$(_PFL) NR=$(SCSID)
3.5. Setup of EXPRESSCLUSTER¶
3.5.1. Setup of Resources¶
Add the exec resource to the failover groups created in 3.3.4. Creating failover groups.
Set up the exec resource to control starting and stopping of each instance.
Include the following string in the resource name:
instance_<SID>_<INO>The words in <> indicate the following items:- SID: SAP System ID- INO: Instance number
Note
Modify the SAP user (SAPUSER), SAP System ID (SID), SAP profile path (PROFILE), and the instance number (INO) in the supplied sample script according to your environment.
For how to add the exec resource, refer to the following document:
3.5.1.1. Setting up the ASCS resource¶
Add the following two exec resources to the group for which the floating IP for ASCS instance is specified.
Add the exec resource for controlling SAP instance services.
Add the exec resource for starting SAP instances.* Include the SID (SAP System ID) and INO (instance number) specified in 3.4.2. Installation of ASCS and ERS instances (Node#1) in the resource name.Example in this manual
exec-ascs-SAP-instance_NEC_10
3.5.1.2. Setting up the ERS1 (Node#1) resource¶
Add the following two exec resources for the group for ERS1.
Add the exec resource for controlling SAP instance services.
Add the exec resource for starting SAP instances.* Include the SID (SAP System ID) and INO (instance number) specified in "3.4.2. Installation of ASCS and ERS instances (Node#1)" in the resource name.Example in this manual
exec-ERS1-SAP-instance_NEC_20
3.5.1.3. Setting up the ERS2 (Node#2) resource¶
Add the following two exec resources for the group for ERS2.
Add the exec resource for controlling SAP instance services.
Add the exec resource for starting SAP instances.* Include the SID (SAP System ID) and INO (instance number) specified in "3.4.4. Installation of ERS instance (Node#2)" in the resource name.Example in this manual
exec-ERS2-SAP-instance_NEC_21
3.5.1.4. Setting up the ERS resource¶
Add the following three EXEC resources to groups with the floating IPs for ERS instances set:
EXEC resource to check the ASCS startup node
EXEC resource to control the SAP service
EXEC resource to start the SAP instance
*Include the SID (SAP System ID) and INO (instance number) in the resource name, as configured in "3.4.2. Installation of ASCS and ERS instances (Node#1)".Example in this document
exec-ERS-SAP-instance_NEC_20
3.5.1.5. Setting up the PAS resource¶
Add the following two exec resources for the group for PAS.
Add the exec resource for controlling SAP instance services.
Add the exec resource for starting SAP instances.* Include the SID (SAP System ID) and INO (instance number) specified in "3.4.3. Installation of PAS instance (Node#1)" in the resource name.Example in this manual
exec-PAS-SAP-instance_NEC_30
3.5.1.6. Setting up the AAS resource¶
Add the following two exec resources for the group for AAS.
Add the exec resource for controlling SAP instance services.
Add the exec resource for starting SAP instances.* Include the SID (SAP System ID) and INO (instance number) specified in "3.4.5. Installation of AAS Instance (Node#2)" in the resource name.Example in this manual
exec-AAS-SAP-instance_NEC_40
3.5.1.7. Setting up the DA1 (Node#1) resource¶
Add the following two exec resources for the group for DA1.
Add the exec resource for controlling SAP instance services.
Add the exec resource for starting SAP instances.* Include the SID (SAP System ID) and INO (instance number) specified in "3.4.3. Installation of PAS instance (Node#1)" in the resource name.Example in this manual
exec-DA1-instance_DAA_97
3.5.1.8. Setting up the DA2 (Node#2) resource¶
Add the following two exec resources for the group for DA2.
Add the exec resource for controlling SAP instance services.
Add the exec resource for starting SAP instances.* Include the SID (SAP System ID) and INO (instance number) specified in "3.4.5. Installation of AAS Instance (Node#2)" in the resource name.Example in this manual
exec-DA2-instance_DAA_96
3.5.1.9. Setting up the hostexec1 (Node#1) resource¶
Add the following exec resource for the group for hostexec1.
Add the exec resource for controlling saphostexec.
3.5.1.10. Setting up the hostexec2 (Node#2) resource¶
Add the following exec resource for the group for hostexec2.
Add the exec resource for controlling saphostexec.
3.5.1.11. Setting up resources for the exclusive control of Node#1 (for ENSA configuration)¶
Add no group resources.
3.5.1.12. Setting up resources for the exclusive control of Node#2 (for ENSA configuration)¶
Add no group resources.
For detailed settings used in this manual please refer to "Sample configuration of EXPRESSCLUSTER in a SAP NW cluster" and "Exec resources" in the supplied SAP NetWeaver Configuration Example document.
Note
Specify a resource name that conforms to the naming conventions for the exec resource that controls start and stop of SAP NW instances. If the resource name does not conform to the naming conventions starting and stopping of SAP NW instances cannot be normally controlled.
3.5.2. Setup of Monitor Resources¶
Add the custom monitor resource, NIC Link Up/Down monitor resource, and disk monitor resource to the group resources created in the previous section.
For details please refer to the following documents.
For detailed settings used in this manual please refer to "Sample configuration of EXPRESSCLUSTER in a SAP NW cluster" and "Custom monitor" in the supplied SAP NetWeaver Configuration Example document.
The sample script specified for the custom monitor resource is included in the installation media of this product.
Note
Modify the SAP user (SAPUSER) and the instance number (INO) in the supplied sample script according to your environment.
3.5.2.1. Add the NIC Link Up/Down monitor resource¶
For how to add the NIC Link Up/Down monitor resource please refer to the following document:
3.5.2.2. Setting up the SAP NW instance monitor resource¶
Specify the sample script included in this product for the custom monitor resource to monitor the following instances:
Use the sapcotrol command to monitor these instances. For the script using the sapcontrol command, see "Custom monitor" in the SAP NetWeaver Configuration Example. For EXPRESSCLUSTER X 3.3, you can download this sample script form the support portal (Content ID: 9510100151).
ASCS
ERS1 (for ENSA configuration)
ERS2 (for ENSA configuration)
ERS (for ENSA2 configuration)
PAS
AAS
DA1
DA2
3.5.2.3. Setting up the SAP NW instance service monitor resource¶
Specify the sample script included in this product for the custom monitor resource to monitor the following instance services:
ASCS
ERS1 (for ENSA configuration)
ERS2 (for ENSA configuration)
ERS (for ENSA2 configuration)
PAS
AAS
DA1
DA2
hostexec1
hostexec2
3.5.2.4. Setting up the disk monitor resource¶
For how to add the disk monitor resource, see the following:
3.6. Connector for SAP¶
3.6.1. Log configuration¶
This section describes the log output of the Connector for SAP.
For information about the logs in EXPRESSCLUSTER please refer to the following document:
3.6.1.1. logrotate configuration¶
Use Linux logrotate to specify the log location and version control.
The following setup file is created when this product is installed:
/etc/logrotate.d/clp_shi_connector
The default setup is as shown below.
/opt/nec/clusterpro/log/clp_shi_connector.log
{
rotate 1
size 1M
}
The Connector for SAP log is written to the following location:
/opt/nec/clusterpro/log/clp_shi_connector.log
3.6.1.2. Setting up the log level¶
To set up the log level of the Connecter for SAP log please change the parameter of following file.
/opt/nec/clusterpro/etc/clp_shi_connector.conf
Parameter |
Level |
Description |
---|---|---|
LOGLEVEL |
0, 1, 2, 4, 8
(The default is 4)
|
Specify the output log level.
* In spite of setting LOGLEVEL, logs of the ERROR level are output to standard error output and syslog.
0: Do not output any log.
1: (ERROR):
Output logs of the ERROR level.
2: (WARNING):
Output logs of the WARNING level and the ERROR level.
4: (INFORMATION):
Output logs of the INFORMSTION level, the WARNING level and the ERROR level.
8: (TRACE):
Output logs of the internal trace, the INFORMSTION level, the WARNING level and the ERROR level.
|
The setting method is as follows. LOGLEVEL is set as 4 in the following example.
LOGLEVEL=4
3.6.1.3. Format of log¶
An example of a Connector for SAP output log is shown below.
Log format
LEVEL YY/MM/DD HH:MM:SS[PID] message
Example of output log
I 12/08/22 18:54:50[32412] ********** main: clp_shi_connector start **********
E 12/08/22 18:54:50[32412] Invalid options. (aaa bbb)
I 12/08/22 18:54:50[32412] main: retval: 2
I 12/08/22 18:54:50[32412] ********** main: clp_shi_connector end **********
3.6.1.4. List of error messages¶
Error messages that the Connector for SAP outputs to syslog
Message |
Description |
Solution |
---|---|---|
clp_shi_connector invoked. (options: args) |
clp_shi_connector started(specified option: args). |
- |
Invalid options. (args) |
The option is incorrectly specified (specified option: args). |
Correctly specify the option referring to the usage. |
failed to get cluster resource name. (SID: ${sid}, INO: ${ino}) |
The name of the resource that controls the SAP instance of which SID is ${sid} and INO is ${ino} could not be acquired. |
- Correctly specify the name of the resource that controls the SAP instance of which SID is ${sid} and INO is ${ino} according to the naming conventions.
- Correctly set up sudo.
- Start the cluster.
- Check the status of the system.
|
failed to get cluster group name. |
The cluster group name could not be acquired. |
- Correctly set up sudo.
- Start the cluster.
- Check the status of the system.
|
failed to get cluster node name. |
The cluster node name could not be acquired. |
- Correctly set up sudo.
- Start the cluster.
- Check the status of the system.
|
failed to get current node name. (ret=${ret}) |
The name of the node on which the group is currently operating could not be acquired. |
- Correctly set up sudo.
- Start the cluster.
- Check the status of the system.
|
resource "${res_name}" is not ONLINE. |
The resource with the resource name ${res_name} is not active. |
- Correctly set up sudo.
- Start the cluster.
- Start the resource ${res_name}.
- Check the status of the system.
|
clpfunctions is missing. |
There is no clpfunctions file. |
- Install EXPRESSCLUSTER again.
- Check the status of the system.
|
clpstat failed. (ret=${ret}) |
Executing clpstat command has failed (return value:${ret}). |
- Correctly set up sudo.
- Start the cluster.
- Check the status of the system.
|
Can't find cluster resource. (SID: $1, INO: $2) |
The cluster resource controlling SID: $1 and INO: $2 could not be found. |
- Following the naming conventions, correct the name of the resource which control SAP instance whose SID and INO are ${sid} and ${ino} respectively.
- Correctly set up sudo.
- Start the cluster.
- Check the status of the system.
|
Failed to analyze resource line. |
The resource line could not be analyzed. |
|
Can't find cluster group. (resource: $1) |
The cluster group related to resource: $1 could not be found. |
- Correctly set up sudo.
- Start the cluster.
- Check the status of the system.
|
failed to control group resource (${res_name}) because group is stopped. |
The group resource (${res_name}) could not be controlled because the group stopped. |
- Correctly set up sudo.
- Start the group to which the resource belongs.
- Check the status of the system.
|
failed to start group resource (${res_name}) because group resource is not OFFLINE. (ret=${ret}) |
The group resource could not be active because the group resource (${res_name}) did not stop (return value: ${set}). |
- Correctly set up sudo.
- Stop the resource.
- Check the status of the system.
|
failed to stop group resource (${res_name}) because group resource is not ONLINE. (ret=${ret}) |
The group could not stop because the group resource (${res_name}) was not active (return value: ${set}). |
- Correctly set up sudo.
- Start the resource.
- Check the status of the system.
|
3.6.2. Timeout settings¶
The below procedures may fail due to a timeout depending on the system load, when executed by the linkage connector.
Obtaining product information of EXPRESSCLUSTER when the cluster is started.
Checking the group resource status when Rolling Kernel Switch is activated.
In these cases adjust the parameters below in the following file.
/opt/nec/clusterpro/etc/clp_shi_connector.conf
Parameter |
Value |
Description |
---|---|---|
GVI_CHECKCOUNT
|
1 - 60
(The default is 30)
|
The number of retries EXPRESSCLUSTER will try to obtain product information when the cluster is started. The interval between these attempts is set by GVI_CHECK_INTERVAL as stated below.
Even if the count does not reach to the setting, obtaining product information finishes when one attempt succeeded.
|
GVI_CHECKINTERVAL |
1 - 60
(The default is 10)
|
The interval in seconds between EXPRESSCLUSTER attempts to obtain product information. If obtaining product information will be done only once (GVI_CHECKCOUNT=1), then this value will be ignored.
|
FRA_CHECKCOUNT |
1 - 60
(The default is 30)
|
The number of retries to check the status of the group resource when the Rolling Kernel Switch is done. The interval between the check is set by FRA_CHECK_INTERVAL as stated below.
Even if the count does not reach to the setting, the status check finishes when one attempt succeeded.
|
FRA_CHECKINTERVAL |
1 - 60
(The default is 10)
|
The interval in seconds between checks of the status of the group resource. If the status check will be done only once (FRA_CHECKCOUNT=1), then this value will be ignored.
|
3.6.3. Configuring the maintenance mode (for Internal Version 4.1.0-1 or later)¶
When the maintenance mode is used, adjust the parameter below in the following file:
/opt/nec/clusterpro/etc/clp_shi_connector.conf
Parameter |
Value |
Description |
---|---|---|
SMM_PATH |
Directory to store the files which the Connector for SAP uses for the maintenance mode (default: none) |
Set the directory to store the files which the Connector for SAP uses for the maintenance mode. Specify the directory on which each cluster node is allowed to write. Under the specified directory, do not manually create files or directories. Only use up to 240 single-bite characters.
Example:
SMM_PATH="/sapmnt/sapmm"
|
3.6.4. ENSA settings (for Internal Version 4.1.0-1 or later)¶
With the parameter below in the following file, specify the ENSA version to be used.
The version of ENSA must be the same as its setting on the SAP NW side.
/opt/nec/clusterpro/etc/clp_shi_connector.conf
Parameter |
Value |
Description |
---|---|---|
ENSA_VERSION |
1, 2
(Default: 1)
|
Set to 1 with ENSA used.
Set to 2 with ENSA2 used.
|
3.6.5. Setting abnormal process judgement for each instance (for Internal Version 4.1.0-1 or later)¶
The bundled scripts allow you to check the statuses of processes composing each instance with sapcontrol -function GetProcessList. To determine whether the result is abnormal or not, you can choose from the following two patterns:
Judge the result as abnormal when not all the process statuses are GREEN.
Judge the result as abnormal when not all the process statuses are GREEN or YELLOW.
If a different pattern is to be set for a specific process, customize the bundled script (e.g. for judging the result as normal when Process A is YELLOW, for judging as abnormal when Process B is YELLOW).
/opt/nec/clusterpro/etc/clp_shi_connector.conf
Parameter |
Value |
Description |
---|---|---|
YELLOW_AS_ERROR |
0, 1
(Default: 1)
|
Set the value to 1 to judge the result as abnormal when not all the process statuses are GREEN. In this case, YELLOW leads to judging the result as abnormal.
Set the value to 0 to judge the result as normal even if any of the process statuses is YELLOW.
|
4. Miscellaneous¶
4.1. SAP NW Update¶
To update SAP NW, use Software Update Manager (hereafter referred to as SUM). The update procedure with SUM involves restarting SAP instances and therefore it may interfere with EXPRESSCLUSTER which tries to keep the SAP components available. To avoid such interference with EXPRESSCLUSTER, suspend EXPRESSCLUSTER's monitoring for all SAP components that SUM has to restart.
Please select from the following two options to suspend EXPRESSCLUSTER's monitoring.
Suspending the whole cluster
Suspending monitor resources for related SAP instances and instance services
Update SAP NW with SUM while the cluster or the monitor resources are suspended. After the update is completed, resume the suspended cluster or the suspended monitor resources.
For how to suspend and resume a cluster or a monitor resource, please refer to the following document.
for Internal Version 3.3.x/4.0.x
"Reference Guide"- "Chapter 1 Functions of the WebManager"- "Operations from the WebManager"for Internal Version 4.1.xor later
Cluster WebUI online manual
4.1.1. Maintenance mode (for Internal Version 4.1.0-1 or later)¶
The maintenance mode can be switched on/off by the sapcontrol command or from the SAP management console. For details on the maintenance mode and the sapcontrol command, see the SAP documents.
The following is an example to enable the maintenance mode with the sapcontrol command:
sapcontrol -nr <SID>-function HASetMaintenanceMode 1
The following is an example to disable the maintenance mode with the sapcontrol command:
sapcontrol -nr <SID>-function HASetMaintenanceMode 0
4.2. Notes and Restrictions¶
- Notes on starting/stopping groups (for Internal Version 3.3.x/4.0.x)Refer to the following sections in the "Reference Guide":"Group resource details"-> "Group start dependence and group stop dependence"-> "Notes"
Notes on starting/stopping groups (for Internal Version 4.1.xor later)
Refer to the following sections in the "Reference Guide":- Restriction of using spacesA node name , a failover group and a resource name must not contain any spaces. If they contain some spaces, starting and stopping of SAP NW instances cannot be controlled correctly.
For the instance number configurations of the SAP NW components and the notes on these configurations, refer to "2.1.1.4. Instance number configuration of the SAP NW components".
- Naming conventions for failover groupsSpecify a failover group name according to the naming conventions for the failover group for exclusive control of ASCS/ERS instance. If the failover group name does not follow the naming conventions, exclusive control of ASCS/ERS instance cannot function correctly.
- Naming conventions for exec resourcesSpecify a resource name that conforms to the naming conventions for the exec resource that controls starting and stopping of SAP NW instances. If the resource name does not conform to the naming conventions, starting and stopping of SAP NW instances cannot be normally controlled.
- Attention when one node recoversWhen the node where ERS instance was working gets recovered and joins the cluster, then the failover group of the ERS instance is not restarted automatically.You need to validate the node is working healthy and then restart the failover group of ERS instance manually.
- Privilege setupSet up the groups automatically created during installation of SAP NW so that sudo can be executed to normally combine SAP NW and EXPRESSCLUSTER. If the SAP NW user cannot execute sudo, starting and stopping of SAP NW instances cannot be normally controlled.
- Languages setting (for Internal Version 3.3.x/4.0.x)Be sure to specify English for Language on the Cluster Definition window of Cluster Generation Wizard of WebManager. If a language other than English is specified, the cluster will not work properly.
- Maintenance mode (for Internal Version 4.1.xor later)When the maintenance mode is used, avoid suspending or resuming the cluster from Cluster WebUI or with the clpcl command in order to avoid conflicts.For the "SMM_PATH" parameter in clp_shi_connector.conf, specify the directory on which each cluster node is allowed to write. Under the specified directory, do not manually create files or directories.