Mission Critical Computing Blog
Your source for the latest insights on HP Integrity, mission critical computing, and other relevant server and technology topics from the BCS team.

Secure NonStop OSM Communication via Custom SSL Configuration

In my last blog, I wrote about various OSMCONF settings that you can use to customize OSM to your specific NonStop environment. In this blog, I will discuss OSMCONF settings specific to OSM client-server communication using Secure Sockets Layer (SSL). In this context, OSM client means not only OSM Service Connection and OSM Event Viewer, but also HP Systems Insight Manager (SIM), HP Insight Remote Support Advanced, HP Insight Control Power Management, or any NonStop Essentials product.


Configure SSL support

T0682 H02 AAM and later versions of OSM provide SSL support for a secure connection between OSM server and its clients. SSL is optional for OSM Service Connection and OSM Event Viewer, but required for HP SIM, HP Insight Remote Support Advanced, HP Insight Control Power Management and NonStop Essentials products.


For HP Integrity NonStop BladeSystems, SSL is always enabled, by default, for OSM client-server communication. T0682 H02 ADD and later versions of OSM have SSL enabled by default for all NonStop systems running J-Series or H-Series software. But if you are running older versions of OSM on a NonStop NS-Series server, use the following in OSMCONF to enable SSL:

UseSSL = On


Configure cipher suites used by SSL

To configure the cipher suites used by SSL for OSM client-server communication, use the following in OSMCONF:



Here, value is a comma-separated list of three different values. The default is 0.4,0.10,0.5. You can choose from the following values in any order:

  • 0.4 - RSA-key-exchange + RC4-128-bit encryption and MD5 (RC4-MD5)
  • 0.5 - RSA-key-exchange + RC4-128-bit encryption and SHA (RC4-SHA)
  • 0.10 - RSA-key-exchange + 3-DES encryption and SHA (DES-CBC3-SHA)

Create your own private SSL certificate

SSL encryption and authentication depend on X.509 certificates to provide keys and identify the parties involved in the communication. The SSL support shipped with OSM relies on a shared SSL certificate. But for more security, you should generate and use a private SSL certificate. OpenSSL toolkits can help you generate your own SSL certificate.


Requirements for a private SSL certificate

Before you create a private SSL certificate, let’s look at the requirements for such a certificate from OSM perspective:

  • The certificate’s Subject Common Name (CN) must include the DNS name or IP address configured for OSM. If you have more than one, select the most outward facing or most used address.
  • A Subject Alternative Name (SAN) is required. If used, it overrides anything in the CN; so ensure to repeat the name you used in the CN, in the SAN.

The Subject Alternative Name (SAN), subjectAltName, is included in a sample config.txt file below. You can use an ASCII text editor to create the custom configuration file config.txt, which is used by OpenSSL.

[ req ]

default_bits= 512

default_md= md5

string_mask= utf8only

distinguished_name= req_DN


[ req_DN ]

C= Country Name

C_min= 2

C_max= 2

ST= State or Province Name

L= Locality Name

O= Organization Name

OU= Organization Unit Name

CN= Common Name

emailAddress = Email Address


[ verisign_CA ]


basicConstraints= critical, CA:true , pathlen:20

subjectKeyIdentifier= hash


[ v3_req_ext_SERVER ]

subjectAltName = @alt_names_SERVER


[ alt_names_SERVER ]

DNS.1 = mynsk.mydomain.com

IP.1 =

IP.2 =

IP.3 =

IP.4 =

IP.5 = 2620:0:a05:e014:a00:111f:f001:0001


[ ca ]

default_ca= CA_default


[ CA_default ]

database= index.txt

serial= serial.txt

default_days= 365

default_crl_days= 30

default_md= md5

email_in_dn= no

name_opt= ca_default

cert_opt= ca_default

copy_extensions= copy

new_certs_dir = C:/OpenSSL-Win32/bin


[ policy_anything ]

countryName= optional

stateOrProvinceName= optional

organizationName= optional

organizationalUnitName= optional

commonName= optional

emailAddress= optional


[ verisign_SERVER ]

basicConstraints= critical, CA:false

keyUsage = keyEncipherment, dataEncipherment

extendedKeyUsage= clientAuth,serverAuth

subjectKeyIdentifier= hash

authorityKeyIdentifier= keyid:always

nsCertType= client, server


The IP addresses for the SAN field are the IP addresses that the OSM client uses to communicate with the OSM server on the NonStop system as well as any additional IP addresses available for OSM Event Viewer. For example, OSM Service Connection may use any of the IP addresses available for the default TCPIP processes ($ZTCP0 and $ZTCP1), and also IP addresses for the user-defined TCPIP processes specified in OSMCONF with stack settings.


Any IP addresses available for OSM server or OSM Event Viewer use which are not listed in the SAN field of a customer-supplied SSL certificate causes security warning dialogs for OSM Service Connection or OSM Event Viewer clients. The certificate SAN field, subjectAltName, must include both numerical IP addresses and any corresponding DNS names to be used for OSM Service Connection or OSM Event Viewer sessions.


OSM Service Connection uses not only the IP address or DNS names visible in the original URL for the web browser session, it also uses multiple IP addresses in the background for fault tolerance. All IP addresses for $ZTCP0 and $ZTCP1, as well as those specified by stack settings in the OSMCONF file, are tried in the background of the user session. If OSM Service Connection is invoked in the maintenance LAN, it uses connections on both the $ZTCP0 and $ZTCP1 TCP/IP stacks, even though only one IP address is included in the original URL of the web browser session. When OSM Service Connection is invoked from outside the maintenance LAN, the IP addresses corresponding to $ZTCP0 and $ZTCP1 are probably inaccessible, but multiple other IP addresses corresponding to OSMCONF stack settings are accessible. The certificate SAN field, subjectAltName, must include all IP addresses which could be used on any LAN, because only one SSL certificate is available for the different LANs used for OSM Service Connection.


OSM Event Viewer uses only the IP address specified in the original URL of the web browser session. It uses the same SSL certificate as OSM Service Connection. 


  • Wildcards are allowed in DNS names only.
  • RSA key lengths can be up to 2048 bits.
  • Although MD5 hashing is supported, SHA1 is recommended as it is much more secure.

Use your own private SSL certificate

Once a private SSL certificate is generated, it needs to be placed on the NonStop system. OSM needs to know the certificate used for SSL and also the Certificate Authority (CA) certificate that signed the server’s certificate. Additionally, the server’s private key file is needed to encrypt the communications. The requirements for these files are:

  • The server certificate and the CA certificate must be in binary DER format.
  • The server key file must also be in binary DER format, but must also be in PKCS#8 format.
  • The server certificate, the CA certificate, and the server key file must be binary code 0 files. This means, you should FTP to NonStop system in binary mode.
  • The server key file must be encrypted with a password.

Now, add the following in OSMCONF:

SERVCERT = <server certificate filename>

CACERTS = <CA certificate filename>

SERVKEY = <server private key filename>

SERVKEYPASS = <key password>


On the client side, the certificate will be trusted by default if all of the following conditions are satisfied:

  • The certificate is not self-signed.
  • The certificate is valid (and not expired).
  • The certificate is signed by a trusted CA.
  • If the certificate has a SAN, the server’s address matches one of the entries in the SAN.
  • If the certificate does not have a SAN, the server’s address matches the Subject CN.

If any of these conditions is not satisfied, OSM Service Connection displays a warning dialog detailing issues with the certificate, with the option to exit or continue. You also have an option to trust this certificate or this CA in the future and skip past the warning dialog.


In OSM Event Viewer, if any of these conditions is not satisfied, various different warnings are possible depending on Internet Explorer’s version and its security settings.


I highly recommend that you create your own more secure private SSL certificate for OSM client-server communication and pass your audits for this communication.

Leave a Comment

We encourage you to share your comments on this post. Comments are moderated and will be reviewed
and posted as promptly as possible during regular business hours

To ensure your comment is published, be sure to follow the community guidelines.

Be sure to enter a unique name. You can't reuse a name that's already in use.
Be sure to enter a unique email address. You can't reuse an email address that's already in use.
Type the characters you see in the picture above.Type the words you hear.
About the Author
Vinay Gupta is an HP Distinguished Technologist and the NonStop Manageability Architect. He joined Tandem in 1994 after graduating from Indi...
About the Author(s)
  • I work as a Master Architect in HP Servers R & D group. I work with teams spread across the lab and outside to build solutions which are highly available on HP-UX, OpenVMS and Mission Critical Linux platforms. In particular I contribute to develop HP Serviceguard clusters, HP-UX Security and Middleware products. I have been with HP for last 17 years and have exposure to HA/DR field from both R & D and customer perspectives.
  • Kirk Bresniker is the Vice President/Chief Technologist for HP Business Critical Systems where he has technical responsibility for all things Mission Critical, including HP-UX, NonStop and scalable x86 platforms. He joined HP in 1989 after graduating from Santa Clara University and has been an HP Fellow since 2008.
  • I’m the worldwide marketing manager for HP NonStop. I’ll be blogging and tweeting out news as it relates to NonStop solutions – you can find me here and on twitter at @CarolynatHP
  • Cynthia is part of the HP ExpertOne team. ExpertOne offers professional IT training and certifications from infrastructure refresh to areas that span across the datacenter like Cloud and Converged Infrastructure.
  • Hi, I work on the HP Servers team as HP-UX worldwide product marketing manager. I´m interested in how customers use our technology and will be blogging about their stories and on how our products evolve to help their businesses be always on.
  • I have worked with NonStop systems since 1982. I am a Master Technologist for HP and am part of the IT SWAT organization, the Cloud SWAT and work with HP Labs. I report into the Enterprise Solutions and Architecture organization.
  • Joe Androlowicz is a Technical Communications and Marketing manager in HP’s NonStop Product Division. Joe is a 25 year journeyman in information systems design, instructional technologies and multimedia development. He left Apple Computer for Tandem Computers to help launch G03 and hasn’t looked back yet. He previously managed the program management team for the NonStop Education and Training Center and drove the development and growth of the NonStop Certification programs.
  • Hello! I am a social media manager for servers, so my posts will be geared towards HP server-related news & info.
  • HP Servers, Converged Infrastructure, Converged Systems and ExpertOne
  • Luke Oda is a member of the HP's BCS Marketing team. With a primary focus on marketing programs that support HP's BCS portfolio. His interests include all things mission-critical and the continuing innovation that HP demonstrates across the globe.
  • I am the Superdome 2 Product Manager. My interest is to learn how mission critical platform helps customers and would also like to share my thoughts on how Superdome has been helping customers and will continue to do so.
  • I work in the HP Servers marketing group, managing a marketing team responsible for marketing solutions for enterprise customers who run mission-critical workloads and depend on HP to keep their business continuously running.
  • Mohan Parthasarathy is a Technical Architect in the HP-UX lab. His primary focus currently is in the core kernel, platform enablement and virtualization areas of HP-UX. Mohan has worked on various modules of HP-UX, including networking protocol stacks, drivers, core kernel and virtualization
  • I’m the Worldwide Product Marketing Manager for HP Serviceguard Solutions for Linux in BCS. I’ll be blogging about the latest news and enhancements as it relates to this product.
  • Greetings! I am on the HP Enterprise Group marketing team. Topics I am interested in include Converged Infrastructure, Converged Systems and Management, and HP BladeSystem.
  • As a Managing Consultant for HP’s Enterprise Solution & Architecture group, I collaborate with client business and IT senior management to understand, prioritize and architect advanced use of data and information, drawing insights required to make informed business decisions. My current focus leverages event-driven business intelligence design techniques and technologies to identify patterns, anticipate outcomes and proactively optimize business response creating a differentiated position in the marketplace for the client.
  • Wendy Bartlett is a Distinguished Technologist in HP’s NonStop Enterprise Division, and focuses on dependability – security and availability - for the NonStop server line. She joined Tandem in 1978. Her other main area of interest is system architecture evolution. She has an M.S. degree in computer science from Stanford University.
  • I am part of the integrated marketing team focused on HP Moonshot System and HP Scale-up x86 and Mission-critical solutions.

Follow Us