Leap Second 2015
Technical Information
1
Table of Contents
Contents
1.Document Information
1.1.Purpose of the Document
1.2.Author of the Document
1.3.Accompanying Documents
1.4.External References
2.Procedure
2.1.Purpose
2.2.Identification
2.3.Scope
2.3.1.Vulnerable Products
2.3.2.Not Vulnerable Products
2.4.Risk of Not Taking Any Action
2.5.July 1st Emergency Support
3.Resolution
3.1.SME
3.1.2.Solution
3.2.HiPath 4000 V6 R1 - OpenScape 4000 V7 R1
3.2.1.Risk
3.2.2.Solution 1
3.2.3.Solution 2
3.3.OpenScape 4000 Manager V7
3.3.1.Risk
3.3.2.Solution 1
3.3.3.Solution 2
3.4.Call Director Sip Service (CDSS)
3.4.1.Risk
3.4.2.Time Estimation
3.4.3.Procedure Description
3.5.OpenScape Voice V7R1 – V8
3.5.1.Risk
3.5.2.Solution 1
3.5.3.Solution 2
3.6.Applications
3.6.1.Risk
3.6.2.Solution 1
3.6.3.Solution 2
4.Appendix
4.1.Glossary
1.Document Information
1.1.Purpose of the Document
This document is delivered for the Leap Second issue that was announced for June 30th 2015
and contains information, instructions and recommended actions for all Unify products.
The announcement was made from Earth Orientation Center of IERS Observatoire de Paris, France (See Chapter 1.6).
1.2.Author of the Document
Copyright © UNIFY SAAuthor:
Name: / T. Aravopoulos
1.3.Accompanying Documents
Document Title / Author / ReferenceOBSO-1505-01 / OpenScape Baseline Security Office
© Unify GmbH & Co KG 2015
Hofmannstr. 63, D-81379 München
/ Security Advisory Report – OBSO (Leap Second on 2015-06-30 - Security Note for Unify Products)
1.4.External References
Information about Leap Second Issue by Suse can be found here:
and
Leap Second announcement:
2.Procedure
2.1.Purpose
Aleap secondis a one-secondadjustment that is occasionally applied toCoordinated Universal Time(UTC) in order to keep its time of day close to themean solar time, or UT1. Without such a correction, time reckoned byEarth's rotationdrifts away fromatomic timebecause ofirregularitiesin the Earth's rate of rotation. Since this system of correction was implemented in 1972, 26 such leap seconds have been inserted. The most recent one happened on June 30, 2012 at 23:59:60 UTC.A leap second will again be inserted at the end of June 30, 2015 at 23:59:60 UTC
There is a leap second being added at midnight on June 30th 2015 (the “2015 Leap Second”). In the past, Linux based systems have had some issues handling the introduction of a leap second and these have led to the operating system becoming deadlocked and affecting server operation. The Linux maintainers have provided operating system updates (kernel) to address these. A certain number of products delivered by Unify are running on the vulnerable Operating System SLES11.
General Note: Only products/servers running on SLES11 connected to an NTP (standard deployment according OSRA) may be vulnerable! (Note that Linux X86 architecture based kernels of multiple Linux operating systems are vulnerable. Some of those non-SLES based Linux distributions are used in Unify embedded platforms (e.g. OpenScape Office MX / OpenScape Business X embedded).
The overall risk for potential system outages or service interruptions in Unify products is rated as low.
2.2.Indemnification
PLEASE NOTE: it is the customer’s responsibility to ensure that the following instructions have been actioned in good time before 30 June 2015, where relevant. Unify shall have no liability for any losses, expenses, claims or damages, whether direct or indirect,in the contemplation of the parties or otherwise, which are sustained by the customer, partner or any third party as a result of the 2015 Leap Second or any failure of technology to function correctly by reason of the 2015 Leap Second, if the customer has not actioned the instructions below within a reasonable time before 30 June 2015. Whilst Unify has used reasonable endeavors to ensure that the instructions below will not have an adverse impact on the rest of the customer’s systems and network, the customer is responsible for taking all reasonable precautions to minimize any such adverse impact.
Unify Partner is responsible for bringing the above notice to the attention of the customer in writing.
To identify if a SLES11 system is vulnerable, proceed as described as follows:
- Login to system via command line.
- Execute the following command:
# uname -a
Example:
If the kernel is below the ones mentioned here, the system is vulnerable to incorrect behavior after the leap second is added.
SP1 - kernel 2.6.32.59-0.19.1 was released mid of April 2015
SP2 - kernel 3.0.101-0.7.29.1 was released end of March 2015
SP3 - kernel 3.0.101-0.47.52.1 was released end of March 2015
2.3.Scope
Belowis a complete list of vulnerable and NOT vulnerable products:
2.3.1.Vulnerable Products
For each vulnerable product listed below, please follow the instructions at the corresponding chapter. In these chapters possible impacts, risk and mitigation are detailed.
Vulnerable Product / Section of current DocOpenScape Business V1 S (server based variant)/ Booster Server V1 / See Chapter 3.1
OpenScape Office V3R3.x, LX / HX (server based variants) / See Chapter 3.1
OpenScape Office MX V3R3.x / See Chapter 3.1
OpenScape Business X [embedded variants] / See Chapter 3.1
HiPath 4000 V6 R1 Platform & Softgate(older versions than R1.12.2)
OpenScape 4000 V7 R1 Platform and Softgate before V7 R1.8.3 or V7 R1.39.1 / See Chapter 3.2
OpenScape 4000 Manager (running on SLES11SP3 64bit) / See Chapter 3.3
Call Director Sip Service (running on OpenSuse 11.1- all versions up to and including V8 R2) / See Chapter 3.4
OpenScape Voice V7R1 & V8.xx (OpenScape Voice V7 R1 before V7 R1.43.1, OpenScape Voice V8 before V8 R0.34.7) / See Chapter 3.5
OpenScape Fault Management / See Chapter 3.6
OpenScape UC Servers (BE, FE & MS) (V7 & V6 running SLES 11) / See Chapter 3.6
OpenScape Voice Survivability Authority / See Chapter 3.6
OpenScape Façade Server (running SLES 11) / See Chapter 3.6
OpenScape OpenFire (running SLES 11) / See Chapter 3.6
OpenScape Google App Server (running SLES 11) / See Chapter 3.6
2.3.2.Not Vulnerable Products
NOT Vulnerable Product / CommentsUnify products running on Microsoft Windows operating systems / not vulnerable
OpenStage / OpenScape Desk Phone IP / not vulnerable
HiPath3000 / not vulnerable
HiPath/OpenScape 4000 Assistant & CSTA (all versions) / not vulnerable
HiPath Cordless IP / not vulnerable
OpenScape Alarm Response (OScAR) Eco & Pro / not vulnerable
HiPath 4000 V6 R2 and OpenScape 4000 V7 R0 Platform and Softgate / Already include the required fixes
HiPath 4000 V5 and earlier / not vulnerable
HiPath 4000 Manager / not vulnerable (running on SLES10SP4 32bit)
OpenScape Contact Center / not vulnerable(running on Windows server 2003/2008)
OpenScape Session Border Controller or OpenScape Branch / Products useupdated kernels
OpenScape Xpert / Debian versions are not vulnerable
OpenScapeXpressions / not vulnerable
OpenScape Fast Viewer / not vulnerable
QCU (QoS Data Collection Unit): / not vulnerable (running on SLES10)
Applications running on Linux systems other than Novell SLES (e.g. Novell SLES 10, or Debian Linux V6 or V7 for OpenScape Xpert MLC / not vulnerable
Each procedure/solution is marked to indicate the perceived risk associated with the procedure
Low Risk Medium Risk High Risk
2.4.Risk of Not Taking Any Action
If a product is listed in the above tables as being vulnerable to the insertion of the leap second, it is strongly recommended that the preventive measures listed in this document are applied proactively.
Without protection, it is possible that one or more processes within the operating system may freeze or otherwise become corrupted and affect the operation of the system. Typically a full system restart will restore the correct operation.
If a system cannot be protected prior to June 30th, it is strongly recommended that a full system backup be taken as a precaution.
2.5.July 1st Emergency Support
If a system is not operating correctly following the leap second period, regardless of whether it is protected or not, the following steps should be followed:
1)Conduct a full system restart
2)Submit a service ticket via your usual support channel. Partners with service rights may open tickets directly with Unify, or in case of priority 1 incidents, contact the expert assistance hotline. Partners that receive services through the distribution channel should reach out for support through their respective distributor.
3.Resolution
3.1.SME
3.1.1.1.1.Risk
The corresponding risk was evaluated to Low
3.1.2.Solution
3.1.2.1.1.Time Estimation
The estimated time is calculated to: 45 Minutes, and is dependent on the speed of the server/internet connection.
3.1.2.1.2.Impact
In case a system software update is required, there will be a downtime due to the system restart.
3.1.2.1.3.Procedure Description
The systems that could be vulnerable to the time adjustment are the server based variants (OpenScape Business S, Booster Server V1 and OpenScape Office LX/HX V3 R3.x) running on SLES11.x and connected to an NTP server and OpenScape Business X embedded variants, OpenScape Office MX V3 R3.x connected to an NTP server.
To ensure no issues are encountered when the time adjustment will be made, some precautionary measures can be implemented. Please be aware that the update of the operating system is the responsibility of service/customer.
- OpenScape Business S, Booster Server V1 [server based variant]:
- Perform a software update to OpenScape Business S V1R3.0.0 or later version (latest General Available Software recommended) – mandatory
- A SLES operating system update is allowed to be applied according to the Service Pack used (please check and proceed as explained in
Chapter 3.6.1.2) – mandatory
- OpenScape Office LX/HX V3R3.x [server based variant]:
- Perform a software update to OpenScape Office V3R3.10.0 or later version (latest General Available Software recommended) – mandatory
- A SLES operating system update is allowed to be applied according to the Service Pack (please check and proceed as explained in
Chapter 3.6.1.2) – mandatory
- OpenScape Business X embedded variants and OpenScape Office MX V3R3.x:
For OpenScape Business and OpenScape Office embedded variants the NTP configuration should be temporarily deactivated by June 29, 2015 23:00:00 UTC at the latest and can be reactivated after July 1, 2015 00:00:01 UTC. The deactivation of the NTP synchronization for a few days does not have any impact on the stability of the system.
To deactivate the NTP configuration, please go to the WBM of the system to Expert Mode/Telephony Server/Basic Settings/Date and Time/SNTP Settings and from the dropdown of the field “Administration Mode of SNTP Client:” choose the option “down” and select “Apply”.
Toreactivatethe NTP configuration, follow the same path in WBM, Expert Mode/Telephony Server/Basic Settings/Date and Time/SNTP Settings and from the dropdown of the field “Administration Mode of SNTP Client:” choose the option “up” and select “Apply”.
3.2.HiPath 4000 V6 R1 - OpenScape 4000 V7 R1
For the HiPath 4000 V6 R1 and OpenScape 4000 V7 R1 there are two options for implementation.
3.2.1.Risk
The corresponding risk for versions before HiPath 4000 V6 R1 was evaluated to High
The corresponding risk for OpenScape 4000 V7 R1 was evaluated to Low
3.2.2.Solution 1
Applying the patch of exchanging ntp-daemon.
The described fix is without downtime and can be applied to the following systems:
-HiPath 4000 V6 R1 systems (including Standalone SoftGates)
-OpenScape 4000 V7 R1.8 systems (including Standalone SoftGates)
-OpenScape 4000 V7 R1.39 systems (including Standalone SoftGates)
3.2.2.1.1.Time Estimation
The estimated time to complete this procedure is calculated to: 10 minutes
3.2.2.1.2.Procedure Description
The fix is without downtime and must be applied to every node in the system.
This applied patch is not further distributed by APE HBR or RAR.
Subsequent official Hotfixes can be activated on top of this fix without issue.
The complete procedure description the correspondingRPM’s can be found at the INF-15-000243, located in the Service Info section of the partner portal.
3.2.3.Solution 2
Perform an Upgrade
Official Hotfixes are or will be available as described in INF-15-000243, located in the Service Info section of the partner portal.
3.2.3.1.1.Time Estimation
Platform Hotfix HiPath 4000 V6 R1.12.2 (HF003013)
Complete upgrade duration depends on the actual deployment, hardware and base PLT version, however the available average durations are as follows:
Simplex deployment ~30 minutes
Duplex deployment ~1 hour
GeoSeparated Duplex~1 hour
APE deployments ~30 minutes (via AP Host backup & APE restore)
3.2.3.1.2.Impact
Platform Hotfix HiPath 4000 V6 R1.12.2 (HF003013)
PLT-HF V6 R1.12.2 is a type 2 hotfix with following impact:
- Simplex deployments:telephony/application downtime due to node reboot
- Duplex deployments:no telephony downtime but CC/ADP switchovers (CSTA down) due to sequential node reboots.
- Note that the PLT-HF V6 R1.12.2 is not available for Standalone Softgates; there, use Solution 1 in any case
3.2.3.1.3.Procedure Description
Upgrade to V6 R1.12.2 (HF003013)
The precondition for the activation of a PLT-Hotfix is the V6 R1.12.0.
Note: Upgrade from V6R1.9.0, V6 R1.10.0 to V6 R1.12.0 is possible.
For more details see the “Service Documentation” located in the Portfolio Information section inthe partner portal.
The PLT-Hotfix is transferred by means of SWT and activated using SWA.
Detailed information on handling the SWT and SWA is contained in the following manuals:
HiPath 4000 V6, Section 3 - Feature Usage Examples, Service Documentation, Issue X, Chapter13 (Software Update of HiPath 4000 Systems) - Link
HiPath 4000 Assistant/Manager V6, Software Transfer 2, Administrator Documentation, Issue X - Link
3.3.OpenScape 4000 Manager V7
3.3.1.Risk
The corresponding risk was evaluated to Low
3.3.2.Solution 1
Kernel Update
3.3.2.1.1.Time Estimation
The estimated time to complete this procedure is calculated to: 45Minutes.
3.3.2.1.2.Impact
This is a permanent solution, so no other actions will need to be performed once the updated kernel is installed. The system will need rebooting, in order to start using the new kernel, so there will be a disruption of service until the system comes back up.
3.3.2.1.3.Procedure Description
The recommended patch must be downloaded from Novell’s web site. Novell’s Patch Finder can be reached via the following link:
- Only 64 bitarchitecture (x86_64)SLES 11 SP3 operating system is supported in V7R0 (and up)
- For SLES 11 SP3 (x86_64)the required patch is Linux Kernel 10546.
To obtain a patch from the website, the technician performing the download will need to have a login account supporting patch downloads (for SLES11 SP3 the account will additionally need Long Term Service Pack Support [LTSS] privileges).
Once the latest kernel updates are downloaded, they must be transferred onto the SLES 11 system requiring the update. On the SLES11 system create a new directory (for example /patches) and upload the RPMs into that directory.
Once all the RPMs are on the system, as user root, start “yast” from the command line:
yast
And perform the following actions:
Software -> Software Repositories -> Add -> Local Directory -> set path to directory containing RPMs -> OKto save configuration and addition of repository -> go to main screen and exit yast.
From the command line check that the repository has been added by executing the command:
zyppersl
The output should include the directory where the new RPMs have been stored.
The next step is to start the update procedure, by entering the command:
zypper up -t package
This should take approximately 5 minutes, and once complete the system will be updated with the new kernel. In order for the system to start running with the new kernel, it will need to be rebooted with the following command:
init 6
Once the system comes back up, check the kernel version by executing:
uname –r
It should display the following:
3.0.101-0.47.52-default
3.3.3.Solution 2
The workaround ("slew mode") described by Novell is not applicable on OpenScape 4000 Manager Servers (as a different NTP configuration interface is implemented). An alternate option (for unpatched installations with high availability requirements) is temporarily to disable the NTP daemon via webmin on June 29th UTC and enable it again not before July 1st, 01:00 am UTC.
3.3.3.1.1.Time Estimation
The estimated time to complete this procedure is calculated to: 15 Minutes.
3.3.3.1.2.Impact
No system downtime and small overall implementation time.
3.3.3.1.3.Procedure Description
Disable the NTP daemon via webmin on 29th June UTC and enable it again after 1st July 01:00 AM UTC or later. To do so, logon to OpenScape 4000 Manager, go to Base Administration – Webmin – Date/Time . Click on Ext. Time Server link and, from the dropdown list on the right side, you can switch between External (via NTP) and Local (HW Clock) then click on Modify button, to save the changes.
3.4.Call Director Sip Service (CDSS)
NTP Server Restart
3.4.1.Risk
The corresponding risk was evaluated to Low
3.4.2.Time Estimation
The estimated time to complete this procedure is calculated to: 10 Minutes.
3.4.3.Procedure Description
The Call Director SIP Server is discussed in this document as the underlying OS platform of this product (openSUSE11.1 with kernel 2.6.27.56) is listed as potentially affected.
Customers with a CDSS server installed, however, are not expected to face any issues due to the nature of message processing employed by the solution. All CDSS messages are processed sequentially rather than synchronously with any external party. The leap second issue is also not valid if there is no NTP service running.
The developers of the CDSS have evaluated the risk to CDSS installations have decided that no fix is required, indeed it is not possible either due to the version of OpenSuse currently used by CDSS. This position is also based on the lack of any reported issues across the global CDSS customer base during the last such occurrence of a leap second back in June 2012.
However, to guard against the minimal risk discussed, customers can, at their discretion, stop the NTP service temporarily and start it again once the leap second has occurred. This has been tested and can be done in the following way:
From the OSCC or another service through which the CDSS machine can be reached, establish a secure connection usingPutty. Please ensure this is installed on the machine from which you will be accessing the CDSS ahead of the work. Use the credentials of user “root” to connect.
Once connected, verify if the NTP service is running. From the root prompt, execute the command:
ps -ef
If the service is running you will see an entry in the list produced for service name “NTP” and a reference to the process “NTPD”. If this does not appear in the list produced, the NTP service is not running and no further action is required.
If the “NTP” service is listed, to stop the service, execute the command: