starlingx
/
metal
Code Issues Proposed changes

Enable host heartbeat in add handler when not in DOR mode 

Two Node System: VMs did not switch to ERROR state after host reboot

A logically failed (rebooted) active controller is not being
administratively failed by maintenance. As a result the host's
offline availability state is not reported to the VIM and the
VMs on that (rebooted) All-in-one host are not evacuated.

This issue only applies to two node systems because of how the heartbeat
enable of an All-in-one host needs to be held off until its compute 
manifests apply in the DOR case so as to avoid maintenance failing the 
peer controller over a DOR.

The challange in maintenance is to distinguish between this spontaneous
failure and a DOR. For All-in-one hosts, DOR mode is active for a 
whopping 600 seconds ; long enough to account for both sets of manifests
to apply.

It's that long delay that is making this silent fault stand out so 
obviously.

This update uses 'active DOR mode' to decide whether or not to enable a
host's heartbeat in the add handler.

To better handle early active controller failure the qualifier for DOR 
mode was reduced from 20 to 15 minutes. Meaning that maintenance DOR 
mode is activated if its host up time is less than 15 minutes ; rather 
than 20 as it was before this update. Note that normally the active 
controller starts maintenance with an uptime of 5-7 minutes.

Story: 2002995
Task: 23009
Change-Id: I749aefef45b9db6e86a2c6b81d131ebeccc68926
Signed-off-by: David Sullivan <david.sullivan@windriver.com>
This commit is contained in:
Eric MacDonald 2018-07-03 12:40:09 -04:00 committed by David Sullivan
parent c38acc947c
commit 7da4eb945f
2 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
mtce-common/cgts-mtce-common-1.0/maintenance/mtcNodeCtrl.cpp
View File

@ -1248,7 +1248,7 @@ void daemon_service_run ( void )
}
#endif
if ( ts.tv_sec < MTC_MINS_20 )
if ( ts.tv_sec < MTC_MINS_15 )
{
/* CPE DOR window is much greater in CPE since heartbeat
* cannot start until the inactive CPE has run both manifests */

2
mtce-common/cgts-mtce-common-1.0/maintenance/mtcNodeHdlrs.cpp
View File

@ -5607,7 +5607,7 @@ int nodeLinkClass::add_handler ( struct nodeLinkClass::node * node_ptr )
if ( NOT_THIS_HOST )
{
if (( LARGE_SYSTEM ) ||
(( CPE_SYSTEM ) && ( is_controller(node_ptr) == false )))
(( CPE_SYSTEM ) && ( this->dor_mode_active == false )))
{
send_hbs_command ( node_ptr->hostname, MTC_CMD_START_HOST );
}