starlingx
/
integ
Code Issues Proposed changes

Fix Ceph processes start race condition 

When nodes are unlocked, mtc will start ceph processes after a
successful boot. It was also observed at the end of AIO-SX optimized
restore playbook.
This is causing racing condition with sm and pmon leading to failure
in some scenarios.

To avoid this, the script called by mtc will not start ceph processes
anymore. It will only set the flag to enable ceph to run on the node
and the processes will be started by sm or pmon later.

Test Plan:
  For each installation setup do:
    - Fresh install and verify Ceph is running with HEALTH_OK status;
    - Swact controllers and verify Ceph has HEALTH_OK status;
    - Run DOR (Dead Office Recover) and verify Ceph has HEALTH_OK
      status;
    - Lock/Unlock Controllers/Storage nodes and check Ceph has
      HEALTH_OK status;
    - Reboot active Controller and check Ceph has HEALTH_OK status.

  PASS: AIO-SX
  PASS: AIO-DX
  PASS: Standard (2+2)
  PASS: Standard with dedicated Storage (2+2+2)
  PASS: B&R AIO-SX
  PASS: B&R Optimized AIO-SX

Closes-bug: 2023445
Change-Id: I0c81749c6db1e17761aa8aca6276eff50f135959
Signed-off-by: Felipe Sanches Zanoni <Felipe.SanchesZanoni@windriver.com>
This commit is contained in:
Felipe Sanches Zanoni 2023-06-10 09:41:48 -03:00
parent d52137d9c7
commit ad20667beb
1 changed files with 5 additions and 12 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

17
ceph/ceph/files/ceph.sh
View File

@ -26,17 +26,10 @@ logecho ()
start ()
{
if [[ "$system_type" != "All-in-one" ]] || [[ "$system_mode" != "duplex" ]]; then
logecho "Starting ceph services..."
${INITDIR}/ceph start >> ${LOGFILE} 2>&1
RC=$?
else
# In an AIO-DX configuration SM manages the floating MON and OSDs and pmon manages
# the ceph-mds process. Here we defer starting all ceph process to allow SM and pmon
# to start them at the appropriate time.
RC=0
fi
# Defer ceph initialization to avoid race conditions. Let SM and Pmon to start the
# processes in the appropriate time.
# Set the flag to let ceph start later.
logecho "Setting flag to enable ceph processes to start."
if [ ! -f ${CEPH_FILE} ]; then
touch ${CEPH_FILE}
fi
@ -55,7 +48,7 @@ stop ()
rm -f ${CEPH_FILE}
fi
${INITDIR}/ceph stop >> ${LOGFILE} 2>&1
${INITDIR}/ceph-init-wrapper stop >> ${LOGFILE} 2>&1
RC=$?
}