Ensure drbd-cephmon becomes primary

Sometimes BnR doesn't complete because the drbd-cephmon
manifest isn't applied when trying to make it primary.
After testing, it was identified that this is caused
by a race condition.

Therefore, to ensure that drbd-cephmon becomes primary,
as this is an intermittent issue, a script was added
that contains a 'for loop', where up to 10 attempts are
made to execute the drbdadm command with an interval of
100ms between each one.

Additionally, in some tests the target was not executed due
to onlyif, when the state was other than 'inconsistent'.
In this case, since the drbd_make_primary_drbd-cephmon
target is running only when it is an initial setup, the
onlyif attribute ends up being redundant and there is no
need to have it.

Although the 'overwrite-data-of-peer' argument has been
replaced by 'force', they are both the same thing, they
are aliases, as can be seen at:
https://linbit.com/man/v84/?linbitman=drbdsetup.8.html

Test Plan:
PASS: Build puppet-drbd package
PASS: Backup and restore on AIO-DX system

Closes-Bug: 2031542
Depends-On: https://review.opendev.org/c/starlingx/ansible-playbooks/+/900555

Change-Id: I497bb02123fb9e4a48424d8bc36325cfd6268199
Signed-off-by: Erickson Silva de Oliveira <Erickson.SilvadeOliveira@windriver.com>
This commit is contained in:
Erickson Silva de Oliveira 2023-11-07 21:12:44 -03:00
parent 3a0e43e056
commit 919a5ab781
2 changed files with 57 additions and 0 deletions

View File

@ -0,0 +1,56 @@
From 463dec51aae0a996a3ca9c781a6d3eeffd501bd2 Mon Sep 17 00:00:00 2001
From: Erickson Silva de Oliveira <Erickson.SilvadeOliveira@windriver.com>
Date: Thu, 9 Nov 2023 15:00:31 -0300
Subject: [PATCH] Ensure drbd becomes primary
Sometimes BnR does not complete because the drbd-cephmon manifest
is not applied when trying to make it primary. To ensure that
drbd becomes primary, a loop was added so that multiple
attempts are made, not just one.
Signed-off-by: Erickson Silva de Oliveira <Erickson.SilvadeOliveira@windriver.com>
---
manifests/resource/up.pp | 17 ++++++++++++++---
1 file changed, 14 insertions(+), 3 deletions(-)
diff --git a/manifests/resource/up.pp b/manifests/resource/up.pp
index 8d2f901..732634b 100644
--- a/manifests/resource/up.pp
+++ b/manifests/resource/up.pp
@@ -72,12 +72,23 @@ define drbd::resource::up (
# these resources should only be applied if we are configuring the
# primary node in our HA setup
if $ha_primary {
+ $cmd = "/bin/true # comment to satisfy puppet syntax requirements
+set -ex
+if ! drbdadm primary ${name} ; then
+ for i in {1..10}; do
+ if drbdadm primary ${name} --force ; then
+ exit 0
+ fi
+ sleep 0.1
+ done
+ drbdadm primary ${name} --force
+fi
+"
# these things should only be done on the primary during initial setup
if $initial_setup {
exec { "drbd_make_primary_${name}":
- command => "drbdadm -- --overwrite-data-of-peer primary ${name}",
+ command => $cmd,
unless => "drbdadm role ${name} | egrep '^Primary'",
- onlyif => "drbdadm dstate ${name} | egrep '^Inconsistent'",
notify => Exec["drbd_format_volume_${name}"],
before => Exec["drbd_make_primary_again_${name}"],
require => Service['drbd'],
@@ -96,7 +107,7 @@ define drbd::resource::up (
}
exec { "drbd_make_primary_again_${name}":
- command => "drbdadm primary ${name}",
+ command => $cmd,
unless => "drbdadm role ${name} | egrep '^Primary'",
require => Service['drbd'],
}
--
2.34.1

View File

@ -10,3 +10,4 @@
0010-Format-DRBD-resource-cpu-mask-to-support-64-or-larger-cpus.patch
0011-Fix-DRBD-cephmon-resize.patch
0012-Convert-strings-to-Numeric-type-to-avoid-warnings.patch
0013-Ensure-drbd-becomes-primary.patch