Increase keystone-api termination grace period

The keystone-api pod gets stuck in CrashLoopBackOff on
AIO-SX lock/unlock attempt.

When Kubernetes decides to kill the keystone-api pod due to readiness
probe failure or other reasons, it calls the preStop hook immediately
before the container is terminated. This hook starts a graceful
shutdown process which includes removing pid, shared memory segment
and wsgi sock files. If the container is not terminated within the
grace period, a SIGKILL is sent, and the container is forced to shut
down.

When the container was forced to terminate without clean up, the
stale files were left behind. On the restart, the application
detected the file existed, and treated it as configuration failure,
hence the exit. As a result, the pod went into a crash loop.

This update sets termination grace period to 60 seconds in
system override and it gives the process more time to gracefully
shut down.

Story: 2004520
Task: 28392

Change-Id: Ibae1e8bf7d5d43bb946a695e565fe204a2077f34
Signed-off-by: Tao Liu <tao.liu@windriver.com>
This commit is contained in:
Tao Liu 2019-01-15 21:03:04 -05:00
parent 7dd943fe46
commit 7f65cec8da
2 changed files with 8 additions and 1 deletions

View File

@ -1,2 +1,2 @@
SRC_DIR="sysinv"
TIS_PATCH_VER=294
TIS_PATCH_VER=295

View File

@ -57,6 +57,13 @@ class KeystoneHelm(openstack.OpenstackBaseHelm):
overrides = {
'replicas': {
'api': self._num_controllers()
},
'lifecycle': {
'termination_grace_period': {
'api': {
'timeout': 60
}
}
}
}
if self.docker_repo_source != common.DOCKER_SRC_OSH: