Update helm charts to support cold migration

To enable cold migration, need to update nova charts in openstack-helm
and helm-toolkit chart in openstack-helm-infra.  These changes build
on existing upstream components which attempt to add a second container
to the nova-compute pod which creates a sshd process listening on port
8022.

Nova chart changes include:
- Fix bug in ssh-config mapping so config file is generated properly in
  /root/.ssh/config in nova-compute container.
- Move private key from sshd container to nova-compute container.
- Map private and public ssh keys to new configmap-ssh which will
  default to acceptable file permissions (400) for ssh.  Keys will be
  provided in overrides.
- Add additional config to /etc/ssh/sshd_config to allow passwordless
  root logins over appropriate subnet passed in from overrides.  This
  is the same as what is done in nova puppet currently.
- Remove chmods from sshd bash script as they are failing.  Function is
  replaced by configmap-ssh.

To enable cold migration in nova helm chart, we need to allow multiple
containers within the same daemonset pod.  This requires a patch to
the helm-toolkit _daemonset_overrides template to remove upstream
restriction.  This issue is tracked upstream by storyboard 2003876.

These changes should be upstreamed but may require further refinement.

Story: 2003909
Task: 28927
Change-Id: Id789ba051cec019e8b7564c713cf1b5296ecf9f6
Signed-off-by: Gerry Kopec <Gerry.Kopec@windriver.com>
This commit is contained in:
Gerry Kopec 2019-01-09 20:52:05 -05:00
parent 83353f1518
commit 6e74844f72
6 changed files with 215 additions and 2 deletions

View File

@ -5,4 +5,4 @@ TAR="$TAR_NAME-$SHA.tar.gz"
COPY_LIST="${CGCS_BASE}/downloads/$TAR $PKG_BASE/files/*"
TIS_PATCH_VER=5
TIS_PATCH_VER=6

View File

@ -18,6 +18,7 @@ BuildArch: noarch
Patch01: 0001-gnocchi-chart-updates.patch
Patch02: Mariadb-Support-adoption-of-running-single-node-mari.patch
Patch03: Mariadb-Share-container-PID-namespaces-under-docker.patch
Patch04: 0004-Allow-multiple-containers-per-daemonset-pod.patch
BuildRequires: helm
@ -29,6 +30,7 @@ Openstack Helm Infra charts
%patch01 -p1
%patch02 -p1
%patch03 -p1
%patch04 -p1
%build
# initialize helm and build the toolkit

View File

@ -0,0 +1,35 @@
From 26844aac43f76afc65ed907fc94ab83ca93c86ae Mon Sep 17 00:00:00 2001
From: Gerry Kopec <Gerry.Kopec@windriver.com>
Date: Wed, 9 Jan 2019 20:11:33 -0500
Subject: [PATCH] Allow multiple containers per daemonset pod
Remove code that restricted daemonset pods to single containers.
Container names will default to name from helm chart template without
hostname and sha though the pod will still have them.
May require further refinement before this can be upstreamed.
---
helm-toolkit/templates/utils/_daemonset_overrides.tpl | 7 -------
1 file changed, 7 deletions(-)
diff --git a/helm-toolkit/templates/utils/_daemonset_overrides.tpl b/helm-toolkit/templates/utils/_daemonset_overrides.tpl
index 8ba2241..b960a84 100644
--- a/helm-toolkit/templates/utils/_daemonset_overrides.tpl
+++ b/helm-toolkit/templates/utils/_daemonset_overrides.tpl
@@ -217,13 +217,6 @@ limitations under the License.
{{- if not $context.Values.__daemonset_yaml.metadata.name }}{{- $_ := set $context.Values.__daemonset_yaml.metadata "name" dict }}{{- end }}
{{- $_ := set $context.Values.__daemonset_yaml.metadata "name" $current_dict.dns_1123_name }}
- {{/* set container name
- assume not more than one container is defined */}}
- {{- $container := first $context.Values.__daemonset_yaml.spec.template.spec.containers }}
- {{- $_ := set $container "name" $current_dict.dns_1123_name }}
- {{- $cont_list := list $container }}
- {{- $_ := set $context.Values.__daemonset_yaml.spec.template.spec "containers" $cont_list }}
-
{{/* cross-reference configmap name to container volume definitions */}}
{{- $_ := set $context.Values "__volume_list" list }}
{{- range $current_volume := $context.Values.__daemonset_yaml.spec.template.spec.volumes }}
--
1.8.3.1

View File

@ -5,4 +5,4 @@ TAR="$TAR_NAME-$SHA.tar.gz"
COPY_LIST="${CGCS_BASE}/downloads/$TAR $PKG_BASE/files/* "
TIS_PATCH_VER=7
TIS_PATCH_VER=8

View File

@ -23,6 +23,7 @@ Patch02: 0002-Add-Aodh-Chart.patch
Patch03: 0003-Add-Panko-Chart.patch
Patch04: Remove-stale-Apache2-service-pids-when-a-POD-starts.patch
Patch05: 0005-Add-heat-purge-deleted-cron-job.patch
Patch06: 0006-Enable-cold-migration-in-nova-helm-chart.patch
BuildRequires: helm
BuildRequires: openstack-helm-infra
@ -38,6 +39,7 @@ Openstack Helm charts
%patch03 -p1
%patch04 -p1
%patch05 -p1
%patch06 -p1
%build
# initialize helm and build the toolkit

View File

@ -0,0 +1,174 @@
From 7760815c98231ffd431f053f8fac35902f420118 Mon Sep 17 00:00:00 2001
From: Gerry Kopec <Gerry.Kopec@windriver.com>
Date: Thu, 10 Jan 2019 00:12:21 -0500
Subject: [PATCH] Enable cold migration in nova helm chart
- Move private key from sshd container to nova-compute container.
- Map private and public keys to configmap-ssh which will default to
correct file permissions.
- Add additional config to /etc/ssh/sshd_config to allow passwordless
root logins over appropriate subnet passed in from overrides.
- Remove chmods from sshd bash script as they are failing.
Depends on helm-toolkit supporting multiple containers per pod.
---
nova/templates/bin/_ssh-start.sh.tpl | 19 ++++++++++++++++---
nova/templates/configmap-etc.yaml | 4 ++--
nova/templates/configmap-ssh.yaml | 35 +++++++++++++++++++++++++++++++++++
nova/templates/daemonset-compute.yaml | 14 +++++++++-----
nova/values.yaml | 5 +++++
5 files changed, 67 insertions(+), 10 deletions(-)
create mode 100755 nova/templates/configmap-ssh.yaml
diff --git a/nova/templates/bin/_ssh-start.sh.tpl b/nova/templates/bin/_ssh-start.sh.tpl
index 1c10cb0..158090b 100644
--- a/nova/templates/bin/_ssh-start.sh.tpl
+++ b/nova/templates/bin/_ssh-start.sh.tpl
@@ -33,8 +33,21 @@ if [[ $(stat -c %U:%G ~nova/.ssh) != "nova:nova" ]]; then
chown nova: ~nova/.ssh
fi
-chmod 0600 ~root/.ssh/authorized_keys
-chmod 0600 ~root/.ssh/id_rsa
-chmod 0600 ~root/.ssh/id_rsa.pub
+{{- if .Values.network.sshd.enabled }}
+subnet_address="{{- .Values.network.sshd.from_subnet -}}"
+cat > /tmp/sshd_config_extend <<EOF
+
+# This Match block prevents Password Authentication for root user
+Match User root
+ PasswordAuthentication no
+
+# This Match Block is used to allow Root Login exceptions over the
+# internal subnet used by Nova Migrations
+Match Address $subnet_address
+ PermitRootLogin without-password
+EOF
+cat /tmp/sshd_config_extend >> /etc/ssh/sshd_config
+rm /tmp/sshd_config_extend
+{{- end }}
exec /usr/sbin/sshd -D -e -o Port=$SSH_PORT
diff --git a/nova/templates/configmap-etc.yaml b/nova/templates/configmap-etc.yaml
index 55aa311..0d1e7a5 100644
--- a/nova/templates/configmap-etc.yaml
+++ b/nova/templates/configmap-etc.yaml
@@ -232,8 +232,8 @@ data:
logging.conf: {{ include "helm-toolkit.utils.to_oslo_conf" .Values.conf.logging | b64enc }}
nova-ironic.conf: {{ include "helm-toolkit.utils.to_oslo_conf" .Values.conf.nova_ironic | b64enc }}
{{- include "helm-toolkit.snippets.values_template_renderer" (dict "envAll" $envAll "template" .Values.conf.wsgi_placement "key" "wsgi-nova-placement.conf" "format" "Secret" ) | indent 2 }}
-# FIXME(portdirect): why is this file suffixed .sh?
-{{- include "helm-toolkit.snippets.values_template_renderer" (dict "envAll" $envAll "template" .Values.conf.ssh "key" "ssh-config.sh" "format" "Secret" ) | indent 2 }}
+{{- include "helm-toolkit.snippets.values_template_renderer" (dict "envAll" $envAll "template" .Values.conf.ssh "key" "ssh-config" "format" "Secret" ) | indent 2 }}
+
{{- end }}
{{- end }}
{{- if .Values.manifests.configmap_etc }}
diff --git a/nova/templates/configmap-ssh.yaml b/nova/templates/configmap-ssh.yaml
new file mode 100755
index 0000000..bab8e33
--- /dev/null
+++ b/nova/templates/configmap-ssh.yaml
@@ -0,0 +1,35 @@
+{{/*
+Copyright 2019 The Openstack-Helm Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/}}
+
+{{- define "nova.configmap.ssh" }}
+{{- $envAll := index . 1 }}
+{{- with $envAll }}
+---
+apiVersion: v1
+kind: Secret
+metadata:
+ name: nova-ssh
+type: Opaque
+data:
+ ssh-key-private: {{ .Values.conf.ssh_private | b64enc }}
+{{- include "helm-toolkit.snippets.values_template_renderer" (dict "envAll" $envAll "template" .Values.conf.ssh_public "key" "ssh-key-public" "format" "Secret" ) | indent 2 }}
+
+{{- end }}
+{{- end }}
+
+{{- if .Values.manifests.configmap_etc }}
+{{- list "nova-ssh" . | include "nova.configmap.ssh" }}
+{{- end }}
diff --git a/nova/templates/daemonset-compute.yaml b/nova/templates/daemonset-compute.yaml
index 850f0b0..82f185f 100644
--- a/nova/templates/daemonset-compute.yaml
+++ b/nova/templates/daemonset-compute.yaml
@@ -217,6 +217,9 @@ spec:
mountPath: /root/.ssh/config
subPath: ssh-config
readOnly: true
+ - name: nova-ssh
+ mountPath: /root/.ssh/id_rsa
+ subPath: ssh-key-private
{{- if .Values.conf.ceph.enabled }}
- name: etcceph
mountPath: /etc/ceph
@@ -273,13 +276,10 @@ spec:
mountPath: /var/lib/nova
- name: varliblibvirt
mountPath: /var/lib/libvirt
- - name: nova-etc
- mountPath: /root/.ssh/id_rsa
- subPath: ssh-key-private
- - name: nova-etc
+ - name: nova-ssh
mountPath: /root/.ssh/id_rsa.pub
subPath: ssh-key-public
- - name: nova-etc
+ - name: nova-ssh
mountPath: /root/.ssh/authorized_keys
subPath: ssh-key-public
- name: nova-bin
@@ -295,6 +295,10 @@ spec:
secret:
secretName: {{ $configMapName }}
defaultMode: 0444
+ - name: nova-ssh
+ secret:
+ secretName: nova-ssh
+ defaultMode: 0400
{{- if .Values.conf.ceph.enabled }}
- name: etcceph
emptyDir: {}
diff --git a/nova/values.yaml b/nova/values.yaml
index 4edf5c6..9646ded 100644
--- a/nova/values.yaml
+++ b/nova/values.yaml
@@ -209,6 +209,9 @@ network:
ssh:
name: "nova-ssh"
port: 8022
+ sshd:
+ enabled: false
+ from_subnet: 0.0.0.0/24
dependencies:
dynamic:
@@ -460,6 +463,8 @@ conf:
StrictHostKeyChecking no
UserKnownHostsFile /dev/null
Port {{ .Values.network.ssh.port }}
+ ssh_private: 'null'
+ ssh_public: 'null'
rally_tests:
run_tempest: false
tests:
--
1.8.3.1