starlingx
/
specs
Code Issues Proposed changes

Update spec of kubernetes root CA certificate update 

Some minor adjustments to kubernetes root CA certficate update
spec based on implementation for accuracy.

Story: 2008675
Signed-off-by: Andy Ning <andy.ning@windriver.com>
Change-Id: I0eecee32e2ca17c64194151d8e96076c4754f7b6
This commit is contained in:
Andy Ning 2021-09-07 09:40:31 -04:00
parent f738144690
commit bf8218355f
1 changed files with 194 additions and 102 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

296
doc/source/specs/stx-6.0/approved/security-2008675-kubernetes-rootca-update.rst
View File

@ -14,6 +14,9 @@ This feature introduces CLI/REST APIs and execution orchestration for updating
Kubernetes root CA certficate and certificates issued by the root CA in a Kubernetes root CA certficate and certificates issued by the root CA in a
rolling fashion so that the impact on the system is minimized. rolling fashion so that the impact on the system is minimized.
This is the updated version of the approved spec security-2008675-kubernetes-rootca-update.rst.
This version reflects the adjustments from implementation.
Problem description Problem description
=================== ===================
@ -84,11 +87,12 @@ Sysinv operations for root CA certificate update
A new set of sysinv CLI commands will be introduced to simplify the update A new set of sysinv CLI commands will be introduced to simplify the update
procedure. It will be a procedure similar to software upgrade, with a start, procedure. It will be a procedure similar to software upgrade, with a start,
execute and complete cycle. There won't be support for "abort", but user can execute and complete cycle. User can retry a step if it fails. There will also
retry the command if it fails. And user can choose to restart the update be support for "abort", where user can choose to exit an on-going update. But
procedure by uploading or re-generating a new root CA certficate. This also the user is supposed to restart the update procedure with either uploading or
provides a mechanism to resume to the original CA certificate if user chooses re-generating a root CA certficate and run the update to full complete. This
to upload the original CA certificate. also provides a mechanism to restore the original CA certificate if user
chooses to upload the original CA certificate.
The following is a summary of the CLI commands and the steps to perform The following is a summary of the CLI commands and the steps to perform
kubernetes root CA certificate update. kubernetes root CA certificate update.
@ -112,28 +116,28 @@ kubernetes root CA certificate update.
certificate and private key from a file instead of generating one certificate and private key from a file instead of generating one
* Change progress state to update-new-rootca-cert-uploaded * Change progress state to update-new-rootca-cert-uploaded
3. system kube-rootca-host-update <hostname> --phase=trustBothCAs 3. system kube-rootca-host-update <hostname> --phase=trust-both-cas
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* Update apiserver's trusted CAs to include the new CA cert * Update apiserver's trusted CAs to include the new CA cert
* Update scheduler's trusted CAs to include the new CA cert * Update scheduler's trusted CAs to include the new CA cert
* Update controller-manager's trusted CAs to include the new CA cert * Update controller-manager's trusted CAs to include the new CA cert
* Update kubelet's trusted CAs to include the new CA cert * Update kubelet's trusted CAs to include the new CA cert
* Update admin.conf's trusted CAs to include the new CA cert * Update admin.conf's trusted CAs to include the new CA cert
* Change progress state to updated-host-trustBothCAs on success * Change progress state to updated-host-trust-both-cas on success
* Change progress state to updating-host-trustBothCAs-failed on failure * Change progress state to updating-host-trust-both-cas-failed on failure
4. system kube-rootca-pods-update --phase=trustBothCAs 4. system kube-rootca-pods-update --phase=trust-both-cas
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* Annotate Daemonsets and Deployments to trigger pod replacement in a safer * Annotate Daemonsets and Deployments to trigger pod replacement in a safer
rolling fashion, to ensure pods to pick up the new root CA cert as its trusted rolling fashion, to ensure pods to pick up the new root CA cert as its trusted
CA along with the old root CA certificate CA along with the old root CA certificate
* Change progess state to updated-pods-trustBothCAs on success * Change progess state to updated-pods-trust-both-cas on success
* Change progess state to updating-pods-trustBothCAs-failed on failure * Change progess state to updating-pods-trust-both-cas-failed on failure
5. system kube-rootca-host-update <hostname> --phase=updateCerts 5. system kube-rootca-host-update <hostname> --phase=update-certs
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* Update admin.conf's client cert/key data with new ones signed by the * Update admin.conf's client cert/key data with new ones signed by the
new root CA new root CA
@ -143,27 +147,27 @@ kubernetes root CA certificate update.
* Update controller-manager's client cert/key with new one signed by the new * Update controller-manager's client cert/key with new one signed by the new
root CA root CA
* Update kubelet's client cert/key with new one signed by the new root CA * Update kubelet's client cert/key with new one signed by the new root CA
* Change progress state to updated-host-updateCerts on success * Change progress state to updated-host-update-certs on success
* Chante progress state to updating-host-updateCerts-failed on failure * Chante progress state to updating-host-update-certs-failed on failure
6. system kube-rootca-host-update <hostname> --phase=trustNewCA 6. system kube-rootca-host-update <hostname> --phase=trust-new-ca
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* Update admin.conf's trusted CAs to remove the old root CA * Update admin.conf's trusted CAs to remove the old root CA
* Update apiserver's trusted CAs to remove the old root CA * Update apiserver's trusted CAs to remove the old root CA
* Update controller-manager's trusted CAs to remove the old root CA * Update controller-manager's trusted CAs to remove the old root CA
* Update scheduler's trusted CAs to remove the old root CA * Update scheduler's trusted CAs to remove the old root CA
* Update kubelet's trusted CAs to remove the old root CA * Update kubelet's trusted CAs to remove the old root CA
* Change progress state to updated-host-trustNewCA on success * Change progress state to updated-host-trust-new-ca on success
* Change progress state to updating-host-trustNewCA-failed on failure * Change progress state to updating-host-trust-new-ca-failed on failure
7. system kube-rootca-pods-update --phase=trustNewCA 7. system kube-rootca-pods-update --phase=trust-new-ca
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* Annotate Daemonsets and Deployments to trigger pod replacement in a safer * Annotate Daemonsets and Deployments to trigger pod replacement in a safer
rolling fashion, to remove the old root CA from pods trusted CA list rolling fashion, to remove the old root CA from pods trusted CA list
* Change progress state to updated-pods-trustNewCA on success * Change progress state to updated-pods-trust-new-ca on success
* Change progress state to updating-pods-trustNewCA-failed on failure * Change progress state to updating-pods-trust-new-ca-failed on failure
8. system kube-rootca-host-update complete 8. system kube-rootca-host-update complete
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@ -171,17 +175,22 @@ kubernetes root CA certificate update.
* Post-check to verify the update * Post-check to verify the update
* Change the progress state to update-complete * Change the progress state to update-complete
system kube-rootca-update-list 9. system kube-rootca-host-update-list
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* Run this command anytime to show the update status of all hosts in the * Run this command anytime to show the update status of all hosts in the
cluster cluster
system kube-rootca-update-show 10. system kube-rootca-update-show
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* Run this command anytime to show the overall update status * Run this command anytime to show the overall update status
11. system kube-rootca-update-abort
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* Run this command to abort the update at any step
VIM Orchestration Operations VIM Orchestration Operations
---------------------------- ----------------------------
@ -272,96 +281,121 @@ each host.
The following is the list of REST resources and APIs to be added: The following is the list of REST resources and APIs to be added:
The new resource /kube_update_ca is added The new resource /kube_rootca_update is added
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* URLS: * URLS:
* /v1/kube_update_ca * /v1/kube_rootca_update
* Request Methods: * Request Methods:
* POST /v1/kube_update_ca * POST /v1/kube_rootca_update
* Creates (starts) a new root CA cert update * Creates (starts) a new root CA cert update
* Response body example:: * Response body example::
{"from_rootca_cert": "kubenetes-5118144266510589551", {"uuid": "47dff2b6-17ba-45a2-b3d3-8b2a85a5dba9",
"to_rootca_cert": null,
"created_at": "2021-08-25T14:57:13.006034+00:00",
"from_rootca_cert": "d70efa2daaee06f8-91764",
"updated_at": null,
"state": "update-started", "state": "update-started",
"uuid": "223ba65e-45d1-4383-baa7-f03bb4c46773", "id": 1}
"created_at": "2021-03-25T12:04:10.372399+00:00",
"updated_at": "2021-03-25T12:04:10.372399+00:00"}
* GET /v1/kube_update_ca * GET /v1/kube_rootca_update
* Return the current kube_update_ca * Return the current root CA update
* Response body example:: * Response body example::
{"from_rootca_cert": "kubenetes-5118144266510589551", {"uuid": "47dff2b6-17ba-45a2-b3d3-8b2a85a5dba9",
"to_rootca_cert": "kubenetes-6118144266510589551", "to_rootca_cert": null,
"created_at": "2021-08-25T14:57:13.006034+00:00",
"from_rootca_cert": "d70efa2daaee06f8-91764",
"updated_at": null,
"state": "update-started", "state": "update-started",
"uuid": "223ba65e-45d1-4383-baa7-f03bb4c46773", "id": 1}
"created_at": "2021-03-25T12:04:10.372399+00:00",
"updated_at": "2021-03-25T14:45:43.252964+00:00"}
* PATCH /v1/kube_update_ca * PATCH /v1/kube_rootca_update
* Modifies the current rootca_update. Used to update the state of the * Modifies the current rootca_update. Used to update the state of the
update (e.g. to update_complete). update (e.g. to update_complete, or update_aborted).
* Request body example::
[{"path": "/state",
"value": "update-completed",
"op": "replace"}]
[{"path": "/state",
"value": "update-aborted",
"op": "replace"}]
* Response body example:: * Response body example::
{"from_rootca_cert": "kubenetes-5118144266510589551", {"uuid": "fb882423-ea26-42bf-b645-fd9de4248fd4",
"to_rootca_cert": "kubenetes-6118144266510589551", "to_rootca_cert": "d70efa2daaee06f8-176046114160516196064588947858918572907",
"state": "update-complete", "created_at": "2021-08-24T13:40:13.318822+00:00",
"uuid": "223ba65e-45d1-4383-baa7-f03bb4c46773", "from_rootca_cert": "d70efa2daaee06f8-199590289735612744821302170157251522966",
"created_at": "2021-03-25T12:04:10.372399+00:00", "updated_at": "2021-08-24T13:52:21.547899+00:00",
"updated_at": "2021-03-25T14:45:43.252964+00:00"} "state": "update-completed",
"id": 20}
* DELETE /v1/kube_update_ca {"uuid": "7d07e384-f06d-4213-8e61-5e300aeb9d1c",
"to_rootca_cert": null,
"created_at": "2021-08-24T13:38:55.376395+00:00",
"from_rootca_cert": "d70efa2daaee06f8-199590289735612744821302170157251522966",
"updated_at": "2021-08-24T13:39:47.108582+00:00",
"state": "update-aborted",
"id": 19}
* Deletes the current rootca_update (after it is completed) The new resource /kube_rootca_update/upload_cert is added
The new resource /kube_rootca_certificate/upload is added
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* URLS: * URLS:
* /v1/kube_rootca_certificate/upload * /v1/kube_rootca_update/upload_cert
* Request Methods: * Request Methods:
* POST /v1/kube_rootca_certificate/upload * POST /v1/kube_rootca_update/upload_cert
* Upload a root CA cert and key from a file * Upload a root CA cert and key from a file
* Request body example:: * Request body example:
(The contents of the body is from a file containing both private key and certificate)::
{"ca.crt": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUMyRENDQWNDZ0..." {"-----BEGIN PRIVATE KEY----- ...... -----END PRIVATE KEY----- ...... -----BEGIN CERTIFICATE----- ...... -----END CERTIFICATE-----}
"ca.key": "LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFcGdJQk..."}
* Return body example:: * Return body example::
{"cert_id": "kubenetes-5118144266510589551"} {"success": "8503e172a63b23e6-12808492498813125379",
"error": ""}
The new resource /v1/kube_rootca_certificate/generate is added The new resource /v1/kube_rootca_update/generate_cert is added
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* URLS: * URLS:
* /v1/kube_rootca_certificate/generate * /v1/kube_rootca_update/generate_cert
* Request Methods: * Request Methods:
* POST /v1/kube_rootca_certificate/generate * POST /v1/kube_rootca_update/generate_cert
* Tell sysinv to generate a new root CA cert and key pair * Tell sysinv to generate a new root CA cert and key pair
* Request body example::
{"expiry_date": "2022-08-25",
"subject": "C=CA O=Company CN=kubernetes"}
* Return body example:: * Return body example::
{"cert_id": "kubenetes-5118144266510589551"} {"success": "a8942428863f292b-253592702972967198587817983178843995169",
"error": ""}
The existing resource /ihosts is modified to add new actions The existing resource /ihosts is modified to add new actions
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@ -378,51 +412,97 @@ The existing resource /ihosts is modified to add new actions
* Request body example:: * Request body example::
{"phase", "trustBothCAs"} {"phase", "trust-both-cas"}
* Response body example:: * Response body example::
{"id": "4", {"target_rootca_cert": "8503e172a63b23e6-12808492498813125379",
"created_at": "2021-08-25T17:13:22.571151+00:00",
"hostname": "controller-1", "hostname": "controller-1",
"updated_at": "2021-08-25T17:58:59.809264+00:00",
"state": "updating-host-trust-both-cas",
"personality": "controller", "personality": "controller",
"target_rootca_cert": "kubenetes-6118144266510589551", "id": 8,
"effective_rootca_cert": "kubenetes-5118144266510589551", "effective_rootca_cert": "d70efa2daaee06f8-91764",
"state": "updating-host-trustBothCAs"} "uuid": "a597c090-731f-48f8-9f3f-344997c41317"}
The new resource /kube_hosts_update_ca The new resource /kube_rootca_update/hosts is added
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* URLs: * URLs:
* /v1/kube_hosts_update_ca * /v1/kube_rootca_update/hosts
* Request Methods: * Request Methods:
* GET /v1/kube_hosts_update_ca * GET /v1/kube_rootca_update/hosts
* Returns the update details of all hosts * Returns the update details of all hosts
* Response body example:: * Response body example::
{ {
"hosts": [ "kube_host_updates": [
{"id": "2", {"target_rootca_cert": null,
"hostname": "controller-1", "created_at": "2021-08-25T17:13:22.558411+00:00",
"personality": "controller", "hostname": "controller-0",
"target_rootca_cert": "kubenetes-6118144266510589551", "updated_at": null,
"effective_rootca_cert": "kubenetes-5118144266510589551", "state": null,
"state": "updating-host-trustBothCAs" "personality": "controller",
}, "id": 7,
{"id": "4", "effective_rootca_cert": "d70efa2daaee06f8-91764",
"hostname": "compute-0", "uuid": "7d7d05dd-900f-4004-951d-d92536faac8e"
"personality": "compute", },
"target_rootca_cert": "kubenetes-6118144266510589551", {"target_rootca_cert": "8503e172a63b23e6-12808492498813125379",
"effective_rootca_cert": "kubenetes-5118144266510589551", "created_at": "2021-08-25T17:13:22.571151+00:00",
"state": "updating-host-updateCerts" "hostname": "controller-1",
} "updated_at": "2021-08-25T17:59:16.097029+00:00",
] "state": "updated-host-trust-both-cas",
"personality": "controller",
"id": 8,
"effective_rootca_cert": "d70efa2daaee06f8-91764",
"uuid": "a597c090-731f-48f8-9f3f-344997c41317"
},
{"target_rootca_cert": null,
"created_at": "2021-08-25T17:13:22.584500+00:00",
"hostname": "worker-0",
"updated_at": null,
"state": null,
"personality": "worker",
"id": 9,
"effective_rootca_cert": "d70efa2daaee06f8-91764",
"uuid": "a4ca4eed-9b2f-4b4c-8ee7-45bbc573a55f"
}
]
} }
The new resource /kube_rootca_update/pods is added
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* URLs:
* /v1/kube_rootca_update/pods
* Request Methods:
* POST /v1/kube_rootca_update/pods
* Update root CA cert for pods
* Request body example::
{"phase", "trust-both-cas"}
* Response body example::
{"uuid": "6cf4157b-75ff-4e86-bc96-8b08e4c9836d",
"to_rootca_cert": "8503e172a63b23e6-12808492498813125379",
"created_at": "2021-08-25T17:13:22.535798+00:00",
"from_rootca_cert": "d70efa2daaee06f8-91764",
"updated_at": "2021-08-25T18:37:02.574836+00:00",
"state": "updating-pods-trust-both-cas",
"id": 3}
Security impact Security impact
--------------- ---------------
@ -496,8 +576,10 @@ Repos Impacted
-------------- --------------
Impacted repo from this spec: Impacted repo from this spec:
* config * config
* stx-puppet * stx-puppet
* fault
Work Items Work Items
---------- ----------
@ -526,7 +608,7 @@ Sysinv
* root CA certficate and issuer creation in cert-manager * root CA certficate and issuer creation in cert-manager
* calculate the ID of the new root certificate * calculate the ID of the new root certificate
* kube-rootca-host-update <hostname> --phase=trustBothCAs CLI/API * kube-rootca-host-update <hostname> --phase=trust-both-cas CLI/API
* basic infrastructure * basic infrastructure
* semantic checks * semantic checks
@ -535,14 +617,14 @@ Sysinv
* agent RPC/implementation (apply puppet manifest, report back config * agent RPC/implementation (apply puppet manifest, report back config
status, etc...) status, etc...)
* kube-rootca-pods-update --phase=trustBothCAs CLI/API * kube-rootca-pods-update --phase=trust-both-cas CLI/API
* basic infrastructure * basic infrastructure
* semantic checks * semantic checks
* conductor implementation (generate hieradata, trigger puppet * conductor implementation (generate hieradata, trigger puppet
manifests apply, handle apply result, update progress state etc...) manifests apply, handle apply result, update progress state etc...)
* kube-rootca-host-update <hostname> --phase=updateCerts CLI/API * kube-rootca-host-update <hostname> --phase=update-certs CLI/API
* basic infrastructure * basic infrastructure
* semantic checks * semantic checks
@ -552,7 +634,7 @@ Sysinv
* agent RPC/implementation (apply puppet manifest, report back config * agent RPC/implementation (apply puppet manifest, report back config
status, etc...) status, etc...)
* kube-rootca-host-update <hostname> --phase=trustNewCA CLI/API * kube-rootca-host-update <hostname> --phase=trust-new-ca CLI/API
* basic infrastructure * basic infrastructure
* semantic checks * semantic checks
@ -561,7 +643,7 @@ Sysinv
* agent RPC/implementation (apply puppet manifest, report back config * agent RPC/implementation (apply puppet manifest, report back config
status, etc...) status, etc...)
* kube-rootca-pods-update --phase=trustNewCA CLI/API * kube-rootca-pods-update --phase=trust-new-ca CLI/API
* basic infrastructure * basic infrastructure
* semantic checks * semantic checks
@ -578,19 +660,29 @@ Sysinv
* kube-rootca-update-show CLI/API * kube-rootca-update-show CLI/API
* basic infrastructure * basic infrastructure
* condutor database query * conductor database query
* kube-rootca-update-list CLI/API * kube-rootca-host-update-list CLI/API
* basic infrastructure * basic infrastructure
* condutor database query * conductor database query
* kube-rootca-update-abort CLI/API
* basic infrastructure
* semantic checks
* system health checks for update abort
* clear 'kube root CA update in progress' alarm
* raise 'kube root CA update aborted' alarm
Puppet Puppet
^^^^^^ ^^^^^^
* runtime manifest for host update trustBothCAs phase * runtime manifest for host update trust-both-cas phase
* runtime manifest for host update updateCerts phase * runtime manifest for host update update-certs phase
* runtime manifest for host update trustNewCA phase * runtime manifest for host update trust-new-ca phase
* runtime manifest for pods update trust-both-cas phase
* runtime manifest for pods update trust-new-ca phase
System Upgrade System Upgrade
^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^