diff --git a/doc/source/specs/stx-9.0/approved/starlingx-2010740-kata-containers-reintegration.rst b/doc/source/specs/stx-9.0/approved/starlingx-2010740-kata-containers-reintegration.rst new file mode 100644 index 0000000..148b908 --- /dev/null +++ b/doc/source/specs/stx-9.0/approved/starlingx-2010740-kata-containers-reintegration.rst @@ -0,0 +1,229 @@ +.. + This work is licensed under a Creative Commons Attribution 3.0 Unported + License. http://creativecommons.org/licenses/by/3.0/legalcode + +========================================== +Reintegrate Kata Containers into StarlingX +========================================== + +Storyboard: https://storyboard.openstack.org/#!/story/2010765 + https://storyboard.openstack.org/#!/story/2010781 + +This spec reintegrates Kata Containers into StarlingX. When the migration +to Debian was made, the Kata containers support wasn't ported. Then, this spec +aims to port it. + +Problem description +=================== +Kata Containers is an open source community working to build a secure +container runtime with lightweight virtual machines that feel and perform +like containers, but provide stronger workload isolation using hardware +virtualization technology as a second layer of defense.[0] + +Since there is a performance gap between Kata Containers and conventional +containers [1], Kata Containers is not used in default unless explicitly +requirement is found in the Kubernetes's pod configuration file. Or in other +words, regardless Kata Containers is enabled or not, containers +currently we have in StarlingX will not use kata container runtime in default, +unless extra change is made. + +Use Cases +--------- +With Kata Containers supported in StarlingX, System developers, testers, +operators, administrators could choose what container runtime to use when run +a container image, based on their needs. For example, if there is security +concern for the pod/container, it could be selected to run with kata container. +For the default case, runc will be selected as default low-level container +runtime, which is the default low-level runtime for Kubernetes. + +Proposed changes +================ + +The StarlingX has only one container orchestration tool: the Kubernetes. Which +is currently already configured to use containerd as container runtime. +As said in the previous sessions, runc will be maintained as the default +low-level container runtime. Then, to run a pod with Kata, we need to +explicitly declare our intention. There are two methods to do so: + +1. Use annotation. Add “io.kubernetes.cri.untrusted-workload” annotation in pod + file.[2] + +2. Use Runtime Class.[3] 1st method is simpler, but it is deprecated. + 2nd method is recommended. Runtime Class is supported in Kubernetes since + v1.14, and current Kubernetes version in StarlingX master is v1.26. + So use Runtime Class to select kata container runtime is recommended. + +The Kata Containers version that will be integrated in StarlingX 9.0 is the +3.1.3. As were not found precompiled debian packages, then, in build-time, +it will be packaged using statically-linked prebuilt binaries from +https://github.com/kata-containers/kata-containers/releases. + +And, for compatibility reasons, it will be necessary upgrade the virtualizer +QEMU from 5.2 to 7.2 version. + +Data model impact +----------------- + +None. This spec does not change any existing data models. + +REST API impact +--------------- + +None. This spec does not change any existing REST APIs. + +Security impact +--------------- + +There is no sensitive data touch for this feature, except containerd need +touch certificate/key for local secure registry "registry.local". +Kata Containers could help enhance system's security by runs in a dedicated +kernel, providing isolation of network, I/O and memory and can utilize +hardware-enforced isolation with virtualization VT extensions." [0] + +Other end user impact +--------------------- + +None. This spec does not impact the end user at all. + +Performance Impact +------------------ + +For containers that doesn't use kata-runtime, the performance should be the +same as previous. +For containers that use kata-runtime, some performance drop is expected. +Per Kata Containers website:"Delivers consistent performance as standard Linux +containers; increased isolation with the performance tax of standard virtual +machines."[0] +Also there are two studies regarding the comparative performance between runc +and Kata Containers that analyzes and quantifies the aforementioned gap. [1][4] + +Other deployer impact +--------------------- + +None. This spec does not impact deployement at all. + +Developer impact +---------------- + +None. This spec does not impact the development at all. + +Upgrade impact +-------------- + +None. This spec does not impact the upgrade process at all. + +Implementation +============== + +Assignee(s) +----------- + +Primary assignee: + +Guilherme Batista Leite + +Other contributors: + +David Liu +Davi Frossard +Fabio Studyny Higa +Marcos Paulo Oliveira Silva +Elson Claudio de Oliveira + +Repos Impacted +-------------- + +* stx-integ +* stx-virt +* stx-root +* stx-metal +* stx-tools + +Work Items +---------- + +Add kata container +* Add kata container support + + Add the support in StarlingX for Kata Containers. The prebuilt runtime + binaries are downloaded and modified to provide customization support. + +Upgrade QEMU from version 5.2 to version 7.2 +* Upgrade QEMU and port patches + + To avoid any kind of incompatibility with Kata Containers 3.1.x version, + the QEMU upgrade to version 7.2 is necessary. + Currently, the QEMU version on the host has WRO-specific patches which + will need to be ported to the new version. + +Investigation and Documentation of Kata Containers Limitations +* Actualize the existent documentation + + Currently, there is a document [5] that describes the known limitations when + using Kata Containers, Although, it is out of date. The new version (3.1.3) + has already had some of these limitations corrected. + +Besides upper work items, there are other work items maybe need added during +implementation of this spec. Please check the storyboard link in the beginning of +this spec to get the full work item list. + +Dependencies +============ + +This specification depends upon the open source upstream: + +https://github.com/kata-containers/kata-containers +https://github.com/qemu/qemu + + +Testing +======= + +System Configurations +--------------------- + +The system configurations assumed for testing are: + + * AIO-SX + * Standard + +Preconditions +------------- + +A Runtimeclass should be created for Kata-containers. + +Test Scenarios +-------------- + + * Basic Test: Kata Container pod checking its kernel version. + * Kata Container support for static CPU policy. + * Kata Container support for non-static CPU policy. + * Kata Container support for SR-IOV. + * Kata Container support for hugepage. + * Kata Container support for IPV6. + +Documentation Impact +==================== + +The existent end-user Kata Containers documentation will need to be changed. +Some outdated info and references will need to be updated. + +References +========== +[0]: https://katacontainers.io/ +[1]: https://www.stackhpc.com/kata-io-1.html +[2]: https://github.com/kata-containers/documentation/blob/master/how-to/how-to-use-k8s-with-cri-containerd-and-kata.md#create-an-untrusted-pod-using-kata-containers +[3]: https://kubernetes.io/docs/concepts/containers/runtime-class/ +[4]: https://gkoslovski.github.io/files/latincom-2021.pdf +[5]: https://docs.starlingx.io/usertasks/kubernetes/known-limitations.html + +History +======= + +.. list-table:: Revisions + :header-rows: 1 + + * - Release Name + - Description + * - stx-9.0 + - Introduced