Add Kata Containers spec
Story: 2010765 Task: 48073 Signed-off-by: mpaulool <Marcos.PauloOliveiraSilva@windriver.com> Change-Id: I42e9b827f276252eb686bb161e4fe3d88336e78f
This commit is contained in:
parent
f46bb1a5b4
commit
36103ca1f7
|
@ -0,0 +1,229 @@
|
||||||
|
..
|
||||||
|
This work is licensed under a Creative Commons Attribution 3.0 Unported
|
||||||
|
License. http://creativecommons.org/licenses/by/3.0/legalcode
|
||||||
|
|
||||||
|
==========================================
|
||||||
|
Reintegrate Kata Containers into StarlingX
|
||||||
|
==========================================
|
||||||
|
|
||||||
|
Storyboard: https://storyboard.openstack.org/#!/story/2010765
|
||||||
|
https://storyboard.openstack.org/#!/story/2010781
|
||||||
|
|
||||||
|
This spec reintegrates Kata Containers into StarlingX. When the migration
|
||||||
|
to Debian was made, the Kata containers support wasn't ported. Then, this spec
|
||||||
|
aims to port it.
|
||||||
|
|
||||||
|
Problem description
|
||||||
|
===================
|
||||||
|
Kata Containers is an open source community working to build a secure
|
||||||
|
container runtime with lightweight virtual machines that feel and perform
|
||||||
|
like containers, but provide stronger workload isolation using hardware
|
||||||
|
virtualization technology as a second layer of defense.[0]
|
||||||
|
|
||||||
|
Since there is a performance gap between Kata Containers and conventional
|
||||||
|
containers [1], Kata Containers is not used in default unless explicitly
|
||||||
|
requirement is found in the Kubernetes's pod configuration file. Or in other
|
||||||
|
words, regardless Kata Containers is enabled or not, containers
|
||||||
|
currently we have in StarlingX will not use kata container runtime in default,
|
||||||
|
unless extra change is made.
|
||||||
|
|
||||||
|
Use Cases
|
||||||
|
---------
|
||||||
|
With Kata Containers supported in StarlingX, System developers, testers,
|
||||||
|
operators, administrators could choose what container runtime to use when run
|
||||||
|
a container image, based on their needs. For example, if there is security
|
||||||
|
concern for the pod/container, it could be selected to run with kata container.
|
||||||
|
For the default case, runc will be selected as default low-level container
|
||||||
|
runtime, which is the default low-level runtime for Kubernetes.
|
||||||
|
|
||||||
|
Proposed changes
|
||||||
|
================
|
||||||
|
|
||||||
|
The StarlingX has only one container orchestration tool: the Kubernetes. Which
|
||||||
|
is currently already configured to use containerd as container runtime.
|
||||||
|
As said in the previous sessions, runc will be maintained as the default
|
||||||
|
low-level container runtime. Then, to run a pod with Kata, we need to
|
||||||
|
explicitly declare our intention. There are two methods to do so:
|
||||||
|
|
||||||
|
1. Use annotation. Add “io.kubernetes.cri.untrusted-workload” annotation in pod
|
||||||
|
file.[2]
|
||||||
|
|
||||||
|
2. Use Runtime Class.[3] 1st method is simpler, but it is deprecated.
|
||||||
|
2nd method is recommended. Runtime Class is supported in Kubernetes since
|
||||||
|
v1.14, and current Kubernetes version in StarlingX master is v1.26.
|
||||||
|
So use Runtime Class to select kata container runtime is recommended.
|
||||||
|
|
||||||
|
The Kata Containers version that will be integrated in StarlingX 9.0 is the
|
||||||
|
3.1.3. As were not found precompiled debian packages, then, in build-time,
|
||||||
|
it will be packaged using statically-linked prebuilt binaries from
|
||||||
|
https://github.com/kata-containers/kata-containers/releases.
|
||||||
|
|
||||||
|
And, for compatibility reasons, it will be necessary upgrade the virtualizer
|
||||||
|
QEMU from 5.2 to 7.2 version.
|
||||||
|
|
||||||
|
Data model impact
|
||||||
|
-----------------
|
||||||
|
|
||||||
|
None. This spec does not change any existing data models.
|
||||||
|
|
||||||
|
REST API impact
|
||||||
|
---------------
|
||||||
|
|
||||||
|
None. This spec does not change any existing REST APIs.
|
||||||
|
|
||||||
|
Security impact
|
||||||
|
---------------
|
||||||
|
|
||||||
|
There is no sensitive data touch for this feature, except containerd need
|
||||||
|
touch certificate/key for local secure registry "registry.local".
|
||||||
|
Kata Containers could help enhance system's security by runs in a dedicated
|
||||||
|
kernel, providing isolation of network, I/O and memory and can utilize
|
||||||
|
hardware-enforced isolation with virtualization VT extensions." [0]
|
||||||
|
|
||||||
|
Other end user impact
|
||||||
|
---------------------
|
||||||
|
|
||||||
|
None. This spec does not impact the end user at all.
|
||||||
|
|
||||||
|
Performance Impact
|
||||||
|
------------------
|
||||||
|
|
||||||
|
For containers that doesn't use kata-runtime, the performance should be the
|
||||||
|
same as previous.
|
||||||
|
For containers that use kata-runtime, some performance drop is expected.
|
||||||
|
Per Kata Containers website:"Delivers consistent performance as standard Linux
|
||||||
|
containers; increased isolation with the performance tax of standard virtual
|
||||||
|
machines."[0]
|
||||||
|
Also there are two studies regarding the comparative performance between runc
|
||||||
|
and Kata Containers that analyzes and quantifies the aforementioned gap. [1][4]
|
||||||
|
|
||||||
|
Other deployer impact
|
||||||
|
---------------------
|
||||||
|
|
||||||
|
None. This spec does not impact deployement at all.
|
||||||
|
|
||||||
|
Developer impact
|
||||||
|
----------------
|
||||||
|
|
||||||
|
None. This spec does not impact the development at all.
|
||||||
|
|
||||||
|
Upgrade impact
|
||||||
|
--------------
|
||||||
|
|
||||||
|
None. This spec does not impact the upgrade process at all.
|
||||||
|
|
||||||
|
Implementation
|
||||||
|
==============
|
||||||
|
|
||||||
|
Assignee(s)
|
||||||
|
-----------
|
||||||
|
|
||||||
|
Primary assignee:
|
||||||
|
|
||||||
|
Guilherme Batista Leite
|
||||||
|
|
||||||
|
Other contributors:
|
||||||
|
|
||||||
|
David Liu
|
||||||
|
Davi Frossard
|
||||||
|
Fabio Studyny Higa
|
||||||
|
Marcos Paulo Oliveira Silva
|
||||||
|
Elson Claudio de Oliveira
|
||||||
|
|
||||||
|
Repos Impacted
|
||||||
|
--------------
|
||||||
|
|
||||||
|
* stx-integ
|
||||||
|
* stx-virt
|
||||||
|
* stx-root
|
||||||
|
* stx-metal
|
||||||
|
* stx-tools
|
||||||
|
|
||||||
|
Work Items
|
||||||
|
----------
|
||||||
|
|
||||||
|
Add kata container
|
||||||
|
* Add kata container support
|
||||||
|
|
||||||
|
Add the support in StarlingX for Kata Containers. The prebuilt runtime
|
||||||
|
binaries are downloaded and modified to provide customization support.
|
||||||
|
|
||||||
|
Upgrade QEMU from version 5.2 to version 7.2
|
||||||
|
* Upgrade QEMU and port patches
|
||||||
|
|
||||||
|
To avoid any kind of incompatibility with Kata Containers 3.1.x version,
|
||||||
|
the QEMU upgrade to version 7.2 is necessary.
|
||||||
|
Currently, the QEMU version on the host has WRO-specific patches which
|
||||||
|
will need to be ported to the new version.
|
||||||
|
|
||||||
|
Investigation and Documentation of Kata Containers Limitations
|
||||||
|
* Actualize the existent documentation
|
||||||
|
|
||||||
|
Currently, there is a document [5] that describes the known limitations when
|
||||||
|
using Kata Containers, Although, it is out of date. The new version (3.1.3)
|
||||||
|
has already had some of these limitations corrected.
|
||||||
|
|
||||||
|
Besides upper work items, there are other work items maybe need added during
|
||||||
|
implementation of this spec. Please check the storyboard link in the beginning of
|
||||||
|
this spec to get the full work item list.
|
||||||
|
|
||||||
|
Dependencies
|
||||||
|
============
|
||||||
|
|
||||||
|
This specification depends upon the open source upstream:
|
||||||
|
|
||||||
|
https://github.com/kata-containers/kata-containers
|
||||||
|
https://github.com/qemu/qemu
|
||||||
|
|
||||||
|
|
||||||
|
Testing
|
||||||
|
=======
|
||||||
|
|
||||||
|
System Configurations
|
||||||
|
---------------------
|
||||||
|
|
||||||
|
The system configurations assumed for testing are:
|
||||||
|
|
||||||
|
* AIO-SX
|
||||||
|
* Standard
|
||||||
|
|
||||||
|
Preconditions
|
||||||
|
-------------
|
||||||
|
|
||||||
|
A Runtimeclass should be created for Kata-containers.
|
||||||
|
|
||||||
|
Test Scenarios
|
||||||
|
--------------
|
||||||
|
|
||||||
|
* Basic Test: Kata Container pod checking its kernel version.
|
||||||
|
* Kata Container support for static CPU policy.
|
||||||
|
* Kata Container support for non-static CPU policy.
|
||||||
|
* Kata Container support for SR-IOV.
|
||||||
|
* Kata Container support for hugepage.
|
||||||
|
* Kata Container support for IPV6.
|
||||||
|
|
||||||
|
Documentation Impact
|
||||||
|
====================
|
||||||
|
|
||||||
|
The existent end-user Kata Containers documentation will need to be changed.
|
||||||
|
Some outdated info and references will need to be updated.
|
||||||
|
|
||||||
|
References
|
||||||
|
==========
|
||||||
|
[0]: https://katacontainers.io/
|
||||||
|
[1]: https://www.stackhpc.com/kata-io-1.html
|
||||||
|
[2]: https://github.com/kata-containers/documentation/blob/master/how-to/how-to-use-k8s-with-cri-containerd-and-kata.md#create-an-untrusted-pod-using-kata-containers
|
||||||
|
[3]: https://kubernetes.io/docs/concepts/containers/runtime-class/
|
||||||
|
[4]: https://gkoslovski.github.io/files/latincom-2021.pdf
|
||||||
|
[5]: https://docs.starlingx.io/usertasks/kubernetes/known-limitations.html
|
||||||
|
|
||||||
|
History
|
||||||
|
=======
|
||||||
|
|
||||||
|
.. list-table:: Revisions
|
||||||
|
:header-rows: 1
|
||||||
|
|
||||||
|
* - Release Name
|
||||||
|
- Description
|
||||||
|
* - stx-9.0
|
||||||
|
- Introduced
|
Loading…
Reference in New Issue