Add spec for enablement on Arm platform
Story: 2010739 Task: 48551 Signed-off-by: Jackie Huang <jackie.huang@windriver.com> Change-Id: Ibedd7a66b01e5f13a56589a7ac1529b1b347c5eb
This commit is contained in:
parent
7b2f5169a7
commit
f46bb1a5b4
|
@ -0,0 +1,342 @@
|
||||||
|
..
|
||||||
|
This work is licensed under a Creative Commons Attribution 3.0 Unported
|
||||||
|
License. http://creativecommons.org/licenses/by/3.0/legalcode
|
||||||
|
|
||||||
|
=====================================
|
||||||
|
StarlingX: Enablement On Arm Platform
|
||||||
|
=====================================
|
||||||
|
|
||||||
|
Storyboard: `#2010739`_
|
||||||
|
|
||||||
|
The objective of this spec is to port and enable StarlingX to support Arm
|
||||||
|
architecture platform.
|
||||||
|
|
||||||
|
Problem description
|
||||||
|
===================
|
||||||
|
|
||||||
|
The Arm platform has been widely used more and more, cloud providers and
|
||||||
|
independent vendors now can provide various Arm platforms, e.g., AWS, Google
|
||||||
|
and Ampere. Arm platform is widely used in edge/cloud servers, IoT and
|
||||||
|
embedded systems. For now StarlingX only works on x86 platform, it assumes
|
||||||
|
x86 is the only cpu architecture in many places, including build system(Debian
|
||||||
|
based), pre-built packages, container images, package meta data, bootstrap
|
||||||
|
scripts, etc. It can't work on Arm platform without porting work.
|
||||||
|
|
||||||
|
Use Cases
|
||||||
|
---------
|
||||||
|
|
||||||
|
After the enablement on Arm platform, the user will be able to install and
|
||||||
|
deploy StarlingX on validated Arm architecture servers, Telecommunication
|
||||||
|
operators have more choices in hardware servers.
|
||||||
|
|
||||||
|
Proposed change
|
||||||
|
===============
|
||||||
|
|
||||||
|
The proposed change here is to extend the support to Arm64 architecture, which
|
||||||
|
will modify the StarlingX Debian build system to add support native builds on
|
||||||
|
Arm64 servers, modify the package lists, container images, package meta data,
|
||||||
|
bootstrap scripts and so on to also handle for Arm architecture, and the
|
||||||
|
produced runtime will be validated using Ampere Altra Processor based servers.
|
||||||
|
|
||||||
|
The change will also include adding documentations of the support for Arm64
|
||||||
|
architecture.
|
||||||
|
|
||||||
|
Alternatives
|
||||||
|
------------
|
||||||
|
|
||||||
|
Cross-compile for Arm on x86 platform, which would be more difficult and need
|
||||||
|
more effort. On the other hand, we have Arm servers for native builds provided
|
||||||
|
by Arm company, so support for native builds is the preferred method.
|
||||||
|
|
||||||
|
Data model impact
|
||||||
|
-----------------
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
REST API impact
|
||||||
|
---------------
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
Security impact
|
||||||
|
---------------
|
||||||
|
|
||||||
|
Secure boot support will be disabled on Arm and need extra work to investigate
|
||||||
|
and re-implement for Arm platform.
|
||||||
|
|
||||||
|
Other end user impact
|
||||||
|
---------------------
|
||||||
|
|
||||||
|
* For x86_64 end user: None
|
||||||
|
* For Arm64 end user: there will be docs for Arm64 for these new users.
|
||||||
|
|
||||||
|
Performance Impact
|
||||||
|
------------------
|
||||||
|
|
||||||
|
* For x86_64: None
|
||||||
|
* For Arm64: the performance is not likely to be the same as on x86_64.
|
||||||
|
|
||||||
|
Other deployer impact
|
||||||
|
---------------------
|
||||||
|
|
||||||
|
The documentation will be changed to add Arm64 support, and the deployer needs
|
||||||
|
choose between one of the two supported architectures before deployment.
|
||||||
|
|
||||||
|
Developer impact
|
||||||
|
----------------
|
||||||
|
|
||||||
|
The documentation for developers (developer guide) will be updated with detail
|
||||||
|
items of the impact, e.g.
|
||||||
|
|
||||||
|
* Avoid hardcoded arch names like 'x86-64' or 'amd64' in the source codes.
|
||||||
|
* Consider if there are any different patches or behavior for x86 and Arm.
|
||||||
|
* Different package lists for x86 and Arm.
|
||||||
|
* Different container image for x86 and Arm.
|
||||||
|
|
||||||
|
Upgrade impact
|
||||||
|
--------------
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
Implementation
|
||||||
|
==============
|
||||||
|
|
||||||
|
Assignee(s)
|
||||||
|
-----------
|
||||||
|
|
||||||
|
Primary assignee:
|
||||||
|
|
||||||
|
* Jackie Huang (jackie-huang)
|
||||||
|
|
||||||
|
Other contributors:
|
||||||
|
|
||||||
|
* Tianyu Li (tianyuli)
|
||||||
|
|
||||||
|
Repos Impacted
|
||||||
|
--------------
|
||||||
|
|
||||||
|
* starlingx/ansible-playbooks
|
||||||
|
* starlingx/config
|
||||||
|
* starlingx/containers
|
||||||
|
* starlingx/fault
|
||||||
|
* starlingx/ha
|
||||||
|
* starlingx/integ
|
||||||
|
* starlingx/kernel
|
||||||
|
* starlingx/metal
|
||||||
|
* starlingx/stx-puppet
|
||||||
|
* starlingx/tools
|
||||||
|
* starlingx/utilities
|
||||||
|
* starlingx/virt
|
||||||
|
* starlintx/root
|
||||||
|
* starlingx/jenkins-pipelines
|
||||||
|
* starlingx/app-harbor
|
||||||
|
* starlingx/app-istio
|
||||||
|
* starlingx/app-kubernetes-power-manager
|
||||||
|
* starlingx/app-kubevirt
|
||||||
|
* starlingx/app-node-feature-discovery
|
||||||
|
* starlingx/app-oran-o2
|
||||||
|
* starlingx/app-security-profiles-operator
|
||||||
|
* starlingx/app-sriov-fec-operator
|
||||||
|
* starlingx/app-sts-silicom
|
||||||
|
* starlingx/audit-armada-app
|
||||||
|
* starlingx/cert-manager-armada-app
|
||||||
|
* starlingx/metrics-server-armada-app
|
||||||
|
* starlingx/monitor-armada-app
|
||||||
|
* starlingx/nginx-ingress-controller-armada-app
|
||||||
|
* starlingx/oidc-auth-armada-app
|
||||||
|
* starlingx/openstack-armada-app
|
||||||
|
* starlingx/platform-armada-app
|
||||||
|
* starlingx/portieris-armada-app
|
||||||
|
* starlingx/ptp-notification-armada-app
|
||||||
|
* starlingx/snmp-armada-app
|
||||||
|
* starlingx/vault-armada-app
|
||||||
|
|
||||||
|
Work Items
|
||||||
|
----------
|
||||||
|
|
||||||
|
StarlingX Debian Build Environment: Setup on Native Arm64 Server
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
* Arm company provides the HPE ProLiant RL300 Gen11 servers with Amper
|
||||||
|
Altra Processor.
|
||||||
|
* Extend LAT-SDK to support Arm64.
|
||||||
|
* Build LAT-SDK from wrlinux with BSP qemuarm64 on Arm64 server.
|
||||||
|
* Configure the build environment on Arm64 server.
|
||||||
|
* Rebuild all the container images on Arm64 server, for stx-lat-tool,
|
||||||
|
change the dockerfile to use the LAT-SDK built from qemuarm64.
|
||||||
|
|
||||||
|
StarlingX Debian Build System: Extend to support Arm64 architecture
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
Build-tools
|
||||||
|
"""""""""""
|
||||||
|
|
||||||
|
The packages lists and image yaml files may differ for different arch, the
|
||||||
|
build-tools should support arch specific lists and files, which will be named
|
||||||
|
with suffix like _arm64, then the build-tools will check the host arch, and
|
||||||
|
the files named with arch suffix will be used instead of the non-arch-specific
|
||||||
|
files.
|
||||||
|
|
||||||
|
So the supported package lists and files will include:
|
||||||
|
|
||||||
|
* base-bullseye.lst
|
||||||
|
* base-bullseye.yaml
|
||||||
|
* base-bullseye_arm64.lst
|
||||||
|
* base-bullseye_arm64.yaml
|
||||||
|
* base-initramfs-bullseye.yaml
|
||||||
|
* base-initramfs-bullseye_arm64.yaml
|
||||||
|
* debian-image.inc
|
||||||
|
* debian-image_arm64.inc
|
||||||
|
* debian_iso_image.inc
|
||||||
|
* debian_iso_image_arm64.inc
|
||||||
|
* debian_pkg_dirs
|
||||||
|
* debian_pkg_dirs_arm64
|
||||||
|
* debian_pkg_dirs_installer
|
||||||
|
* debian_pkg_dirs_installer_arm64
|
||||||
|
* debian_pkg_dirs_rt
|
||||||
|
* debian_pkg_dirs_rt_arm64
|
||||||
|
* os-std.lst
|
||||||
|
* os-std_arm64.lst
|
||||||
|
|
||||||
|
Package meta data
|
||||||
|
"""""""""""""""""
|
||||||
|
|
||||||
|
The items in dl_path/dl_files in meta_data.yaml may have different values and
|
||||||
|
the patches in the patches or deb_patches directory may be different for arm64
|
||||||
|
arch, the extend change will allow multiple dl_path/dl_files items exist in
|
||||||
|
meta_data.yaml, and multiple series files in patches or deb_patches
|
||||||
|
directories exist, and the ones with arch suffix will be used instead of the
|
||||||
|
non-arch-specific ones.
|
||||||
|
|
||||||
|
As an example, the supported meta data will look like:
|
||||||
|
|
||||||
|
e.g. in meta_data.yaml
|
||||||
|
|
||||||
|
.. code-block:: yaml
|
||||||
|
|
||||||
|
dl_files:
|
||||||
|
pkg-file-amd64.tar.gz:
|
||||||
|
topdir: pkg-amd64
|
||||||
|
url: https://path/to/pkg-file-amd64.tar.gz
|
||||||
|
sha256sum: <sha256sum value>
|
||||||
|
dl_files_arm64:
|
||||||
|
pkg-file-arm64.tar.gz:
|
||||||
|
topdir: pkg-arm64
|
||||||
|
url: https://path/to/pkg-file-arm64.tar.gz
|
||||||
|
sha256sum: <sha256sum value>
|
||||||
|
|
||||||
|
dl_path:
|
||||||
|
name: pkg-name-amd64.tar.gz
|
||||||
|
url: https://path/to/pkg-name-amd64.tar.gz
|
||||||
|
sha256sum: <sha256sum value>
|
||||||
|
dl_path_arm64:
|
||||||
|
name: pkg-name-arm64.tar.gz
|
||||||
|
url: https://path/to/pkg-name-arm64.tar.gz
|
||||||
|
sha256sum: <sha256sum value>
|
||||||
|
|
||||||
|
e.g. for patches/deb_patches
|
||||||
|
|
||||||
|
.. code-block:: shell
|
||||||
|
|
||||||
|
deb_patches
|
||||||
|
├── 0001-deb-patch1.patch
|
||||||
|
├── 0002-deb-patch2.patch
|
||||||
|
├── 0003-arm64-deb-patch1.patch
|
||||||
|
├── series
|
||||||
|
└── series_arm64
|
||||||
|
|
||||||
|
patches
|
||||||
|
├── 0001-patch1.patch
|
||||||
|
├── 0002-patch2.patch
|
||||||
|
├── 0002-arm64-patch1.patch
|
||||||
|
├── series
|
||||||
|
└── series_arm64
|
||||||
|
|
||||||
|
Package deb files
|
||||||
|
"""""""""""""""""
|
||||||
|
|
||||||
|
* control: extend some of the packages to support Arm64 architecture.
|
||||||
|
* rules: use arch variable to replace hardcoded arch names.
|
||||||
|
|
||||||
|
Container images
|
||||||
|
^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
* Some OpenSource container images don't have arm64 on the original registry,
|
||||||
|
need to use ones on different registry url or rebuild them for Arm64 and
|
||||||
|
push to different registry.
|
||||||
|
* In ansible playbook, a new task will be added to check the arch and reset
|
||||||
|
the images url.
|
||||||
|
* Existing docker files may not support Arm64 and will need to be ported.
|
||||||
|
* Docker image and python wheel build scripts in starlingx/root will need
|
||||||
|
to be extended to support multiple architectures.
|
||||||
|
|
||||||
|
Secure boot support
|
||||||
|
^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
* The secure boot support will be disabled in this implementation on Arm64.
|
||||||
|
* Extra work is needed to investigate and re-implement the support for secure
|
||||||
|
boot on Arm64 architecture (in another Story/Spec).
|
||||||
|
|
||||||
|
Dependencies
|
||||||
|
============
|
||||||
|
|
||||||
|
* Arm company provides the HPE ProLiant RL300 Gen11 servers with Amper
|
||||||
|
Altra Processor.
|
||||||
|
* Extend LAT-SDK to support Arm64.
|
||||||
|
* Build LAT-SDK in wrlinux with BSP qemuarm64 on Arm64 server.
|
||||||
|
|
||||||
|
Testing
|
||||||
|
=======
|
||||||
|
|
||||||
|
Build
|
||||||
|
-----
|
||||||
|
|
||||||
|
* Build packages and image on x86 platform to ensure there is no impact after
|
||||||
|
the changes for Arm64.
|
||||||
|
* Build packages and image on Arm64 platform.
|
||||||
|
* Build docker images on Arm64 platform
|
||||||
|
|
||||||
|
System configuration
|
||||||
|
--------------------
|
||||||
|
|
||||||
|
The system deployment configurations that we are assuming for testing are:
|
||||||
|
|
||||||
|
|
||||||
|
* AIO-SX on x86-64 platform
|
||||||
|
* AIO-SX on Arm64 platform
|
||||||
|
* AIO-SX with Low Latency(LL) on Arm64 platform
|
||||||
|
* AIO-DX on Arm64 platforms
|
||||||
|
* AIO-DX(LL) on Arm64 platforms
|
||||||
|
* Standard (2+2) on Arm64 platform
|
||||||
|
* Standard (2+2) on Arm64 platform
|
||||||
|
* DC (AIO-DX for Central + 2 sub-cloud with AIO-SX)
|
||||||
|
* DC (AIO-DX-LL for Central + 2 sub-cloud with AIO-SX-LL)
|
||||||
|
|
||||||
|
Documentation Impact
|
||||||
|
====================
|
||||||
|
|
||||||
|
The documentation will need to be changed to add notes about Arm support,
|
||||||
|
validated Arm servers, difference between x86 and Arm platform, known
|
||||||
|
issues, limitations and etc.
|
||||||
|
|
||||||
|
References
|
||||||
|
==========
|
||||||
|
|
||||||
|
* `Starlingx-discuss Arm engagement`_
|
||||||
|
* `Starlingx-discuss CI/CD setup on Arm platform`_
|
||||||
|
|
||||||
|
History
|
||||||
|
=======
|
||||||
|
|
||||||
|
.. list-table:: Revisions
|
||||||
|
:header-rows: 1
|
||||||
|
|
||||||
|
* - Release Name
|
||||||
|
- Description
|
||||||
|
* - stx-9.0
|
||||||
|
- Introduced
|
||||||
|
|
||||||
|
.. Links
|
||||||
|
.. _#2010739: https://storyboard.openstack.org/#!/story/2010739
|
||||||
|
.. _Starlingx-discuss Arm engagement: https://lists.starlingx.io/pipermail/starlingx-discuss/2023-May/014160.html
|
||||||
|
.. _Starlingx-discuss CI/CD setup on Arm platform: https://lists.starlingx.io/pipermail/starlingx-discuss/2023-July/014354.html
|
Loading…
Reference in New Issue