diff --git a/doc/source/specs/stx-9.0/approved/starlingx-2010739-enablement-on-Arm-platform.rst b/doc/source/specs/stx-9.0/approved/starlingx-2010739-enablement-on-Arm-platform.rst new file mode 100644 index 0000000..6ee322d --- /dev/null +++ b/doc/source/specs/stx-9.0/approved/starlingx-2010739-enablement-on-Arm-platform.rst @@ -0,0 +1,342 @@ +.. + This work is licensed under a Creative Commons Attribution 3.0 Unported + License. http://creativecommons.org/licenses/by/3.0/legalcode + +===================================== +StarlingX: Enablement On Arm Platform +===================================== + +Storyboard: `#2010739`_ + +The objective of this spec is to port and enable StarlingX to support Arm +architecture platform. + +Problem description +=================== + +The Arm platform has been widely used more and more, cloud providers and +independent vendors now can provide various Arm platforms, e.g., AWS, Google +and Ampere. Arm platform is widely used in edge/cloud servers, IoT and +embedded systems. For now StarlingX only works on x86 platform, it assumes +x86 is the only cpu architecture in many places, including build system(Debian +based), pre-built packages, container images, package meta data, bootstrap +scripts, etc. It can't work on Arm platform without porting work. + +Use Cases +--------- + +After the enablement on Arm platform, the user will be able to install and +deploy StarlingX on validated Arm architecture servers, Telecommunication +operators have more choices in hardware servers. + +Proposed change +=============== + +The proposed change here is to extend the support to Arm64 architecture, which +will modify the StarlingX Debian build system to add support native builds on +Arm64 servers, modify the package lists, container images, package meta data, +bootstrap scripts and so on to also handle for Arm architecture, and the +produced runtime will be validated using Ampere Altra Processor based servers. + +The change will also include adding documentations of the support for Arm64 +architecture. + +Alternatives +------------ + +Cross-compile for Arm on x86 platform, which would be more difficult and need +more effort. On the other hand, we have Arm servers for native builds provided +by Arm company, so support for native builds is the preferred method. + +Data model impact +----------------- + +None + +REST API impact +--------------- + +None + +Security impact +--------------- + +Secure boot support will be disabled on Arm and need extra work to investigate +and re-implement for Arm platform. + +Other end user impact +--------------------- + +* For x86_64 end user: None +* For Arm64 end user: there will be docs for Arm64 for these new users. + +Performance Impact +------------------ + +* For x86_64: None +* For Arm64: the performance is not likely to be the same as on x86_64. + +Other deployer impact +--------------------- + +The documentation will be changed to add Arm64 support, and the deployer needs +choose between one of the two supported architectures before deployment. + +Developer impact +---------------- + +The documentation for developers (developer guide) will be updated with detail +items of the impact, e.g. + +* Avoid hardcoded arch names like 'x86-64' or 'amd64' in the source codes. +* Consider if there are any different patches or behavior for x86 and Arm. +* Different package lists for x86 and Arm. +* Different container image for x86 and Arm. + +Upgrade impact +-------------- + +None + +Implementation +============== + +Assignee(s) +----------- + +Primary assignee: + +* Jackie Huang (jackie-huang) + +Other contributors: + +* Tianyu Li (tianyuli) + +Repos Impacted +-------------- + +* starlingx/ansible-playbooks +* starlingx/config +* starlingx/containers +* starlingx/fault +* starlingx/ha +* starlingx/integ +* starlingx/kernel +* starlingx/metal +* starlingx/stx-puppet +* starlingx/tools +* starlingx/utilities +* starlingx/virt +* starlintx/root +* starlingx/jenkins-pipelines +* starlingx/app-harbor +* starlingx/app-istio +* starlingx/app-kubernetes-power-manager +* starlingx/app-kubevirt +* starlingx/app-node-feature-discovery +* starlingx/app-oran-o2 +* starlingx/app-security-profiles-operator +* starlingx/app-sriov-fec-operator +* starlingx/app-sts-silicom +* starlingx/audit-armada-app +* starlingx/cert-manager-armada-app +* starlingx/metrics-server-armada-app +* starlingx/monitor-armada-app +* starlingx/nginx-ingress-controller-armada-app +* starlingx/oidc-auth-armada-app +* starlingx/openstack-armada-app +* starlingx/platform-armada-app +* starlingx/portieris-armada-app +* starlingx/ptp-notification-armada-app +* starlingx/snmp-armada-app +* starlingx/vault-armada-app + +Work Items +---------- + +StarlingX Debian Build Environment: Setup on Native Arm64 Server +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +* Arm company provides the HPE ProLiant RL300 Gen11 servers with Amper + Altra Processor. +* Extend LAT-SDK to support Arm64. +* Build LAT-SDK from wrlinux with BSP qemuarm64 on Arm64 server. +* Configure the build environment on Arm64 server. +* Rebuild all the container images on Arm64 server, for stx-lat-tool, + change the dockerfile to use the LAT-SDK built from qemuarm64. + +StarlingX Debian Build System: Extend to support Arm64 architecture +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Build-tools +""""""""""" + +The packages lists and image yaml files may differ for different arch, the +build-tools should support arch specific lists and files, which will be named +with suffix like _arm64, then the build-tools will check the host arch, and +the files named with arch suffix will be used instead of the non-arch-specific +files. + +So the supported package lists and files will include: + +* base-bullseye.lst +* base-bullseye.yaml +* base-bullseye_arm64.lst +* base-bullseye_arm64.yaml +* base-initramfs-bullseye.yaml +* base-initramfs-bullseye_arm64.yaml +* debian-image.inc +* debian-image_arm64.inc +* debian_iso_image.inc +* debian_iso_image_arm64.inc +* debian_pkg_dirs +* debian_pkg_dirs_arm64 +* debian_pkg_dirs_installer +* debian_pkg_dirs_installer_arm64 +* debian_pkg_dirs_rt +* debian_pkg_dirs_rt_arm64 +* os-std.lst +* os-std_arm64.lst + +Package meta data +""""""""""""""""" + +The items in dl_path/dl_files in meta_data.yaml may have different values and +the patches in the patches or deb_patches directory may be different for arm64 +arch, the extend change will allow multiple dl_path/dl_files items exist in +meta_data.yaml, and multiple series files in patches or deb_patches +directories exist, and the ones with arch suffix will be used instead of the +non-arch-specific ones. + +As an example, the supported meta data will look like: + +e.g. in meta_data.yaml + +.. code-block:: yaml + + dl_files: + pkg-file-amd64.tar.gz: + topdir: pkg-amd64 + url: https://path/to/pkg-file-amd64.tar.gz + sha256sum: + dl_files_arm64: + pkg-file-arm64.tar.gz: + topdir: pkg-arm64 + url: https://path/to/pkg-file-arm64.tar.gz + sha256sum: + + dl_path: + name: pkg-name-amd64.tar.gz + url: https://path/to/pkg-name-amd64.tar.gz + sha256sum: + dl_path_arm64: + name: pkg-name-arm64.tar.gz + url: https://path/to/pkg-name-arm64.tar.gz + sha256sum: + +e.g. for patches/deb_patches + +.. code-block:: shell + + deb_patches + ├── 0001-deb-patch1.patch + ├── 0002-deb-patch2.patch + ├── 0003-arm64-deb-patch1.patch + ├── series + └── series_arm64 + + patches + ├── 0001-patch1.patch + ├── 0002-patch2.patch + ├── 0002-arm64-patch1.patch + ├── series + └── series_arm64 + +Package deb files +""""""""""""""""" + +* control: extend some of the packages to support Arm64 architecture. +* rules: use arch variable to replace hardcoded arch names. + +Container images +^^^^^^^^^^^^^^^^ + +* Some OpenSource container images don't have arm64 on the original registry, + need to use ones on different registry url or rebuild them for Arm64 and + push to different registry. +* In ansible playbook, a new task will be added to check the arch and reset + the images url. +* Existing docker files may not support Arm64 and will need to be ported. +* Docker image and python wheel build scripts in starlingx/root will need + to be extended to support multiple architectures. + +Secure boot support +^^^^^^^^^^^^^^^^^^^ + +* The secure boot support will be disabled in this implementation on Arm64. +* Extra work is needed to investigate and re-implement the support for secure + boot on Arm64 architecture (in another Story/Spec). + +Dependencies +============ + +* Arm company provides the HPE ProLiant RL300 Gen11 servers with Amper + Altra Processor. +* Extend LAT-SDK to support Arm64. +* Build LAT-SDK in wrlinux with BSP qemuarm64 on Arm64 server. + +Testing +======= + +Build +----- + +* Build packages and image on x86 platform to ensure there is no impact after + the changes for Arm64. +* Build packages and image on Arm64 platform. +* Build docker images on Arm64 platform + +System configuration +-------------------- + +The system deployment configurations that we are assuming for testing are: + + +* AIO-SX on x86-64 platform +* AIO-SX on Arm64 platform +* AIO-SX with Low Latency(LL) on Arm64 platform +* AIO-DX on Arm64 platforms +* AIO-DX(LL) on Arm64 platforms +* Standard (2+2) on Arm64 platform +* Standard (2+2) on Arm64 platform +* DC (AIO-DX for Central + 2 sub-cloud with AIO-SX) +* DC (AIO-DX-LL for Central + 2 sub-cloud with AIO-SX-LL) + +Documentation Impact +==================== + +The documentation will need to be changed to add notes about Arm support, +validated Arm servers, difference between x86 and Arm platform, known +issues, limitations and etc. + +References +========== + +* `Starlingx-discuss Arm engagement`_ +* `Starlingx-discuss CI/CD setup on Arm platform`_ + +History +======= + +.. list-table:: Revisions + :header-rows: 1 + + * - Release Name + - Description + * - stx-9.0 + - Introduced + +.. Links +.. _#2010739: https://storyboard.openstack.org/#!/story/2010739 +.. _Starlingx-discuss Arm engagement: https://lists.starlingx.io/pipermail/starlingx-discuss/2023-May/014160.html +.. _Starlingx-discuss CI/CD setup on Arm platform: https://lists.starlingx.io/pipermail/starlingx-discuss/2023-July/014354.html