diff --git a/doc/source/specs/stx-6.0/approved/starlingx_2008704_debian_transition.rst b/doc/source/specs/stx-6.0/approved/starlingx_2008704_debian_transition.rst new file mode 100644 index 0000000..d332c6a --- /dev/null +++ b/doc/source/specs/stx-6.0/approved/starlingx_2008704_debian_transition.rst @@ -0,0 +1,294 @@ +.. + This work is licensed under a Creative Commons Attribution 3.0 Unported + License. http://creativecommons.org/licenses/by/3.0/legalcode + +======================================== +StarlingX Userspace transition to Debian +======================================== + +Storyboard: +https://storyboard.openstack.org/#!/story/list?status=active&project_group_id=86 + +This specification will discuss the current challenge and proposed +solution with the loss of on-going support for CentOS-8 [#]_ and CentOS +Linux in general. Red Hat and the CentOS Project have decided to move +to CentOS Stream, which is a "rolling preview" of what’s next in RHEL +according to this Red Hat blog [#]_. Due to the nature of this type +of release process there is no guarantee of getting the level of support +needed to meet the needs of the StarlingX community. This specification +will layout options that could be used to replace the existing CentOS-7 +base OS with an alternative. + +Problem description +=================== + +Recently, the CentOS Project announced that it would stop supporting +CentOS Linux 8 as of the end of 2021 with the replacement being CentOS +Stream. CentOS Stream is described as a rolling preview for RHEL with a +5 year support window. It's also been pointed out that CentOS Stream will +receive critical fixes later than RHEL or via package updates, thus +removing package versions that the StarlingX Community might rely on. +In doing this, Red Hat has turned CentOS into a "testing" release forcing +organizations looking for a "stable" release to adapt to RHEL. + +Evaluation of Options +===================== + +The StarlingX community is now in a position to make an important decision +regarding the Base OS. StarlingX is currently CentOS & RPM based, there +are alternatives in the GNU/Linux distribution space. The following chart +shows the set of options along data that will help to make the best +decision. + +.. csv-table:: Base OS Comparison + :header: "Distro","Package System","Time to EOL","Commercial","Kernel Version","Low Latency Kernel","HW Support","Effort (t-shirt)","Package Translation" + :widths: 15,15,15,15,15,15,15,15,15 + + "Fedora","RPM","13 month, no LTS",no,"5.8","buildable","buildable","Med","no" + "CentOS-Stream","RPM","Rolling/5yrs",no,"4.18","yes","yes","Med","no" + "RHEL","RPM","10 years","yes","4.18","yes","yes","Med","no" + "Rocky","RPM","Unknown",no,"4.18","yes","yes","Med","no" + "CloudLinux","RPM","10 yrs","yes","4.18","yes","yes","Med",no + "AlmaLinux","RPM","8 yrs",no,"4.18","yes","yes","Med","no" + "Oracle Linux","RPM","10yrs","yes","4.18","yes","yes","Med",no + "Amazon Linux","RPM","5 yrs","yes","4.14/4.19","buildable","buildable","??","no" + "SUSE","RPM","10+3","yes","4.12","yes","yes","Med","limited" + "openSUSE","RPM","10+3",no,"4.12","yes","yes","Med","limited" + "Debian","DEB","5 yrs + $$ELTS 2yrs",no,"5.10","yes","buildable","XL","yes" + "Ubuntu","DEB","5+5","yes","5.4/5.8","yes","yes","XL","yes" + "OpenEmbedded / Yocto Project","any","2 yrs",no,"5.10","yes","buildable","XL","yes" + "Alpine Linux","APK","rolling/ 1.5 yrs",no,"5.10","buildable","buildable","??","yes" + +The Hardware Support column shows either yes or buildable, means that +either the distribution contain a kernel with some of the latest +hardware support, wheras buildable means we would have to backport +patches as needed. Package Translation column shows if a given row +requires package meta-data to be translated or converted from RPM to +a new package system. + +Based on the available data of the existing distros a number of these +can be ruled out. Any distros that are only available from commercial +vendors such as Red Hat, Canonical, Oracle, and SUSE are not viable +choices. Due to the potential for additional disruptions because a vendor +makes some strategic change (Red Hat/CentOS, Docker, ...), future cost of +support, possible license issues, or branding. While Oracle Linux is +available with free redistribution rights, there is no guarentee that +this will continue in the future and there is no clear upstream strategy, +which could impact quality of the release. While there are some new +non-commercial distributions available such as RockyLinux and AlmaLinux +(CloudLinux's free version) that are attempting to provide a similar +solution as CentOS Linux did, they are young communities with no track +record on how they will be able to support their community members. + +While Base OS should be a non-differentiation item, the kernel might be, +therefore looking at newer kernel options that supports newer hardware +should be important, along with a kernel that is updated regularly. +What remains is a limited number of non-commercial candidates. Of those, +it is important a distribution be selected that has a strong community +that can provide long term support for critical fixes and a security +process to address CVEs. This would also suggest using the latest kernel +available from Debian or OE/Yocto. + +The remaining distros such as OE/Yocto and Alpine are not considered by +operators/deployers to be enterprise / datacenter ready. OpenSuSE is +primarily supported by SUSE and has similar problem of being tied to a +Commercial Vendor, in addition to using an older kernel. This leaves +Debian, while this would be a major change to the package and build +system, the Debian community is well structured and supports a wide +variety of packages and hardware. Debian has a robust community with a +large userspace beyond the standard open source userspace applications, +such as Kubernetes, helm, ceph and OpenStack packages. + +Use Cases +--------- + +This change will most impact StarlingX Community Developers with limited +impact to Operators and End Users. + +System Admin: Has to deal with distro related tools and stuff in /etc and +difference with sysconfig tools, also ansible and puppet. + +Operators deal with WebUI, but may deal with CLI and have some interaction +on the shell. + +End Users should not see any difference as they work with the OpenStack +and Kubernetes interfaces + +Proposed change +=============== + +Shift Base Operating System from CentOS to Debian Stable Release. + +This will entail starting a new project to create a build system based +around Debian packaging, this would be in parallel to the existing +CentOS as the current base will need to be supported for some time to +come. Since many of the existing Debian build tools operate on a merge +of upstream source with a Debian meta-data sub-directory a new set of +repos would be needed to support this. Development can still occur on +the current set of repos, tools can be used to merge between the two +sets of repos. A new specification will be written to address the new +build tools once the proof of concept is completed. + +Which exact Debian build tools [#]_ to use is still under evaluation. + +This work will require a complete switch away from CentOS once the +initial release of the Debian base occurs, unless the community has +the additional resources to support maintaining and testing both a CentOS +and Debian base. + +Debian stable releases happen every few years, Debian11 (Bullseye) is +currently the in-flight release under test. The Debian team has started +the version lockdown will become the stable release before the end of the +year. This will line up nicely with StarlingX's OS transition. + +Alternatives +------------ + +Given the amount of effort to pivot to a new Base OS vs staying the +course with CentOS, its important to make the decision sooner rather +than later if any of the above alternatives are not viable. + +1) Do nothing, stay with CentOS-7, which is supported until July 2024 and +see how the new RHEL derivatives such as RockyLinux or AlmaLinux play +out. This would require continuing the CentOS-8 work in order to be +prepared for such a transition. By doing nothing or the minimum to move +to CentOS-8, the risk increases if the new communities do not work out. + +2) Use CentOS Stream, same work to move to RockyLinux, but will also +require continued updating of userspace as CentOS Stream moves forward +and increased testing. The on-going effort to support a rolling style +release would require constant up-rev of packages and/or backporting +CVEs and other critical fixes. Additionally, there may be a delay getting +a CVE or fix into the StarlingX packages. + +3) Another option would be to put more resources into the multi-os +project and complete the OpenEmbedded implementation. As mentioned +above, operators and deployers of StarlingX don't view OE as a viable +base userspace. Additionally, this is still a proof of concept. + +Data model impact +----------------- + +Changing the Base OS will impact a number of different aspects of the +system installation and deployment since Debian does not use anaconda +and has a different sysconfig interface. The ansible, puppet and +config-file scripts and packages will need to be modified to account +for the difference in Debian. + +This will not impact any data models between the flock and associated +interfaces with Kubernetes or OpenStack components. + +REST API impact +--------------- + +The change to the Base OS should not impact the various REST APIs +that are part of the flock. + +Security impact +--------------- + +This change will update a large part of the userspace to newer versions +of the package base, along with the kernel. Moving to newer version +will address some CVEs and improve security, there will be an impact to +the scanning tool we use to monitor and report security issues. + +Other end user impact +--------------------- + +The base os package names will be different for the operator will need +to understand these difference when doing on-system administration. + +This will not directly affect any of the python clients. + +Developer impact +---------------- + +StarlingX Community Developers/Designers may need to learn and understand +a new build and packaging system. Developers may also need to modify code +areas that interfaces with sysadmin functions that are different between +CentOS and Debian + + +Upgrade impact +-------------- + +The upgrade and patching mechanisms will need to be changed, this will +be looked at in more details with additional specifications. The upgrade +to Debian will be done as part of a normal StarlingX platform upgrade as +described in [#]_. Since a platform upgrade is done one host at a time, +the key consideration is that the upgraded hosts (running a Debian based +release) must be able to communicate with un-upgraded hosts (running a +CentOS based release). This will be done by ensuring key services (e.g. +DRBD, Ceph, Kubernetes) are running compatible software versions on the +Debian and CentOS releases. + + +Implementation +============== + +Assignee(s) +----------- + +Primary assignee: + TDB + +Other contributors: + TDB + +Repos Impacted +-------------- + +All repos will be impacted by this change and it will require packaging +and tooling changes. + +Work Items +---------- + +Each of these will likely need additional specifications: + +* Build Tools and Mirror creation +* Creating an installable image +* Updating installation process + +Testing +======= + +Lots of testing will be required, this will require modification to +the test infrastructure in order to handle the Debian environment. + + +Documentation Impact +==================== + +Any documentation that references CentOS specific command for packaging +will need to be update to Debian commands. Developer documentation will +need to have the workflows updated to reflect Debian workflows vs our +current workflows. + + +References +========== + +* https://etherpad.opendev.org/p/stx-distro-other + +* https://etherpad.opendev.org/p/stx-multios + +.. rubric:: Footnotes + +.. [#] https://blog.centos.org/2020/12/future-is-centos-stream +.. [#] https://www.redhat.com/en/blog/centos-stream-building-innovative-future-enterprise-linux +.. [#] https://wiki.debian.org/SystemBuildTools +.. [#] https://docs.starlingx.io/specs/specs/stx-4.0/approved/starlingx-2007403-platform-upgrades.html + +History +======= + +.. list-table:: Revisions + :header-rows: 1 + + * - Release Name + - Description + * - STX-6.0 + - Introduced + +