From d56843eb679d445dcecc90d1356e12257dd3763d Mon Sep 17 00:00:00 2001 From: Mark Asselstine Date: Thu, 29 Apr 2021 16:08:18 -0400 Subject: [PATCH] stx-6.0: Initial spec for new stx tool Initial spec submission for a new stx tool to be implemented as part of the transition to Debian OS. Story: 2008862 Signed-off-by: Mark Asselstine Change-Id: Ib9d0c02d8f1b3e3d64397dd141430da129e3289d --- .../approved/starlingx_2008862_stx_tool.rst | 316 ++++++++++++++++++ 1 file changed, 316 insertions(+) create mode 100644 doc/source/specs/stx-6.0/approved/starlingx_2008862_stx_tool.rst diff --git a/doc/source/specs/stx-6.0/approved/starlingx_2008862_stx_tool.rst b/doc/source/specs/stx-6.0/approved/starlingx_2008862_stx_tool.rst new file mode 100644 index 0000000..5d03e78 --- /dev/null +++ b/doc/source/specs/stx-6.0/approved/starlingx_2008862_stx_tool.rst @@ -0,0 +1,316 @@ +.. + This work is licensed under a Creative Commons Attribution 3.0 Unported + License. http://creativecommons.org/licenses/by/3.0/legalcode + +=================== +StarlingX: STX Tool +=================== + + +STX Tool - Script consolidation for build management, build and more +==================================================================== + +Storyboard Story: +https://storyboard.openstack.org/#!/story/2008862 + +Create a new 'stx' tool which would consolidate many of the tools +currently available in the starlingx/tools repository. These tools are +used to manage the STX Build container, manipulate the package +repository mirror, create platform builds, etc.. + +By consolidating many tools into a single tool with many functions we +can offer many benefits + +* Rolled up help text +* Improved tab completion +* Consistency from sub-command to sub-command +* Facilitate code re-use +* Provide a familiar interface as found in git or kubectl for example + +When combined with the Debian Build spec the 'stx' tool will help +ensure that the more complex build architecture is abstracted from the +developer. + +Problem description +=================== + +Navigating the StarlingX tools repository today requires former +knowledge of the many available tools or heavy reliance on +documentation resources. This is often not a friendly environment for +new users who are just starting to build StarlingX for the first time. + +The tool authors have done a good job keeping coding standards, +variable use, naming conventions, etc. consistent, but as new +developers contribute to the project this consistency becomes harder +to enforce. It is also hard to share functionality between scripts to +encourage code reuse. + +If this spec is not accepted, in addition to major updates to support +the transition to Debian, most of the tools would have to be made +conditional on if CentOS or Debian operations would be desired. If +this spec is accepted most of the current tools can be left in place +and as-is throughout the Debian transition, allowing continued use to +build CentOS builds alongside the Debian transition work. + +Use Cases +--------- + +This change would affect most of the use cases described in the +StarlingX Build Guide[1]. Such as, but not limited to: + +* Build packages +* Build ISOs +* Build installer +* Build StarlingX OpenStack Application + +This would impact the StarlingX Developer. All key operations +performed by developers will have equivalents available in the 'stx' +tool. As part of this spec all StarlingX resources will be updated to +reflect the changes, but will not extend to developer's own +resources. Any wrapper scripts written by developers, notes, +documents, etc. written for their personal use will have to be +updated by them. + +After this spec is implemented the developer will have access to a +command structure matching that of common tools such as git and +kubectl. This type of command structure brings with it many benefits +that would hopefully make it easier for new developers to start +building and contributing to the StarlingX project. + +[1][https://docs.starlingx.io/developer_resources/build_guide.html] + +Proposed change +=============== + +A new 'stx' tool is available in the starlingx/tools repository. This +tool will be written in a language such as Python(3) to facilitate the +development of modules which can be reused by the various +sub-commands. + +The 'stx' tool will be developed as part of the transition to Debian, +allowing the existing tools to remain untouched and available for +CentOS based builds throughout the OS transition (and possibly into +the future). + +Running the 'stx' tool with no arguments will display help text +describing the available sub-commands. This behavior matches what you +see with git or kubectl. + +Running the 'stx' tool with a single argument, a sub-command name, +will display help text for that specific sub-command. This will +include things such as command line options, details about inputs or +outputs, etc.. It may be possible for sub-commands to have +sub-commands of their own. + +The 'stx' tool will have a sub-command to manage its own +configuration. Users can choose to edit the configuration file on +their own, but using 'stx config' will ensure valid values are used +and provide other useful feedback. Again, this should be similar to +the experience found with 'git config', though it should not be +required to be as complex. + +A review of existing tools in the starlingx/tools repository will be +done to ensure equivalent operations are available in the new 'stx' +tool, with adjustments made to reflect the Debian build +requirements. Determinations can be made to not include an existing +tool in the 'stx' tool for technical or practical reasons, these +should be the minority. + +The 'stx' tool will not wrap other tools such as git or repo unless it +makes sense to do so. For instance if a call to repo is made as a step +in the middle of a build we do not want to force a user to run a 'stx' +tool command, run a repo command and then run a second 'stx' command +to complete a build, in this case the 'stx' tool would make the call +to repo. Discretion will be made in this area to ensure the right tool +is used for the job. + +The 'stx' tool will have sub-commands for work such as, but not limited to: + +* controlling build container(s) state (stx control) +* building (stx build) +* configuration get/set (stx config) +* repository management (stx repomgmt) +* status information (stx status) +* version information (stx versions) + +(name for demonstration purposes only, subject to change) + +NOTE: "repository management" or "repomgmt" in the above refers to the +handling of a Debian binary package and source package repository, as +described by Debian Repository Format [2]. This context will be +provided by help text and documentation to provide disambiguation from +the 'repo tool' or 'git repository'. + +[2][https://wiki.debian.org/DebianRepository/Format] + +Alternatives +------------ + +We could reuse the existing scripts, make them conditional on CentOS +vs Debian and implement the Debian analogues. On the one hand this +would be one less change for developers to absorb as we transition to +Debian, but on the other it does not address the issues described +here. + +Additionally to reusing the scripts with Debian modifications we could +review the organization of the tools repository, such as consolidating +scripts in a 'bin' directory, having a docs directory which map to +individual commands... This would address some of the issues described +but would leave others open, such as code reuse. + +Data model impact +----------------- + +None + +REST API impact +--------------- + +None + +Security impact +--------------- + +None + +Other end user impact +--------------------- + +None + +Performance Impact +------------------ + +None + +Other deployer impact +--------------------- + +None + +Developer impact +---------------- + +Developer impact will not be insignificant. Many/most of the commands +they are using today will be dropped in favor of using commands in the +new 'stx' tool. Any personal resources they have developed to assist +in their workflows which make use of tools which will be impacted by +this change and will be required to be updated by them. Unfortunately +this impact is hard to measure as these resources fall outside of the +project scope. + +New and existing developers should find the new 'stx' tool easy to use +and familiar if they are used to this command structure. Even if they +are not familiar with this command structure, the rolled up help text, +easy tab completion and other benefits should translate to them +quickly being proficient with the tool. + +Developers who work on the tools themselves will hopefully benefit +from code reuse making changes and expanded functionality easier to +implement. + +Upgrade impact +-------------- + +None + + +Implementation +============== + +Assignee(s) +----------- + +Primary assignee: + Mark Asselstine - markawr + +Other contributors: + +* Yue Tao - wrytao +* Hongxu Jia - hjia +* Xiao Zhang - xzhang1 +* Peng Yan - yanpenghit +* Haiqing Bai - hqbai + +Repos Impacted +-------------- + +starlingx/tools + +Work Items +---------- + +See storyboard. Additional items will be added after spec approval. + +Dependencies +============ + +This spec is related to the Debian Transition spec and the Debian +Build spec. This spec does not depend on the Debian Build spec and can +be accepted and implemented regardless of the outcome of that spec. + +Essentially the existing tools must be re-worked to support the +already accepted Debian Transition spec. If this re-work were to be +done with the introduction of a 'stx' tool, as described in this spec, +it would simplify the tools work needed to support the updated Debian +Build. + +Testing +======= + +As the scope of this spec is restricted to the building of StarlingX +it does not introduce any additional runtime testing requirements. As +this change is proposed to take place alongside the move to Debian, +full runtime testing is expected related to that spec. + +Every command, sub-command, option, input and output of the new tool +should have test code submitted alongside code implementing +functionality. Since this work is taking place alongside the Debian +Transition work it will not be possible to test existing and new +outcomes to ensure a match. Because of this existing tool's outcomes +will be reviewed and the equivalent 'stx' tool operation outcomes will +be checked for parity. + +In cases where new commands and sub-commands are created, where no +existing tool equivalent exists, requirements will be developed and +outcomes will be tested against these requirements. + +Documentation Impact +==================== + +Revisit much of the StarlingX Build Guide +https://docs.starlingx.io/developer_resources/build_guide.html + +Revisit much of the StarlingX Layered Build Guide +https://docs.starlingx.io/developer_resources/layered_build_guide.html + +Possible updates to the Build StarlingX Docker Images document +https://docs.starlingx.io/developer_resources/build_docker_image.html + +NOTE: updates to these documents are already required to be updated +for the Debian transition, regardless of the changes suggested in this +spec, though the scope with these changes will be larger. + +References +========== + +* https://etherpad.opendev.org/p/stx-ptg-planning-april-2021 + +* https://docs.starlingx.io/developer_resources/build_guide.html + +* https://www.debian.org/doc/manuals/maint-guide/ + +* https://docs.starlingx.io/specs/specs/stx-6.0/approved/starlingx_2008704_debian_transition.html + +* https://review.opendev.org/c/starlingx/specs/+/788498 + +History +======= + +.. list-table:: Revisions + :header-rows: 1 + + * - Release Name + - Description + * - STX-6.0 + - Introduced