Add spec for publishing build artifacts on server
This change adds the spec file which described how official build artifacts will be published on the world-facing server. Change-Id: I0704b3ecbd1f34138edc7c3a5efe5cd888821fe5 Signed-off-by: Jason McKenna <jason.mckenna@windriver.com> Authored-By: Jason McKenna <jason.mckenna@windriver.com> Co-Authored-By: Scott Little <scott.little@windriver.com> Story: 2003906 Task: 27974
This commit is contained in:
parent
59bd941ad3
commit
1bd070a63d
|
@ -0,0 +1,307 @@
|
|||
..
|
||||
This work is licensed under a Creative Commons Attribution 3.0 Unported
|
||||
License. http://creativecommons.org/licenses/by/3.0/legalcode
|
||||
|
||||
..
|
||||
|
||||
===================================================
|
||||
StarlingX: Host Build Artifacts on StarlingX Mirror
|
||||
===================================================
|
||||
|
||||
https://storyboard.openstack.org/#!/story/2003906
|
||||
|
||||
An external mirror hosting build artifacts allows for quicker ramp-up
|
||||
of community members, and provides for reference artifacts.
|
||||
|
||||
This story will implement the building of artifacts on a server and
|
||||
the population of those artifacts to the existing community-accessible
|
||||
mirror.
|
||||
|
||||
|
||||
Problem description
|
||||
===================
|
||||
|
||||
A new community member who wishes to evaluate StarlingX currently needs
|
||||
to download the source code, set up a build environment, and build all
|
||||
the StarlingX components. They will then generate their own ISO. This
|
||||
process can take a significant amount of time, and lead to a poor initial
|
||||
impression of StarlingX.
|
||||
|
||||
The initial implementation of hosted builds would have a server:
|
||||
* Build all RPMs used for the build
|
||||
* Build an installer image based on the produced RPMs
|
||||
* Build a reference ISO based on the produced RPMs
|
||||
* Publish RPMs, installer, and reference ISO to the existing mirror
|
||||
|
||||
The initial implementation would provide artifacts for the r/2018.10
|
||||
and the master branch of StarlingX. Future work would build and host
|
||||
artifacts for evolving branches at an interval to be determined.
|
||||
|
||||
|
||||
Use Cases
|
||||
=========
|
||||
|
||||
Evaluators of the StarlingX project want to use a prebuilt ISO image to
|
||||
install and test StarlingX.
|
||||
|
||||
Developers want to compare their local changes to a package with a
|
||||
reference image containing an unmodified version of the package
|
||||
|
||||
Producers of an ISO want to use the prebuilt installer image rather than
|
||||
one they produce themselves.
|
||||
|
||||
|
||||
Proposed change
|
||||
===============
|
||||
|
||||
Provision a server such that it is able to build the StarlingX
|
||||
project. Allow for this build to produce artifacts based on either
|
||||
the master branch, release branches (e.g. r/2018.10), or feature branches
|
||||
(e.g. f/somefeature). Building and hosting of other builds (branches or
|
||||
tags) should be accomodated. Provide a mechanism for the build to publish its
|
||||
artifacts (RPMs, ISO, installer) to the existing mirror. Initial
|
||||
implementation will use a build server at CENGN.
|
||||
|
||||
Periodic building of the master branch, and publishing of the resulting
|
||||
artifacts via the StarlingX mirror, will be performed. Other branches
|
||||
(e.g. r/2018.10) will also be built and published, but only as required.
|
||||
|
||||
The publication path for each build shall be unique and will include branch
|
||||
and UTC time stamp. An alias for the latest build of the branch shall be
|
||||
provided.
|
||||
|
||||
Initial rate of builds is 1 build of the master branch per day. This
|
||||
cadence may be adjusted in the future. Non-master branch builds shall
|
||||
be performed on an on-demand basis as determined by the build team.
|
||||
Non-master branch builds include release builds, and milestone builds.
|
||||
Milestone builds are to be built periodically, at a cadence to be determined.
|
||||
|
||||
The retention period of master branch build shall initially be 14 days.
|
||||
The retention period may be revisited and adjusted at a later time.
|
||||
The retention time for release branch builds will be much greater then
|
||||
master branch builds, possibly indefinitely. Milestone builds shall
|
||||
be retained for at least one release period past their introduction.
|
||||
|
||||
Hosted artifacts proposed directory structure:
|
||||
|
||||
Build artifacts hosted by the mirror shall be available in following
|
||||
directory structure:
|
||||
|
||||
http://mirror.starlingx.cengn.ca/mirror/starlingx/<path>
|
||||
|
||||
<path> = <release-identifier>/<distro>/<build-date>/outputs/<output-artifact-type>
|
||||
<release-identifier>/<distro>/<build-date>/inputs/<input-artifact-type>
|
||||
|
||||
<release-identifier> = 'master'
|
||||
'feature/'<feature-branch-name>
|
||||
'release/'<release-branch-name>
|
||||
'milestone/'<milestone-tag>
|
||||
...
|
||||
'r2018.10' (legacy symlink to release/2018.10)
|
||||
|
||||
<distro> = 'centos'
|
||||
|
||||
<feature-branch-name> = 'centos76'
|
||||
...
|
||||
|
||||
<release-branch-name> = '2018.10'
|
||||
...
|
||||
|
||||
<build-date> = $(date +%Y%m%dT%H%M%SZ) # time stamp of start of 'repo sync'
|
||||
<latest_build> # a symlink to the last successful build date
|
||||
|
||||
<input-artifact-type> = RPMS/
|
||||
SRPMS/
|
||||
downloads/
|
||||
|
||||
<output-artifact-type> = RPMS/<build-phase>/
|
||||
SRPMS/<build-phase>/
|
||||
iso/
|
||||
installer/
|
||||
wheels/
|
||||
helm-charts/
|
||||
CONTEXT
|
||||
<build-result>
|
||||
logs/
|
||||
|
||||
<build-phase> = std
|
||||
rt
|
||||
installer
|
||||
container
|
||||
...
|
||||
|
||||
<build-result> = 'success' | 'fail'
|
||||
|
||||
Non-hosted artifacts:
|
||||
|
||||
Other artifacts produced by the build may be hosted outside of the
|
||||
mirror. In particular, container images shall be hosted on DockerHub.
|
||||
|
||||
The DockerHub container image path shall be
|
||||
|
||||
<image-name>=<org>/<image>:<tag>
|
||||
|
||||
<org>=starlingx
|
||||
|
||||
<image>=stx-<component>
|
||||
|
||||
<tag>=<git-tag-or-branch>-<os>-<openstack-release>[-<qualifier>]
|
||||
|
||||
<os>=centos | ubuntu | clear-linux
|
||||
|
||||
<openstack-release>=pike | queens | rocky ...
|
||||
|
||||
<component>=aodh | ceilometer| cinder | glance | gnocchi | heat | horizon |
|
||||
ironic | keystone | libvirt | magnum | murano | neutron |
|
||||
nova-api-proxy | nova | panko ...
|
||||
|
||||
<qualifier>=<timestamp> | latest | stable
|
||||
|
||||
<git-tag-or-branch>=dev | r-2018.10.0 | m-2019.05b1 | ...
|
||||
(This naming convention is for reference only, and has not been finalized)
|
||||
|
||||
|
||||
|
||||
Alternatives
|
||||
============
|
||||
|
||||
The existing mechanism of requiring all persons to perform their own
|
||||
builds is an alternative.
|
||||
|
||||
Providing a prebuilt version of StarlingX via a container (rather than
|
||||
as build artifacts) has not been considered, as it would still depend
|
||||
on an official build of StarlingX artifacts to install in said
|
||||
container.
|
||||
|
||||
|
||||
Data model impact
|
||||
=================
|
||||
|
||||
None
|
||||
|
||||
|
||||
REST API impact
|
||||
===============
|
||||
|
||||
None
|
||||
|
||||
|
||||
Security impact
|
||||
===============
|
||||
|
||||
Server administration:
|
||||
The build server is to be administered by a small group of core
|
||||
developers from the existing StarlingX Build team. The server
|
||||
should be kept up to date with security related OS updates. The
|
||||
address and access mechanism of the server are not to be published
|
||||
to those outside the StarlingX Build team or StarlingX Security
|
||||
team.
|
||||
|
||||
Build initiation:
|
||||
There shall be a mechanism for triggering builds. This mechanism
|
||||
shall support self-initiated builds (periodic, or based on
|
||||
observed external git commits, etc). The mechanism shall also
|
||||
support manually-initiated builds. The mechanism for manually
|
||||
initiated builds shall initially be through the administration
|
||||
interface (discussed above). Providing other mechanisms to
|
||||
initialize builds (for example, providing a web interface through
|
||||
Jenkins, or a similar mechanism) is beyond the scope of this
|
||||
document, however security concerns must be taken into account
|
||||
if providing an additional interface.
|
||||
|
||||
|
||||
Other end user impact
|
||||
=====================
|
||||
|
||||
End user will only see a change in StarlingX deployment if they
|
||||
choose to make use of the pre-published artifacts. The automated
|
||||
use of pre-published artifacts (example: having build scripts
|
||||
use the pre-built installer by default) are potential future
|
||||
improvements outside the scope of this document.
|
||||
|
||||
|
||||
Performance Impact
|
||||
==================
|
||||
|
||||
No impact to running systems. Time to obtain and evaluation
|
||||
copy of StarlingX will be drastically reduced.
|
||||
|
||||
|
||||
Other Deployer Impact
|
||||
=====================
|
||||
|
||||
None
|
||||
|
||||
|
||||
Developer impact
|
||||
=================
|
||||
|
||||
Reference RPMs, Installer image, and ISO can be used to compare code
|
||||
in development.
|
||||
|
||||
|
||||
Upgrade impact
|
||||
===============
|
||||
|
||||
None
|
||||
|
||||
|
||||
Implementation
|
||||
==============
|
||||
|
||||
|
||||
Assignee(s)
|
||||
===========
|
||||
|
||||
Primary assignee:
|
||||
Scott Little <slittle1>
|
||||
|
||||
Other contributors:
|
||||
Jason McKenna <jmckenna>
|
||||
Don Penney <dpenney>
|
||||
Al Bailey <albailey>
|
||||
|
||||
|
||||
Repos Impacted
|
||||
==============
|
||||
|
||||
stx-tools - scripts to assist in publication of build outputs
|
||||
|
||||
|
||||
Work Items
|
||||
===========
|
||||
|
||||
* Setup base metal (CENGN) Server to perform builds
|
||||
* Create mechanism for server to perform periodic build of master branch
|
||||
* Create mechanism for server to perform build of designated release branch
|
||||
* Create script or mechanism to publish builds to existing mirror
|
||||
|
||||
|
||||
Dependencies
|
||||
============
|
||||
|
||||
None
|
||||
|
||||
|
||||
Testing
|
||||
=======
|
||||
|
||||
Basic installation and load sanity testing of produced iso.
|
||||
|
||||
|
||||
Documentation Impact
|
||||
====================
|
||||
|
||||
Documentation for the artifacts will be posted on the StarlingX wiki
|
||||
|
||||
|
||||
References
|
||||
==========
|
||||
|
||||
https://www.cengn.ca/
|
||||
http://mirror.starlingx.cengn.ca/mirror/
|
||||
|
||||
|
||||
History
|
||||
=======
|
||||
|
Loading…
Reference in New Issue