summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorJason McKenna <jason.mckenna@windriver.com>2018-11-16 14:26:20 -0500
committerJason McKenna <jason.mckenna@windriver.com>2019-02-05 15:49:12 -0500
commit1bd070a63d6963d19d05d0d755db582b651a4cbc (patch)
treeefaceb2a81de007a5d814066f5f758aefaf1dd73
parent59bd941ad362421dd8a63413556c9fd923bcd58e (diff)
Add spec for publishing build artifacts on server
This change adds the spec file which described how official build artifacts will be published on the world-facing server. Change-Id: I0704b3ecbd1f34138edc7c3a5efe5cd888821fe5 Signed-off-by: Jason McKenna <jason.mckenna@windriver.com> Authored-By: Jason McKenna <jason.mckenna@windriver.com> Co-Authored-By: Scott Little <scott.little@windriver.com> Story: 2003906 Task: 27974
Notes
Notes (review): Code-Review+2: Brent Rowsell <brent.rowsell@windriver.com> Code-Review+1: Ken Young <ken.young@windriver.com> Code-Review+1: Al Bailey <al.bailey@windriver.com> Code-Review+2: Ian Jolliffe <ian.jolliffe@windriver.com> Code-Review+2: Curtis Collicutt <curtis@serverascode.com> Code-Review+2: Saul Wold <sgw@linux.intel.com> Code-Review+2: Dean Troyer <dtroyer@gmail.com> Workflow+1: Ian Jolliffe <ian.jolliffe@windriver.com> Verified+2: Zuul Submitted-by: Zuul Submitted-at: Tue, 05 Mar 2019 14:33:19 +0000 Reviewed-on: https://review.openstack.org/618587 Project: openstack/stx-specs Branch: refs/heads/master
-rw-r--r--specs/2019.03/approved/mirror_2003906_published_builds.rst307
1 files changed, 307 insertions, 0 deletions
diff --git a/specs/2019.03/approved/mirror_2003906_published_builds.rst b/specs/2019.03/approved/mirror_2003906_published_builds.rst
new file mode 100644
index 0000000..94d7908
--- /dev/null
+++ b/specs/2019.03/approved/mirror_2003906_published_builds.rst
@@ -0,0 +1,307 @@
1..
2 This work is licensed under a Creative Commons Attribution 3.0 Unported
3 License. http://creativecommons.org/licenses/by/3.0/legalcode
4
5..
6
7===================================================
8StarlingX: Host Build Artifacts on StarlingX Mirror
9===================================================
10
11https://storyboard.openstack.org/#!/story/2003906
12
13An external mirror hosting build artifacts allows for quicker ramp-up
14of community members, and provides for reference artifacts.
15
16This story will implement the building of artifacts on a server and
17the population of those artifacts to the existing community-accessible
18mirror.
19
20
21Problem description
22===================
23
24A new community member who wishes to evaluate StarlingX currently needs
25to download the source code, set up a build environment, and build all
26the StarlingX components. They will then generate their own ISO. This
27process can take a significant amount of time, and lead to a poor initial
28impression of StarlingX.
29
30The initial implementation of hosted builds would have a server:
31* Build all RPMs used for the build
32* Build an installer image based on the produced RPMs
33* Build a reference ISO based on the produced RPMs
34* Publish RPMs, installer, and reference ISO to the existing mirror
35
36The initial implementation would provide artifacts for the r/2018.10
37and the master branch of StarlingX. Future work would build and host
38artifacts for evolving branches at an interval to be determined.
39
40
41Use Cases
42=========
43
44Evaluators of the StarlingX project want to use a prebuilt ISO image to
45install and test StarlingX.
46
47Developers want to compare their local changes to a package with a
48reference image containing an unmodified version of the package
49
50Producers of an ISO want to use the prebuilt installer image rather than
51one they produce themselves.
52
53
54Proposed change
55===============
56
57Provision a server such that it is able to build the StarlingX
58project. Allow for this build to produce artifacts based on either
59the master branch, release branches (e.g. r/2018.10), or feature branches
60(e.g. f/somefeature). Building and hosting of other builds (branches or
61tags) should be accomodated. Provide a mechanism for the build to publish its
62artifacts (RPMs, ISO, installer) to the existing mirror. Initial
63implementation will use a build server at CENGN.
64
65Periodic building of the master branch, and publishing of the resulting
66artifacts via the StarlingX mirror, will be performed. Other branches
67(e.g. r/2018.10) will also be built and published, but only as required.
68
69The publication path for each build shall be unique and will include branch
70and UTC time stamp. An alias for the latest build of the branch shall be
71provided.
72
73Initial rate of builds is 1 build of the master branch per day. This
74cadence may be adjusted in the future. Non-master branch builds shall
75be performed on an on-demand basis as determined by the build team.
76Non-master branch builds include release builds, and milestone builds.
77Milestone builds are to be built periodically, at a cadence to be determined.
78
79The retention period of master branch build shall initially be 14 days.
80The retention period may be revisited and adjusted at a later time.
81The retention time for release branch builds will be much greater then
82master branch builds, possibly indefinitely. Milestone builds shall
83be retained for at least one release period past their introduction.
84
85Hosted artifacts proposed directory structure:
86
87Build artifacts hosted by the mirror shall be available in following
88directory structure:
89
90http://mirror.starlingx.cengn.ca/mirror/starlingx/<path>
91
92<path> = <release-identifier>/<distro>/<build-date>/outputs/<output-artifact-type>
93 <release-identifier>/<distro>/<build-date>/inputs/<input-artifact-type>
94
95<release-identifier> = 'master'
96 'feature/'<feature-branch-name>
97 'release/'<release-branch-name>
98 'milestone/'<milestone-tag>
99 ...
100 'r2018.10' (legacy symlink to release/2018.10)
101
102<distro> = 'centos'
103
104<feature-branch-name> = 'centos76'
105 ...
106
107<release-branch-name> = '2018.10'
108 ...
109
110<build-date> = $(date +%Y%m%dT%H%M%SZ) # time stamp of start of 'repo sync'
111 <latest_build> # a symlink to the last successful build date
112
113<input-artifact-type> = RPMS/
114 SRPMS/
115 downloads/
116
117<output-artifact-type> = RPMS/<build-phase>/
118 SRPMS/<build-phase>/
119 iso/
120 installer/
121 wheels/
122 helm-charts/
123 CONTEXT
124 <build-result>
125 logs/
126
127<build-phase> = std
128 rt
129 installer
130 container
131 ...
132
133<build-result> = 'success' | 'fail'
134
135Non-hosted artifacts:
136
137Other artifacts produced by the build may be hosted outside of the
138mirror. In particular, container images shall be hosted on DockerHub.
139
140The DockerHub container image path shall be
141
142<image-name>=<org>/<image>:<tag>
143
144<org>=starlingx
145
146<image>=stx-<component>
147
148<tag>=<git-tag-or-branch>-<os>-<openstack-release>[-<qualifier>]
149
150<os>=centos | ubuntu | clear-linux
151
152<openstack-release>=pike | queens | rocky ...
153
154<component>=aodh | ceilometer| cinder | glance | gnocchi | heat | horizon |
155 ironic | keystone | libvirt | magnum | murano | neutron |
156 nova-api-proxy | nova | panko ...
157
158<qualifier>=<timestamp> | latest | stable
159
160<git-tag-or-branch>=dev | r-2018.10.0 | m-2019.05b1 | ...
161 (This naming convention is for reference only, and has not been finalized)
162
163
164
165Alternatives
166============
167
168The existing mechanism of requiring all persons to perform their own
169builds is an alternative.
170
171Providing a prebuilt version of StarlingX via a container (rather than
172as build artifacts) has not been considered, as it would still depend
173on an official build of StarlingX artifacts to install in said
174container.
175
176
177Data model impact
178=================
179
180None
181
182
183REST API impact
184===============
185
186None
187
188
189Security impact
190===============
191
192Server administration:
193The build server is to be administered by a small group of core
194developers from the existing StarlingX Build team. The server
195should be kept up to date with security related OS updates. The
196address and access mechanism of the server are not to be published
197to those outside the StarlingX Build team or StarlingX Security
198team.
199
200Build initiation:
201There shall be a mechanism for triggering builds. This mechanism
202shall support self-initiated builds (periodic, or based on
203observed external git commits, etc). The mechanism shall also
204support manually-initiated builds. The mechanism for manually
205initiated builds shall initially be through the administration
206interface (discussed above). Providing other mechanisms to
207initialize builds (for example, providing a web interface through
208Jenkins, or a similar mechanism) is beyond the scope of this
209document, however security concerns must be taken into account
210if providing an additional interface.
211
212
213Other end user impact
214=====================
215
216End user will only see a change in StarlingX deployment if they
217choose to make use of the pre-published artifacts. The automated
218use of pre-published artifacts (example: having build scripts
219use the pre-built installer by default) are potential future
220improvements outside the scope of this document.
221
222
223Performance Impact
224==================
225
226No impact to running systems. Time to obtain and evaluation
227copy of StarlingX will be drastically reduced.
228
229
230Other Deployer Impact
231=====================
232
233None
234
235
236Developer impact
237=================
238
239Reference RPMs, Installer image, and ISO can be used to compare code
240in development.
241
242
243Upgrade impact
244===============
245
246None
247
248
249Implementation
250==============
251
252
253Assignee(s)
254===========
255
256Primary assignee:
257 Scott Little <slittle1>
258
259Other contributors:
260 Jason McKenna <jmckenna>
261 Don Penney <dpenney>
262 Al Bailey <albailey>
263
264
265Repos Impacted
266==============
267
268stx-tools - scripts to assist in publication of build outputs
269
270
271Work Items
272===========
273
274* Setup base metal (CENGN) Server to perform builds
275* Create mechanism for server to perform periodic build of master branch
276* Create mechanism for server to perform build of designated release branch
277* Create script or mechanism to publish builds to existing mirror
278
279
280Dependencies
281============
282
283None
284
285
286Testing
287=======
288
289Basic installation and load sanity testing of produced iso.
290
291
292Documentation Impact
293====================
294
295Documentation for the artifacts will be posted on the StarlingX wiki
296
297
298References
299==========
300
301https://www.cengn.ca/
302http://mirror.starlingx.cengn.ca/mirror/
303
304
305History
306=======
307