Commit Graph

140 Commits

Author SHA1 Message Date
OpenDev Sysadmins 0d2c53eb5e OpenDev Migration Patch
This commit was bulk generated and pushed by the OpenDev sysadmins
as a part of the Git hosting and code review systems migration
detailed in these mailing list posts:

http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.html
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004920.html

Attempts have been made to correct repository namespaces and
hostnames based on simple pattern matching, but it's possible some
were updated incorrectly or missed entirely. Please reach out to us
via the contact information listed at https://opendev.org/ with any
questions you may have.
2019-04-19 19:52:37 +00:00
Saul Wold 76ae6ed805 Merge remote-tracking branch 'gerrit/master' into f/centos76
Change-Id: I2e9b9348043cf03bd7b49c222b9d26965cd22724
Signed-off-by: Saul Wold <sgw@linux.intel.com>
2019-03-04 17:40:35 -08:00
Don Penney dc2d33642a Increase yum timeout in stx-centos base image
There have been intermittent failures building StarlingX
docker images due to failures to reach the mirrors within
the default timeout, during the formal CENGN builds. This
update adds a step to the docker files for the stx-centos
base image and the base wheel builder image to set a new
timeout value in the yum.conf file, increasing from the
default 30 seconds to 120 seconds.

Change-Id: Ide52cdebb053f639cfe6d4c3a6c5eef71e6fa8ab
Story: 2004751
Task: 29636
Signed-off-by: Don Penney <don.penney@windriver.com>
2019-02-21 15:04:40 -05:00
Saul Wold 8d756e3b59 Merge branch 'master' into f/centos76
Change-Id: I7501212f81381043860d67a5c576e84d6bb0dd6a
Signed-off-by: Saul Wold <sgw@linux.intel.com>
2019-02-20 14:16:18 -08:00
Jesus Ornelas Aguayo ca2bf8e41e Fix mismatch in wheelname
Build wheels process fails while is copying the wheel to the tarball due
to a mismatch in the name, this commit changes the wheelname
to upper case.

Closes-Bug: 1816666

Change-Id: Ie9e1c41d921447f15788d3c2a5cb068d4794ee61
Signed-off-by: Jesus Ornelas Aguayo <jesus.ornelas.aguayo@intel.com>
2019-02-19 15:41:54 -05:00
Don Penney 047c424daa Update loci to latest version in build-stx-images.sh
The build-stx-images.sh utility references a fixed loci SHA in order
to isolate the image builds from tool churn.

A recent update to virtualenv has broken loci builds, as described here:
http://lists.openstack.org/pipermail/openstack-discuss/2019-February/002592.html

This update sets the loci SHA to the commit that addresses this issue.

Change-Id: I4c5404c72935433b217d534fc9129f6ea69f23b1
Story: 2004751
Task: 29461
Signed-off-by: Don Penney <don.penney@windriver.com>
2019-02-12 19:20:33 -05:00
Zuul efac264bab Merge "Provide pecan and wsme modules as wheels packages" 2019-02-12 20:13:12 +00:00
Saul Wold d6fc72093b Merge remote-tracking branch 'gerrit/master' into f/centos76
Change-Id: I0eafa9c0c017f96b9fac357f9de4b32f79f1ec43
Signed-off-by: Saul Wold <sgw@linux.intel.com>
2019-02-12 07:56:33 -08:00
Mario Alfredo Carrillo Arevalo dc7cf42993 Provide pecan and wsme modules as wheels packages
The fm-api application requires pecan and wsme python
modules, this commit adds them to pike-wheels.cfg in
order to be built and they can be available in the wheels
tarball.

Story: 2004008
Task: 29400

Change-Id: Ieb15f49f18e8a6598e91819a5b76b4a6763e3e5e
Signed-off-by: Mario Alfredo Carrillo Arevalo <mario.alfredo.c.arevalo@intel.com>
2019-02-07 23:28:49 +00:00
Don Penney 1f56c5036e Add support for building wheels from master branches
This update enhances build-base-wheels.sh to support using a
wildcard in wheels.cfg for the wheel name, to allow for whatever
the version is when building from master for a given repo.

This also updates master-wheels.cfg to build the following modules
from master, to pick up the latest code for these supporting modules:
neutron-lib
python-openstackclient
openstacksdk

Story: 2004751
Task: 29395
Change-Id: Ifab8a209170a00f58e0f4a53b6cbd7e4fe86e8f0
Signed-off-by: Don Penney <don.penney@windriver.com>
2019-02-07 10:52:11 -05:00
Zuul 896a066211 Merge "Remove unused variables" 2019-02-06 16:38:03 +00:00
Zuul b652455b91 Merge "Add support to set proxy as argument" 2019-02-06 15:07:58 +00:00
Al Bailey 8849490971 Fix docker image build failures due to google-api-python-client
This enforces google-api-python-client 1.7.7 which
is available on pypi site as a wheel that is both python2
and python3.

This affected only master docker images, pike was not
impacted

Closes-Bug: 1814835
Change-Id: I1bae1a5a3a1b0fcd323d072a113381c3340f9134
Signed-off-by: Al Bailey <Al.Bailey@windriver.com>
2019-02-05 16:38:26 -06:00
Mario Alfredo Carrillo Arevalo d98601c46a Add support to set proxy as argument
When the building scripts involved in the containerization
processes are executed behind a proxy, the user must set the
proxy manually making modifications in the scripts or dockerfiles.
These changes allows to set it as an argument.

Story: 2004925
Task: 29305

Change-Id: I52e13e4832daf488d8ff29334686a20daa7d2826
Signed-off-by: Mario Alfredo Carrillo Arevalo <mario.alfredo.c.arevalo@intel.com>
2019-02-02 03:50:55 +00:00
Felipe de Jesus Ruiz Garcia d41eff31ea Remove unused variables
Partial cleanup of the python code.

Story: 2004825
Task: 29286

Change-Id: I146c626342a8be2c93da21cf6cb229a39e8eac10
Signed-off-by: Felipe de Jesus Ruiz Garcia <felipe.de.jesus.ruiz.garcia@intel.com>
2019-02-01 22:58:37 +00:00
Don Penney 87cbff8822 Add panko to master wheels tarball
The stx-ceilometer master image requires a specific
version of panko, which does not have a pre-built wheel
available in pypi, so we're adding it to the master
wheels tarball to make it available to the image build.

Change-Id: I854b75577b6dbc3f0a8171190f5a1aa839412dc8
Story: 2004520
Task: 29046
Signed-off-by: Don Penney <don.penney@windriver.com>
2019-01-23 11:43:19 -05:00
Saul Wold ae9d73fc02 Merge remote-tracking branch 'gerrit/master' into f/centos76
Change-Id: Icda32d3d7683aa38ec84e5e038d668367cec0567
Signed-off-by: Saul Wold <sgw@linux.intel.com>
2019-01-22 08:28:58 -08:00
Zuul 9cb8be1e9d Merge "Allow helm charts to be built without requiring network access" 2019-01-16 15:41:03 +00:00
Saul Wold c49b55c03c Merge remote-tracking branch 'gerrit/master' into f/centos76
Change-Id: Iefe4f7547c43dd69539262ed9cb8faa1d701ef0e
Signed-off-by: Saul Wold <sgw@linux.intel.com>
2019-01-11 09:01:10 -08:00
Felipe de Jesus Ruiz Garcia e687eab44e Change file permissions of RPM's to 644 in the ISO
This commit adds 644 permissions for all the files under
/www/pages/feed/<release>/Packages, allowing the web server to reach
all the required files to provision compute nodes.

The internal web server used to provide rpms for the installation of
computes and additional controller can fail to serve packages if these
has different permission less than the required by the web server.

lighttpd runs as a non-root user, however the content under
/www/pages/feed/<release>/Packages is owned by root. If the files hasn't
the "read for other" bit set, then this files won't be provided by the
web server and thus causing a failure in the installation of the
computes.

We ensure that all the RPM's has the 644 permissions in that
location after to build the StarlingX ISO image.

Closes-Bug: #1794894

Change-Id: I928b156dad1cffcda2d9a8d41631a56ed7ee29f7
Signed-off-by: Felipe de Jesus Ruiz Garcia <felipe.de.jesus.ruiz.garcia@intel.com>
2019-01-10 17:52:56 -05:00
Al Bailey 2ae7465f07 Allow helm charts to be built without requiring network access
openstack-helm-infra, openstack-helm and stx-openstack-helm
no longer require network access, and can therefore be built
as part of build-pkgs in std mode.

The build-helm-charts.sh now packages from std results

Story: 2004005
Task: 28792
Change-Id: If39686cfad5f0f92f6ddfb2351684294b134b5fa
Depends-On: I113ec91b64faebf2e7e8154e4bfbe75acc3fbf43
Signed-off-by: Al Bailey <Al.Bailey@windriver.com>
2019-01-08 11:54:56 -06:00
Don Penney 189114b978 Add master openstack support to stx-base build tool
Update the build-stx-base.sh tool to support building
stx-centos for use with images based on Openstack master.

Change-Id: Ic77036e15529dadf39df57e5e88a2000b88bd6cb
Story: 2004520
Task: 28726
Signed-off-by: Don Penney <don.penney@windriver.com>
2019-01-04 13:54:01 -05:00
Saul Wold eb6e03b078 Merge remote-tracking branch 'gerrit/master' into f/centos76
Change-Id: Ia6db918d0deabf2b9203f57e72500353650f747c
Signed-off-by: Saul Wold <sgw@linux.intel.com>
2019-01-02 14:55:32 -08:00
lihaijing 9106b90dba Change single quota mark to double quota
In order to get correct "$PLATFORM_RELEASE" value

Change-Id: I842411baf7c2b9bb862fceb920919334e40194aa
Closes-Bug: #1809637
Signed-off-by: lihaijing <lihaijing@fiberhome.com>
2018-12-26 19:13:57 -05:00
Zuul f99c48ad0b Merge "build-stx-base.sh --latest --clean fails to clean the latest image" 2018-12-21 19:24:46 +00:00
Saul Wold 8312ffad07 Merge remote-tracking branch 'gerrit/master' into f/centos76
Change-Id: I66ff26289b850d0fcfcb7c4c3db32ded81beba32
Signed-off-by: Saul Wold <sgw@linux.intel.com>
2018-12-20 12:13:21 -08:00
Don Penney 550c28895f Update keyring and os-vif in pike wheels tarball
StarlingX requires a higher version of the keyring and os-vif
python modules than is specified in the stable/pike
upper-constraints.txt file. This update specifies a newer
version for these modules in the pike-wheels.cfg file to
override the constraints for the docker image build.

Story: 2004520
Task: 28585
Change-Id: I9bdaaad2575f8832f19f4fdac26dae76fd83c442
Signed-off-by: Don Penney <don.penney@windriver.com>
2018-12-19 08:59:02 -06:00
Scott Little 32c08a8dd4 build-stx-base.sh --latest --clean fails to clean the latest image
When both --latest and --clean flags are passed to build-stx-base.sh
we expect the docker image tagged as '...-latest' to be deleted
from the local build environment.

This did not occure ...

starlingx/stx-centos      dev-latest          279c713e0b5d        7
hours ago         331MB
starlingx/stx-centos      <none>              dda342637517        19
hours ago        331MB

Adding a code block to handle the case when these flags are combined.

Closes-Bug: 1808544
Change-Id: Ic2d17bc285f3746965c2a5e7ec49c278f0fbf066
Signed-off-by: Scott Little <scott.little@windriver.com>
2018-12-14 10:49:37 -05:00
Scott Little f1bfa53e87 Fix 'build-pkgs --container' when building in docker
build-pkgs --container fails when run within docker
due to DNS lookup failure.

Need to add pkg bind-utils to mock when building in
the 'container' context, and to add an /etc/resolv.conf
via 'config_opts['use_host_resolv'] = True'

Story: 2003907
Task: 28370

Change-Id: I5f9ec16d29545ba74457a060323e07967e085556
Signed-off-by: Scott Little <scott.little@windriver.com>
2018-12-11 13:43:14 -05:00
Dean Troyer aeaf791ff8 Update .gitreview for f/centos76
Change-Id: I8494fe2a9a1ce7911c57c3425c3cea947035ea76
Signed-off-by: Dean Troyer <dtroyer@gmail.com>
2018-12-10 12:59:02 -06:00
Scott Little cf0135e886 Add greater flexibility in docker image naming
The existing user interface for docker image and wheel creation
is not flexible enough to name images is described in the spec,
and to push those images to the starlingx organization at docker.hub

This update adds new arguements that will allow creation of
docker images with compliant names, and will allow them to
be pushed to https://hub.docker.com/r/starlingx.

I will also change build-base-wheels.sh to allow the
'os-version' to be pushed through to build-wheels-tarball.sh.

Story: 2003907
Task: 28370

Change-Id: I4378db63d5ff906ffd636034351e580afa5e3472
Signed-off-by: Scott Little <scott.little@windriver.com>
2018-12-10 13:11:09 -05:00
Don Penney a9cdcae974 Update docker image and wheels tools for build on master
This update adds support to the docker image and wheels build tools
to allow the build of StarlingX service images from openstack master
branches.

The master wheels tarball makes use of the loci-built master-OS wheel
image as a base, with any additional base wheels not provided by loci.

Change-Id: I6de67e29a9b05189f2a0bad3dfdd1668fbdab42b
Story: 2003907
Task: 28179
Signed-off-by: Don Penney <don.penney@windriver.com>
2018-12-03 14:53:56 -06:00
Zuul dbcf315a70 Merge "Move the cd statement to the correct location" 2018-11-29 15:33:50 +00:00
Zuul 71a7ba7a81 Merge "Fix build-srpms cannot edit a variety of packages" 2018-11-29 07:52:39 +00:00
Shuicheng Lin 3f5687b3e5 Fix build-srpms cannot edit a variety of packages
Closes-Bug: 1792001

Fix edit issue for package puppet-nova/bash/resource-agents.

The cause of the issue is, there is "rm -rf %{buildroot}"
or similar line in the spec, which cause the buildroot folder
be deleted, and git info be deleted also. So lead to the
"git add" cmd failure.
Fix:
Try to ignore this cmd to avoid git info be deleted.

Test:
This script is for edit cmd only, no impact to normal build.
Tested several packages locally, no issue found yet.

Change-Id: I667c363bd82e68f107a6c51f65a2a9dd0d2434d4
Signed-off-by: Shuicheng Lin <shuicheng.lin@intel.com>
2018-11-27 00:29:52 +08:00
Don Penney a6a7d1380c Remove -i and -t options from docker run command
When the build-base-wheels.sh is run via Jenkins, there
is no tty, so docker rejects the -i and -t options.
These are not necessary for the build, and are removed
from the command execution.

Story: 2003907
Task: 27531
Signed-off-by: Don Penney <don.penney@windriver.com>
Change-Id: I4b1f4c2573df7b62acd48ee404d5aa3c098a50b1
2018-11-23 09:00:21 -06:00
Zuul 81a4adc96b Merge "Add tools for building StarlingX docker images" 2018-11-20 20:48:10 +00:00
Zuul ae8c653f47 Merge "Enhancements to wheel build tools" 2018-11-20 20:15:27 +00:00
Don Penney f37823be9c Add tools for building StarlingX docker images
This update adds tools for building a base image (ie. stx-centos)
and the set of images for the StarlingX services.

build-stx-base.sh will build the base image
build-stx-images.sh will build the StarlingX services

Change-Id: I499583552f633b9e29566dde9233f5c5208d0593
Story: 2003907
Task: 27938
Signed-off-by: Don Penney <don.penney@windriver.com>
2018-11-19 13:43:38 -06:00
Don Penney 2b68edd955 Enhancements to wheel build tools
This update provides the following enhancements to the wheel tools:
- Add a prefix to config files for openstack release
- Change auto-generated version format
- Skip base wheel build if all base wheels are present
- Clean unneeded images from docker after build
- Fix bash globbing issue in get-stx-wheels.sh

Change-Id: I17dbf01288c29662fddae4fd3d9d90ab2521037f
Story: 2003907
Task: 27531
Signed-off-by: Don Penney <don.penney@windriver.com>
2018-11-19 13:43:26 -06:00
Scott Little 0b8e1605db sign-rpms: Don't attempt off-box signing if variables not set.
Problem:
User 'jenkins' hit a build-iso failure in sign-rpms.
Problem was a test that assumes user jenkins will always
provide environment variable setting for SIGNING_USER,
SIGNING_SERVER and SIGNING_SERVER_SCRIPT.  This is not
a valid assumption.

Solution:
Explicitly test that all three environment variables are set
before invoking sign_packages_on_server.

Story: 2003906
Task: 27974
Change-Id: If9eb61b8001ca58f9262c2eccab96b4820c68102
Signed-off-by: Scott Little <scott.little@windriver.com>
2018-11-19 11:15:51 -05:00
Felipe de Jesus Ruiz Garcia bfb9ca607d Move the cd statement to the correct location
Move the cd statement to the correct location above the sanity
check that the operation completed.
the validation below, comparate the latest statement was sucessfull.
so this, should be the cd statement.

Change-Id: Idd093cf215cf1b637e37b89e8496231c64260972
Signed-off-by: Felipe de Jesus Ruiz Garcia <felipe.de.jesus.ruiz.garcia@intel.com>
2018-11-14 23:22:45 +00:00
Al Bailey 8dd7a8fe2a Adding a build tool to create a helm charts tarball
The helm charts are packaged in rpms by the build process
and the purpose of this script is to package all those
artifacts together into a single tarball, for use by
other components in StarlingX.

Uses centos_helm.inc in each repo to determine what to search for.

Currently tested with packages from stx-config and stx-upstream:
 openstack-helm-infra
 openstack-helm
 stx-openstack-helm

Multiple tarballs are created, one per manifest.
A calculated checksum.md5 is included in each tarball

Story: 2004005
Task: 27933
Depends-On: I3727a15fbdadc345a104c2af3446aee845171309
Change-Id: I06259d27e8f25d0b05eecdea49b2365246ac3112
Signed-off-by: Al Bailey <Al.Bailey@windriver.com>
2018-11-14 11:21:21 -06:00
Zuul ec94557c95 Merge "Provide a new build mode for containers" 2018-11-13 16:52:10 +00:00
Zuul 6f0aa61026 Merge "Handle multiple optional dependancies and copy list" 2018-11-12 08:48:31 +00:00
Al Bailey aa15df5887 Provide a new build mode for containers
Providing a new build target --containers.

Container build targets typically need network access to
build, and do not need to be built as part of a regular build.

Container build targets are defined by: centos_pkg_dirs_containers
Container build targets can utilize the std and rt repos for
their build requirements.
Container target enables the mock flag: rpmbuild_networking

This also includes a fix where build-pkgs --installer in parallel
mode was building in serial mode instead.

Story: 2003909
Task: 27632
Change-Id: I3727a15fbdadc345a104c2af3446aee845171309
Signed-off-by: Al Bailey <Al.Bailey@windriver.com>
2018-11-08 14:01:44 -06:00
Jason McKenna 331e07254f Handle multiple optional dependancies and copy list
RPMs which are built from a build_srpm.data file may have optional
COPY_LIST_TO_TAR and OPT_DEP_LIST items listed in the file.  These
variables are assumed to be single items by the md5sums_from_input_vars
function used by build-srpms during build-avoidance.

What this means is that listing multiple items are not currently
handled by build-srpms during build-avoidance.

This change handles each item in the variables independently.

Change-Id: I57494d23680b726eb8dca712444c45879897ca50
Signed-off-by: Jason McKenna <jason.mckenna@windriver.com>
Closes-Bug: 1800675
2018-11-02 14:48:05 +00:00
Al Bailey 1615f28109 Support for network access from within mock.
In order to run certain networking actions within mock,
this flag needs to be a part of the config for the build env.
For example: running helm commands in a spec file.

Story: 2003909
Task: 27632
Depends-On: I0331bf409c25078bb61d6a9b09e3107366030c01
Change-Id: I5c761b9261e72783f1771492d653e641193f7c52
Signed-off-by: Al Bailey <Al.Bailey@windriver.com>
2018-11-02 09:15:24 -05:00
Don Penney 0ed04ca11c Add tools to build python wheels and wheel tarball
This update adds tools for building wheels for upstream python
modules, and generating a StarlingX wheels tarball.

Building upstream python modules is done within a docker container,
with a wheels.cfg configuration file that identifies the upstream
sources and expected python wheel output. The build-base-wheels.sh
tool will create the wheel builder image, launch it, and run the
build script.

The get-stx-wheels.sh tool will extract the python wheels built by
the StarlingX load.

The build-wheel-tarball.sh script will combine the wheels from the
base and StarlingX wheel sets into a single tarball that can be used
when building container images for StarlingX services.

Story: 2003907
Task: 27531
Change-Id: Icf6c63699a635d1173dcef9eb1d12ae7c4e39108
Signed-off-by: Don Penney <don.penney@windriver.com>
2018-10-25 23:15:57 -04:00
Hayde Martinez 867323149e Adding new packagereq for hostname in repo_files
New packagereq was added to file mocks.xml for hostname package
required by different packages

Closes-Bug: 1799798

Change-Id: I8677404eb7623212f2ec52ef9c82afdaed438973
Signed-off-by: Hayde Martinez <hayde.martinez.landa@intel.com>
2018-10-24 21:06:45 +00:00