This reverts commit 40a9622d77.
The URL for tss2 is not always working and this causes an
incomplete mirror in a clean environment.
Closes-bug:1816813
Change-Id: I9c8fa64e1f0f9154dd45a67088fdc27f4b8f8313
Signed-off-by: Marcela Rosales <marcela.a.rosales.jimenez@intel.com>
tss2 is the only tarball that is generated using git clone, this
commit makes the downloader get all the tarballs using wget.
Change-Id: Ib6009341f6b7c890f43962f2c24fc73e95ec81b7
Story: 2004413
Signed-off-by: Marcela Rosales <marcela.a.rosales.jimenez@intel.com>
Previous StarlingX API Reference documentation was hosted under
restapi-doc package and managed using maven and wadl files.
StarlingX has enabled the OpenStack API guideline to manage REST API
services allowing all previous restapi-doc references to be removed.
Maven is part of the restapi-doc requirements.
Depends-On: https://review.openstack.org/#/c/609714/
Change-Id: I007f8c7b8f736ab5b6c0b712f350d350df14744f
Signed-off-by: Abraham Arce <abraham.arce.moreno@intel.com>
StarlingX needs to download a variety of rpms and tarballs
from various upstream sources. Unfortunately the upstream sources
are not always dependable. Either servers go down, are unreachable,
or drop older content that we still depend on.
Our proposed solution is to run our own mirror to capture an
independent copy of the content needed by StarlingX.
For this purpose, a server has been set up at
http://mirror.starlingx.cengn.ca/mirror/centos
The mirror will use deterministic paths derived from the upstream
urls. Scripts will be used to convert an upstream url to
the mirror's equivalent url (see function url_to_stx_mirror_url in
url_utils.sh)
The mirror will be refreshed daily. New .lst entries will be
processed at that time. Processing of changes under yum.repos.d
is not automated by this update. Expect a follow-up update to
address this issue soon. These scripts are found under the
'stx_mirror_scripts' subdirectory.
Changes are made to the download_mirror.sh script, and it's
supporting scripts. New arguments have been added to each
script to select the download source.
-s StarlingX mirror only
-S StarlingX mirror, with upstream source as backup
-u Upstream source only
-U Upstream source, with StarlingX mirror as backup
You do not need to provide any of these flags. Continue to
us download_mirror.sh as you always have. The default
behavior is currently set to '-S', i.e. first try the
StarlingX mirror, with upstream source as backup.
If this proves to place to heavy a load on the existing
server, we might switch the default to '-U', i.e. first
try the upstream source, with StarlingX mirror as backup.
If poor download performance is seen, you might want to try
explicitly adding -U as an argument.
The remaining two options are not recommended for regular use.
Upstream only, i.e. '-u', restores original behaviour, but
you may once again encounter rpms that have aged out, and
been removed from their original repos. StarlingX only,
i.e. '-s', is vulnerable if a .lst file has been updated,
but the mirror has not yet processed it.
Change-Id: I7e0f3d9fb99253662f9f4bf12457d39250408c0b
Story: 2003906
Task: 26785
Signed-off-by: Scott Little <scott.little@windriver.com>
EPEL7 rpms builds are visible at kojipkgs.fedoraproject.org
and often remain available there long after they age out
of the official repo.
Use kojipkgs.fedoraproject.org as a backup source for
packages when not found in the repo. Log seperately
so that these dropouts can be identified and corrected
in the lst files.
Also improved the tests for already downloaded packages
in several places to accelerate the test cycle.
Change-Id: Iaaa9fe05ac605a3604acf0048c571c6e88303692
Story: 2003157
Task: 23292
Signed-off-by: Scott Little <scott.little@windriver.com>
This change improves support for organizational or personal local
mirrors. Download tools work in a wider variety of environmnets,
and the population of build environments is now driven by .lst
files.
The download_mirror.sh (and related files) now have options to
better support running in non-containerized environments. This
includes options to run using local yum.conf files (rather than
system files) and not executing sudo or chgrp commands. These
options are enabled by flags. Default behaviour remains
unchanged.
The generate-cgcs-centos-repo.sh tool now populates build
environment as driven by lst files. This allows a mirror to be
maintained (and grown) at an organizational level, while allowing
a developer to produce exactly the environment they need as per
the lst files they have checked out.
generate-cgcs-centos-repo.sh now also populates the downloads
directory by invoking the populate_downloads.sh script (also
driven by lst files)
Change-Id: I8aae32c776df7618a5ef545a0ce43903b41112d0
Story: 2003422
Task: 24586
Signed-off-by: jmckenna <jason.mckenna@windriver.com>
This in order to keep consistency between downloader script names
(dl_rpms.sh, dl_other_from_centos_repo.sh and now dl_tarball.sh).
Change-Id: I50d029bced2db6c97039d82f08e1e120476e47d9
Signed-off-by: Marcela Rosales <marcela.a.rosales.jimenez@intel.com>
Saul Wold