tools/centos-mirror-tools
Scott Little cf435e6c2a download_mirror.sh fails with 404 errors
The 'yum makecache' step is failing. The issue is seen when the fast
mirror plugin for yum is in use (default).

The metadata for a yum repo as scattered across several files. The
master file is repomd.xml, and it may list several supporting files,
e.g.
   filelists.xml.gz
   primary.xml.gz
   filelists.sqlite.bz2
   other.sqlite.bz2
   other.xml.gz
   primary.sqlite.bz2

When fast mirror plugin is in use, each file might download from a
separate source. This opens the door for a race condition when a repo
update is rolling out across the mirrors.

The error suggests we have already downloaded a repomd.xml, but it is
obsolete and references a supporting file that no longer exists.

A second possible source of inconsistent data is caching proxies.

Solution:
1) Disabling the yum fast cache plugin is undesirable, and there
is no option forcing metadata iof a given repo to be single sourced.
So this update adds a limited number of retries for the
'yum makecache' command.

So far we've never seen 'yum makecache' fail twice in a row... 
largely because the 'fastest mirror' plugin rarely downloaded from the
same source twice.

2) Add 'http_caching=packages' to yum.conf, which asks upstream proxies
to not cache repodata.

Change-Id: I3fa3f61e38d868fb14f4175b87c0d323abb57def
Closes-bug: 1818911
Signed-off-by: Scott Little <scott.little@windriver.com>
2019-03-13 14:32:03 +00:00
..
rpm-gpg-keys Add key for PaaS SIG repository 2018-12-21 08:44:13 -06:00
stx_mirror_scripts Mirror failed to download 7.6 installer 2019-01-08 10:27:04 -05:00
yum.repos.d Centos 7.5 moved to vault 2019-01-07 14:29:25 -05:00
dl_other_from_centos_repo.sh Enable starlingx mirror 2018-11-02 13:33:00 -04:00
dl_rpms.sh download_mirror.sh fails with 404 errors 2019-03-13 14:32:03 +00:00
dl_tarball.sh Revert "Change download mechanism for tss2 (git clone to wget)" 2019-02-20 10:35:26 -06:00
download_mirror.sh failed to pass extra_args 2018-11-30 12:23:20 -05:00
make_stx_mirror_yum_conf.sh Enable starlingx mirror 2018-11-02 13:33:00 -04:00
mirror-check.sh Check for releasever in yum.conf before grepping 2018-09-06 11:18:28 -04:00
other_downloads.lst Add build tools 2018-06-08 17:01:43 -05:00
rpms_3rdparties.lst Merge remote-tracking branch 'starlingx/master' into HEAD 2019-02-07 12:10:09 -05:00
rpms_centos.lst Merge remote-tracking branch 'starlingx/master' into HEAD 2019-02-07 12:10:09 -05:00
rpms_centos3rdparties.lst refactor audit 2018-11-16 02:21:03 +00:00
show_imported_rpm_gpg_keys.sh Add build tools 2018-06-08 17:01:43 -05:00
starlingx_add_pkgs.sh Merge "Split and move repo data files to yum.repo.d subdirectory." 2018-08-08 16:25:51 +00:00
tarball-dl.lst Revert "Change download mechanism for tss2 (git clone to wget)" 2019-02-20 10:35:26 -06:00
url_utils.sh Enable starlingx mirror 2018-11-02 13:33:00 -04:00
utils.sh failmoved logs polluted by non-errors 2018-11-19 16:27:57 -05:00
utils_tests.sh Refactor on dl_rpms.sh 2018-09-10 09:57:39 -05:00
yum.conf.sample download_mirror.sh fails with 404 errors 2019-03-13 14:32:03 +00:00