tools/centos-mirror-tools
Abraham Arce 2f10f6f34d Add virtual environemnt deployment scripts
This adds two sets of deployments scripts to aid in setting up virtual
environments for testing StarligX, for libvirt/qemu and VirtualBox.

This is the first exposure of an internal project and is not fully
error-proofed, it should be examined carefully to understand what
the scripts will do to your system before running them.

The first set of fixes for this to make it less instrusive and work
in a shared server follow in https://review.openstack.org/#/c/597643.

There are also a couple of bashate errors fixed since that is now a voting job...

[NOTE(dtroyer): I converted the original commit message into a README
because that's information that needs to be with the scripts.]

Needed-by: https://review.openstack.org/#/c/597643
Change-Id: I0a6a148720b7a239380fd48f7ffdab272472e664
Co-authored-by: Lianhao Lu <lianhao.lu@intel.com>
Co-authored-by: Brian Avery <brian.avery@intel.com>
Co-authored-by: Nakul Dahiwade <nakul.dahiwade@intel.com>
Co-authored-by: Yan Chen <yan.chen@intel.com>
Co-authored-by: Ruijing Guo<ruijing.guo@intel.com>
Co-authored-by: Shuicheng Lin <shuicheng.lin@intel.com>
Co-authored-by: Felipe de Jesus Ruiz Garcia <felipe.de.jesus.ruiz.garcia@intel.com>
Co-authored-by: Jose Perez Carranza <jose.perez.carranza@intel.com>
Co-authored-by: Eddie Ramirez <eddie.ramirez@intel.com>
Co-authored-by: Kailun Qin <kailun.qin@intel.com>
Co-authored-by: Forrest Zhao <forrest.zhao@intel.com>
Co-authored-by: Kailun Qin <kailun.qin@intel.com>
Signed-off-by: Abraham Arce <abraham.arce.moreno@intel.com>
Signed-off-by: Dean Troyer <dtroyer@gmail.com>
2018-08-30 10:16:53 -05:00
..
rpm-gpg-keys Add missing key for CentOS Virtualization. 2018-08-16 15:26:41 -05:00
yum.repos.d Remove contentdir var from openstack queens repo 2018-08-23 18:49:58 +00:00
Dockerfile Split and move repo data files to yum.repo.d subdirectory. 2018-08-07 16:48:16 -04:00
README.rst Port documentation from markdown into rst file 2 2018-07-09 09:43:00 -05:00
dl_other_from_centos_repo.sh Use kojipkgs.fedoraproject.org as a backup rpm source. 2018-08-17 12:57:36 -04:00
dl_rpms.sh Add virtual environemnt deployment scripts 2018-08-30 10:16:53 -05:00
dl_tarball.sh Use kojipkgs.fedoraproject.org as a backup rpm source. 2018-08-17 12:57:36 -04:00
download_mirror.sh Pass optional args to RPM download scripts 2018-08-21 09:19:23 -04:00
mirror-check.sh Add mirror-check.sh to verify changes in upstream packages. 2018-08-28 16:50:38 -05:00
mvn-artifacts.lst StarlingX: Tar Compressed files repository 2018-07-03 06:04:28 -07:00
other_downloads.lst Add build tools 2018-06-08 17:01:43 -05:00
rpms_3rdparties.lst Change lists names and downloading scripts 2018-08-15 16:56:43 -05:00
rpms_centos.lst Merge "Add required packages for kuberentes api upversion" 2018-08-29 15:46:27 +00:00
rpms_centos3rdparties.lst Change lists names and downloading scripts 2018-08-15 16:56:43 -05:00
show_imported_rpm_gpg_keys.sh Add build tools 2018-06-08 17:01:43 -05:00
starlingx_add_pkgs.sh Merge "Split and move repo data files to yum.repo.d subdirectory." 2018-08-08 16:25:51 +00:00
tarball-dl.lst Tarball: expect-lite removal 2018-08-16 09:46:47 -05:00

README.rst

Create mirror for StarlingX

Step 0 - Build the container

Build the docker image on your Linux host (with Docker supported). NOTE: if necessary you might have to set http/https proxy in your Dockerfile before building the docker image below.

$ docker build -t <your_docker_image_name>:<your_image_version> -f Dockerfile .

Step 1 - Run the container

The container shall be run from the same directory where the other scripts are stored.

$ docker run -it -v $(pwd):/localdisk <your_docker_image_name>:<your_image_version>

The container can also be run the following way, so the download_mirror.sh runs automatically without having to enter the container and the step 2can be simplified.

$ docker run -it -v $(pwd):/localdisk <your_docker_image_name>:<your_image_version> download_mirror.sh

As /localdisk is defined as the workdir of the container, the same folder name should be used to define the volume. The container will start to run and populate a logs and output folders in this directory. The container shall be run from the same directory where the other scripts are stored.

step 2 - Run the download_mirror.sh script

Once inside the container run the downloader script

$ ./download_mirror.sh

NOTE: in case there are some downloading failures due to network instability (or timeout), you should download them manually, to assure you get all RPMs listed in "rpms_from_3rd_parties.lst" and "rpms_from_centos_repo.lst".

step 3 - Copy the files to the mirror

After all downloading complete, copy the download files to mirror.

$ find ./output -name "*.i686.rpm" | xargs rm -f $ chown 751:751 -R ./output $ cp -rf output/stx-r1/ <your_mirror_folder>/

In this case <your_mirror_folder> can be whatever folder you want to use as mirror.

step 4 - Tweaks in the StarlingX build system.

NOTE: step below is not needed if you've synced the latest codebase.

Go into StarlingX build system (another container which hosts cgcs build system), and follow up below steps:

Debugging issues

The download_mirror.sh script will create log files in the form of centos_rpms_*.txt. After the download is complete, it's recommended to check the content of these files to see if everything was downloaded correctly.

A quick look into these files could be:

$ cd output/ $ cat missing

In this case, there shoudn't be any package in the "missing" files.