The StarlingX mirror now uses https. Fix the port number to match.
Partial-Bug: 2033555
Signed-off-by: Scott Little <scott.little@windriver.com>
Change-Id: I19d2b18d64c26b65cf2b8c05fa642556d9724956
mirror.starlingx.cengn.ca no longer exists. CENGN is kindly forwarding
requests to the new location mirror.starlingx.windriver.com for now, but
that will only last a few months. We need to replace all the references
with the new URL.
I will also remove as many 'cengn' references as possible, replacing
them with 'stx_mirror'
Partial-Bug: 2033555
Signed-off-by: Scott Little <scott.little@windriver.com>
Change-Id: I09e3f564edef2049786c965a86dbcaacac359801
Previous fix failed to give the cengn mirror a unique repo name.
Closes-Bug: 1998234
Signed-off-by: Scott Little <scott.little@windriver.com>
Change-Id: Ib43ea12bc29f5ffb1c67813c55504f565bc2984d
The upstream rpm repo for katacontainer no longer exists.
Switch to our CENGN mirror copy.
Partial-Bug: 1998234
Signed-off-by: Scott Little <scott.little@windriver.com>
Change-Id: Iabdf83d990f71ccae1ee9efb5af8e638c1c4543b
Helm plugin mapkubeapis needs to be installed to help application user
deal with deprecated kubernetes resources apis.
The plugin tarball must be added to the exceptions of dl_tarball as
the script by default untars with `--strip-components 1`, thus removing
the plugin binary before recompressing
Story: 2009138
Task: 46022
Signed-off-by: Lucas Cavalcante <lucasmedeiros.cavalcante@windriver.com>
Change-Id: I4aac5b865bd1c4db519717135fbaa81ea2bda6cb
download_mirror.sh fails when run without '-n'.
It seems that some functions were relocated to utils.sh that
require the 'SUDO' variable to be defined. Those common utilities
are called from both download_mirror.sh and dl_rpms.sh.
download_mirror.sh sets it correctly, but dl_rpms.sh does not.
Instead dl_rpms.sh is setting 'SUDOCMD', which the utils.sh ignores.
The result is that some yum commands are runs under sudo, while others
are not. Eventually yumdownloader failes with ...
Permission denied: '/var/lib/rpm/.dbenv.lock'
The fix is to convert dl_rpms.sh to use SUDO rather than SUDOCMD.
Closes-Bug: 1980684
Signed-off-by: Scott Little <scott.little@windriver.com>
Change-Id: Ib4afdba0260e67f06997c360272f40f28fb11834
Following merge of 845644, we no longer need expat-devel
since it will be built from source.
Partial-Bug: 1975755
Change-Id: Ia3fc7c4d45fe400acb759018803b7fc5b44d53b0
Signed-off-by: Joe Slater <joe.slater@windriver.com>
After download_mirror.sh, system download kernel-rt and centos-git-common.
However, one important script, "get_sources.sh", is not executable.
It will cause built error.
this patch will change the file mode to fix this problem.
Closes-Bug: #1979341
Signed-off-by: ChantYuCN <chengde.yu@intel.com>
Change-Id: I136b92ebc943b2a9a8c6ad9cb85fe802acce3c09
Helm plugin 2to3 needs to be installed to upgrade apps to FluxCD.
The plugin tarball must be added to the exceptions of dl_tarball as
the script by default untars with `--strip-components 1`, thus removing
the plugin binary before recompressing
Story: 2009138
Task: 45584
Signed-off-by: Lucas Cavalcante <lucasmedeiros.cavalcante@windriver.com>
Change-Id: I19b541500fd0872660c98e2f20baccf3f52c77da
CVE-2021-45960
CVE-2022-22822
CVE-2022-22823
CVE-2022-22824
CVE-2022-23852
CVE-2022-25235
CVE-2022-25236
CVE-2022-25315
Advance to expat-2.1.0-14.el7_9.
=== Testing ===
build-iso; install; boot
# run test to see if an xml file is well-formed
$ xmlwf -c -d /tmp /etc/firewalld/zones/public.xml
$ cat /tmp/public.xml # should look like an xml file
===
Closes-bug: 1969362
Change-Id: I78f1abc4253d0016fed6845202e00cab91e9ed11
Signed-off-by: Joe Slater <joe.slater@windriver.com>
Unsafe deserialization in chainsaw. Advance to
version 1.2.17-18.el7_4.
=== Testing ===
build-pkgs/build-iso and boot.
log4j is not in the runtime system, nor is it in
the mock build environment.
===
Closes-bug: 1969993
Signed-off-by: Joe Slater <joe.slater@windriver.com>
Change-Id: I0e16887da7c22173c0c05c60a49bf026521d93a7
NOTE! commit fc00096e8... purports to fix the first 3 CVEs
but uses the wrong rpm version.
CVE-2021-26691: heap overflow
CVE-2021-39275: out-of-bounds write
CVE-2021-44790: buffer overflow
CVE-2022-22720: http request smuggling
Advance to version 2.4.6-97.el7.centos.5.
=== testing
boot iso and log in; become root; httpd is not running
systemctl stop lighttpd # free up port 80
systemctl start httpd # takes a while
echo arf > /var/www/html/arf.txt # something to fetch
wget http://localhost/arf.txt
cat arf.txt
This shows httpd is processing requests.
===
Closes-bug: 1960765
Closes-bug: 1969363
Change-Id: I4c90213f020762f037e1f207f73e0622a38984c2
Signed-off-by: Joe Slater <joe.slater@windriver.com>
ICE driver update to support the Intel Logan Beach NIC (E810-CQDA2T).
and that dictates the upgrade of ddp firmware to ice_comms-1.3.35.0,
which is included in 27_1.zip.
https://www.intel.com/content/www/us/en/download/15084/intel-ethernet-adapter-complete-driver-pack.html
Remove the extra extraction step because it is zip file that we
want, rather than zips of zips.
Verification:
Success Path
- run download_mirror.sh, verify that the ice_comms zip file is
downloaded in downloads folder.
Story: 2009952
Task: 44895
Signed-off-by: Jiping Ma <jiping.ma2@windriver.com>
Change-Id: I7561543ba0bce5d2fd82a0315f81ba3dc3f1ba0b
This commit adds support for downloading Broadcom's NetXtreme-E
driver/library tar archive file. The tar file consists of multiple
nested archives and source RPM files. We are only interested in the
libbnxt_re Infiniband verbs library SRPM and the bnxt_en/bnxt_re kernel
driver source code archive:
- libbnxt_re-220.0.5.0-rhel7u9.src.rpm
- netxtreme-bnxt_en-1.10.2-220.0.13.0.tar.gz
If the archive has already been downloaded, the sha256sum of the archive
is verified. (The checksum is stored in the utility/"util" field in the
listing file.) If not already downloaded, then the archive is downloaded
and the sha256sum of the archive is checked. Finally, the desired files
are extracted from the main tar archive.
(Checksum verification is added as a package-specific behaviour, because
this feature does not exist in the build system, and we would like to be
aware in case the software package is modified.)
Testing:
- dl_tarball.sh correctly removes a pre-existing tar archive with an
incorrect sha256sum and re-downloads the tar archive.
- If the tar archive does not already exist, then the archive is
correctly downloaded and the sha256sum is correctly checked.
- If the download_file function or the check_sha256sum function fails,
then the shell script correctly reports an error message and
continues. (This was verified with shell script instrumentation.)
- If the tar archive extraction fails or if the expected/desired files
cannot be found, an error is correctly reported by the shell script,
and the tar archive is removed to allow follow-up attempts to try
again. (Also verified with instrumentation.)
Story: 2009915
Task: 44761
Change-Id: Id021a33e7f26643139d6ef0dda5c7146cfb7f172
Signed-off-by: M. Vefa Bicakci <vefa.bicakci@windriver.com>
CVE-2021-44142: out-of-bounds heap read/write
CVE-2020-25717: user can become root
CVE-2020-25719: AD DC does not always rely on the SID and PAC
=== testing
Boot iso and check rpm versions. Only samba
libraries are included in the image.
===
Closes-bug: 1964842
Signed-off-by: Joe Slater <joe.slater@windriver.com>
Change-Id: I55a97b662ac24c1ba9852a09d8e40b5a40f67945
CVE-2021-26691: heap overflow
CVE-2021-39275: out-of-bounds-write
CVE-2021-44790: buffer overflow
Advance to version 2.4.6-97.el7.centos.
=== testing
boot iso and log in; become root; httpd is not running
systemctl stop lighttpd # free up port 80
systemctl start httpd # takes a while
echo arf > /var/www/html/arf.txt # something to fetch
wget http://localhost/arf.txt
cat arf.txt
This shows httpd is processing requests.
===
Closes-bug: 1960765
Signed-off-by: Joe Slater <joe.slater@windriver.com>
Change-Id: Idcff71fe505a187e7bcfaea7a8818233a4ef76ac
dl_tarball.sh assumes an el8 kernel, and lacks of error
handling. This may result in an attempted download of a
non-el8 reporting success, but producing a src.rpm containing
an el8 kernel instead.
Testing:
- Download current el8 rt kernel
- Download previous el7 rt kernel
- DL using invalid URL - error caught
- DL using invalid sha - error caught
Closes-Bug: 1964156
Change-Id: I5ca5130a7f1c4e967c876060d95f9d6a2c2e8cf5
Signed-off-by: Scott Little <scott.little@windriver.com>
Copy the following packages to downloads folder, the ones are included
in MLNX_OFED_SRC-5.5-1.0.3.2.tgz.
mlnx-ofa_kernel-5.5-OFED.5.5.1.0.3.1.src.rpm
rdma-core-55mlnx37-1.55103.src.rpm
mlnx-tools-5.2.0-0.55103.src.rpm
mstflint-4.16.0-1.55103.src.rpm
Removed the original mlnx-ofa_kernel-5.3-OFED.5.3.1.0.0.1.src.rpm from
rpms_3rdparties.lst, that added in commit
e8d164e801e3fec796fa24b2eaf90f50726c38a(Prep: Upgrade kernel
5.10 related packages).
Story: 2009878
Task: 44610
CentOs Test:
- PASS: Run download_mirror.sh that can download all mlnx related
packages.
- PASS: Build image
- PASS: Boot image
Signed-off-by: Jiping Ma <jiping.ma2@windriver.com>
Change-Id: I67e058f9c9bdab31e87d46daf0dec3b353dc68a6
pkexec always assumes there is at least one argument, which can be
exploited by crafting the environment and calling it with no
arguments. No specific exploit has been published.
Update to polkit-0.112-26.el7_9.1.
== testing ==
We just want to see if pkexec stills works.
build and install an iso, then
$ sudo pkexec --user puppet id
Password: # enter sysadmin password
uid=52(puppet) gid=52(puppet) groups=52(puppet)
$
====
Closes-bug: 1960087
Signed-off-by: Joe Slater <joe.slater@windriver.com>
Change-Id: I267e29d90e75dc772e17f0b5866850b4bb5ac3d2
This is a firmware package for Intel Ice NIC hardware,
addressing dynamic device personalization for the
communications market.
Intel likes their zips of zips, causing us to have to
perform an extra extraction step to get the zip file
that we want.
Verification:
Success Path
- create a test .lst file with just the entry for the
intel ice comms ddp firmware in it
- run the dl_tarball.sh script against the test .lst file
- verify that the ice_comms zip file is successfully
produced in the proper directory
- verify that the wrapper zip file is also left in the
proper directory
Failure Path
- modify test .lst so that the url to the wrapper zip
is no longer valid
- verify that the script reports the download error
properly and handles it properly
Story: 2009823
Task: 44410
Change-Id: I2f8229d9a67ca986c4ad97049e8abf141cd07895
Signed-off-by: Jim Somerville <Jim.Somerville@windriver.com>
nss is vulnerable to a heap overflow when handling DER-encoded
DSA or RSA-PSS signatures. We update nss packages and nspr to
the latest centos7 versions.
*** Testing ***
To be sure we will work with existing databases, before updating,
create a database.
$ mkdir arf
$ echo "Pword22*" > arf/pass.
$ certutil -N -d arf -f arf/pass
$ certutil -G -d arf -f arf/pass # put a key pair in the database
Save the arf directory. Install an iso with the updated nss packages.
Import arf. Then...
$ certutil -K -d arf -f arf/pass # display the keyID
$ certutil -G -d arf -f arf/pass # add a key
$ certutil -K -d arf -f arf/pass # display both keyID's
***
Closes-bug: 1957929
Change-Id: I960e42d1e361dace4443d6a052fe06206c6675dd
Signed-off-by: Joe Slater <joe.slater@windriver.com>
Removing from distro dependencies the initscripts package. This was
added without a need as StarlingX itself has a initscripts package and
this is the one included in the image.
Adding python 3 dependency needed to compile Ceph's python3 packages.
Test plan:
Complete build run
Starlingx installation
Story: 2009074
Task: 44281
Signed-off-by: Delfino Curado <delfinogomes.curadofilho@windriver.com>
Change-Id: I381adbec209bcbbb3561457db1460a25e1809f5f
Upstream hasn't created any repodata for these two repos yet.
StarlingX-C8.5.2111-centosplus-Source
StarlingX-C8.5.2111-extras-Source
The mirroring job fails if there is a reference to invalid upstream repos.
Closes-bug: 1952478
Change-Id: I039a3577e1e8dab1b828fb0c114d226d9fbf4d3c
Signed-off-by: Scott Little <scott.little@windriver.com>
Intermittent build failures are seen that always include rdma-core.
It appears that rdma-core is both built by StarlingX, and listed as
a pre-built binary, and the two are not entirely compatible.
It seems to depend on what jobs were submitted to the same build engine
prior to building rdma-core. If the pre-built rdma-core was pulled into
that mock environment previously, the new rdma-core won't build.
There is no need for the pre-built rdma-core. It needs to be
removed.
Closes-Bug: 1951463
Signed-off-by: Scott Little <scott.little@windriver.com>
Change-Id: Id3f84d2d12c6ca5ee16222504edd4b9d8d6356bc
Mirror scripts sometimes leave corrupted/partial files behind.
Problems
========
1) wget is called with the -O flag, and the server returns an HTTP
error for the requested URL (404 etc). Wget leaves a zero-length file
behind. This doesn't seem to happen without the -O flag.
2) wget starts the download which stalls & times out half-way; wget
gives up and requests the same file with a byte offset of the form
"Range: bytes=1234-", and the web server doesn't support open-ended
ranges. In this case wget prints out a warning, leaves a partial file
behind and returns success.
3) Sites like GitHub generate repo tarballs on the fly, eg:
https://github.com/kubernetes/kubernetes/archive/refs/tags/v1.19.3.tar.gz
Since tags can move, downloading such a file twice may result in a
different file. Therefore HTTP "resume download" may corrupt files in
this case.
4) Git "keyword expansion" feature may result in differences in source
files being downloaded. For example, this file:
https://github.com/kubernetes/kubernetes/blob/v1.19.3/staging/src/k8s.io/component-base/version/base.go
contains lines similar to:
gitVersion = "v0.0.0-master+$Format:%h$"
where %h is replaced with a short SHA when the tar file is
exported/downloaded. How short the SHA is depends on git history and
sometimes results in shortened SHAs of different lengths. So
downloading that file may result in different files.
Therefore HTTP "Range" header may corrupt files in this case as
well.
5) Curl is invoked with the "--retry" option and starts the download;
connection stalls; curl gives up, connects again, skips the 1st N
bytes and appends to the partial file. If the file changes while we
are doing this, it will end up corrupting the file. This is very
unlikely to happen and I haven't been able to reproduce this case.
Problems with HTTP Range header
===============================
Curl/wget "resume/continue download" feature has no way of verifying
whether the partial file on disk, and the one being re-requested, are in
fact the same file. If the file changes on the server between
downloads, "resume download" will corrupt it.
Some web servers don't support this at all, which triggers case (2)
with wget.
Some web servers support the Range header, but require that the end
byte position is present. This is not compatible with wget & curl.
For example curl & wget add headers similar to: "Range: bytes=1234-"
means give me the file starting at offset 1234 and till EOF. This also
triggers case (2).
This patch
==========
* Always download the file to a temporary name, then rename into place
* Use curl instead wget (better error handling). The only exception is
"recursive downloads", which curl doesn't support.
Bug: https://bugs.launchpad.net/starlingx/+bug/1950017
Change-Id: Iaa89009ce23efe5b73ecb8163556ce6db932028b
Signed-off-by: Davlet Panech <davlet.panech@windriver.com>
With the plugin yum-plugin-priorities now it's possible
to add a priority for each repo through --repo-priority
on command line and cfg file as well.
This is need because of the usage of ceph mirror and to force the
images to use ceph packages on that repo.
Test plan:
Docker images build succeeded.
Closes-Bug: #1949518
Signed-off-by: Delfino Curado <delfinogomes.curadofilho@windriver.com>
Change-Id: I3e2e4df3d75f290789188875b089eab4ca49bbb0
This commit removes the iptables RPMs from the distro and flock layers,
because iptables is now built by the distro layer due to commit
I63d557112c653d59b88ac3a4798dee0e89246612.
Verification: A layered build is successful with this change.
Partial-Bug: #1949217
Depends-On: I63d557112c653d59b88ac3a4798dee0e89246612
Signed-off-by: M. Vefa Bicakci <vefa.bicakci@windriver.com>
Change-Id: I5275d1aefcf603d79aac7dcb2e3d97cae9e7d5d2
This commit adds iptables package's to the distro layers, because
iptables will be built by the distro layer due to commit
I63d557112c653d59b88ac3a4798dee0e89246612.
Verification: A layered build is successful with this change.
Partial-Bug: #1949217
Signed-off-by: M. Vefa Bicakci <vefa.bicakci@windriver.com>
Change-Id: If95c2d24c98cb2add5e24548bc45f505c94b4b79
According to the layered build referense at
https://docs.starlingx.io/developer_resources/Layered_Build.html
all run-time dependencies of packages in all layers need to be included
in the flock layer's rpms_centos.lst. Quoting from the linked guide:
If the package will be installed to iso, the package’s ‘Requires’ as
well as the transitive Requires of those Requires, should be added to
a lst file under stx-tools/centos-mirror-tools/config/<os>/flock. Yes
I said ‘flock, and not <layer>, because the ISO is build from the
flock layer.
This commit adds the missing runtime dependencies of ceph-14 to ensure
that "build-iso" does not fail with the following errors:
Warning: Infinite loop detected in dependency resolution. \
See .../loadbuild/.../flock/export/deps.txt for details -- exiting
These RPMS had problems (likely version conflicts)
librabbitmq.so.4()(64bit)
librdkafka.so.1()(64bit)
libstoragemgmt
Could not install dependencies
Verification: Layered build and build-iso are successful.
Partial-Bug: #1949112
Signed-off-by: M. Vefa Bicakci <vefa.bicakci@windriver.com>
Change-Id: I7c061b1f219061ad8d34e3ea9b5aafe237476440
ceph now needs sudo, sudo needs vim-minimal not vim-enhanced.
Partial-Bug: 1949112
Signed-off-by: Scott Little <scott.little@windriver.com>
Change-Id: I56ef778a8403d22d673426cd231fc3092b5268aa
This commit adds a build dependency (iptables-devel-1.4.21) for the
iproute package to the distro layer's package list. iproute is
uprevisioned in the distro layer to support IPv6 segment routing, with
change I6de9659dfec830f954661a0b0f82e69dc9637a5d.
In addition, given that iproute will be uprevisioned, this commit also
removes iproute from the distro and flock layers' pre-requisite/chroot
package lists.
Verification:
- Updated iproute package was confirmed to build as expected with this
commit in a monolithic StarlingX build environment.
- In a layered StarlingX build environment, the distro layer and the
flock layer were successfully built with this commit and the
aforementioned iproute uprevisioning companion commit, and an ISO
image was generated.
Story: 2008921
Task: 43663
Signed-off-by: M. Vefa Bicakci <vefa.bicakci@windriver.com>
Change-Id: I5e272dc59b8b69611474706c165644a8dd5d7f52
Scott Little