The constraints file used for tox.ini was removed. We need to
update the file to use the StarlingX Debian constraints file.
Test Plan:
PASS - Run tox command
Closes-bug: 2055734
Change-Id: I306be11f6edc4538cbb3f7a164bac9e1ad08501f
Signed-off-by: Hugo Brito <hugo.brito@windriver.com>
This commit updates patch extraction process to read and untar
specific files. Additionally, it updates the process used to
extract sw_version from deeply nested tar archives of a patch
file.
Test Plan:
[PASS] Ran below command in DC env that invokes this code path
sw-patch --os-region-name SystemController upload <patch>.patch
[PASS] Uploaded an in-service and RR patch
Story: 2010993
Task: 49289
Change-Id: I7ada7b55f458c50ed3bf51e66841cc49592f2f71
Signed-off-by: Jessica Castelino <jessica.castelino@windriver.com>
This change will allow this repo to pass zuul now
that this has merged:
https://review.opendev.org/c/zuul/zuul-jobs/+/866943
Tox 4 deprecated whitelist_externals.
Replace whitelist_externals with allowlist_externals
Remove the deprecated cgcs-patch(CentOS) jobs from zuul
sw-patch directory contains the Debian support.
Move from xenial to bionic for patch-alarm zuul job.
Partial-Bug: #2000399
Signed-off-by: Al Bailey <al.bailey@windriver.com>
Change-Id: Ia0dcf61fc4a2c7911434158dd2e8bca7a787249a
Details: For feature parity, the upgrade-start-pkg-extract has
to be updated to support upgrading to Debian 22.12 and
CentOS 22.12.
Preview change in this CR
https://review.opendev.org/c/starlingx/update/+/850906
only runs on Debian to Debian upgrade.
This change will allow the package extraction running
on CentOS to Debian upgrade.
Test Plan:
PASS: manually tested the script on Debian
PASS: manually tested the script on CentOS
Task: 46269
Story: 2009303
Signed-off-by: Junfeng (Shawn) Li <junfeng.li@windriver.com>
Change-Id: I05f9e631b0bbddcad3e9ea21000ff7f283b2f097
The sysinv method is no longer valid.
The code needs to be updated to accomodate the new
signature.
Test Plan:
PASS: upload/apply a patch
PASS: Apply a patch using NFV
Closes-Bug: #1983504
Signed-off-by: Al Bailey <al.bailey@windriver.com>
Change-Id: I12a8defabc64155dd085f028b12c2d67bb52d99d
lxml library tostring() accepts only one argument
instead of 2 like it was before. This commit removes
the second argument.
Closes-Bug: 1977869
Signed-off-by: Luis Sampaio <luis.sampaio@windriver.com>
Change-Id: I9bb43a758bb660c8ab6edbf54984ff993b7f8598
The original cgcs-patch is rpm based which requires a
complete re-write to work on ostree/dpkg systems like Debian.
The code has been forked, since the older Centos env and
python2.7 are end-of-life.
Forking the code allows all new development to not
require re-testing on Centos.
The debian folder under cgcs-patch has been moved
under sw-patch
Renaming and refactoring will be done in later commits.
pylint is un-clamped in order to work on python3.9
Some minor pylint suppressions have been added.
Test Plan:
Verify that this builds on Debian
Verify that the ISO installs the new content on Debian without
breaking packages that import cgcs_patch.
Verify patching service runs on Debian
Co-Authored-By: Jessica Castelino <jessica.castelino@windriver.com>
Story: 2009101
Task: 43076
Signed-off-by: Al Bailey <al.bailey@windriver.com>
Change-Id: I3f1bca749404053bae63d4bcc9fb2477cf909fcd
Ensure that the services needed for cgcs-patch are
enabled when the package is installed.
Story: 2009101
Task: 43076
Test Plan
PASS Build and Test ISO
PASS Check for /etc/systemd/system-preset/00-cgcs-patch.preset
Signed-off-by: Chuck Short <charles.short@windriver.com>
Change-Id: Ie74d9925b66f767d623ca0c8ec00081fe63a3a8f
Lintian complains that bash-completion scripts are installed in the
wrong place. Install them in the right place.
Story: 2009101
Task: 43076
Signed-off-by: Charles Short <charles.short@windriver.com>
Change-Id: I0e6afbcbd9b2d59da025667156ccd0a53b6e3b4b
During upgrade-start we call upgrade-start-pkg-extract to extract the
kickstarts and pxeboot data from the N+1 load. This data is used to boot
controller-1 in duplex environments. As these packages can be patched we
need to select the latest version of the package.
Currently the dnf repoquery call returns every version of the rpm
queried. This results in the base version being used during the package
extraction. This commit updates the command to use --latest-limit=1.
The command is also updated to use --disablerepo=*. This will result in
the command being restricted to the specified N+1 repos. Without the
disablerepo option the N repo packages are included in the results.
This brings the call in line with the behavior of
utilities/utilities/platform-util/scripts/gen-bootloader-iso.sh
Testing:
AIO-DX upgrade with patched kickstart package
Closes-Bug: 1955410
Change-Id: Ia1cd778791b64133667327031305d0f1914aed2d
Signed-off-by: David Sullivan <david.sullivan@windriver.com>
The previous bindep fix was only for centos, so zuul
workers running ubuntu (dpkg) would still not work.
Updating the file to handle those nodes.
Updating a python file to ensure pylint target is executed
The sitepackages for the cgcs-patch pylint tox target
also needs to be set to True to correspond to the bindep
values.
Story: 2008943
Task: 44183
Signed-off-by: albailey <Al.Bailey@windriver.com>
Change-Id: I2a9e630aa26c2823ccdc6a361c46575b58a1c39c
This change is to enable the upgrade to next release, which will
support OSTree. In the next release, /www and /pxeboot will be
moved to /var/www and /var/pxeboot respectively. During upgrade
to next release, rpms from next release will need to be extracted
into current release structure (/var and /pxeboot).
TCs:
Passed: upgrade to build with /www and /pxeboot moved under /var.
Change-Id: Id0dec2dee89dcad04c24b12a7a6072d03078f65e
Story: 2009101
Task: 43539
Signed-off-by: Bin Qian <bin.qian@windriver.com>
- Add blacklist for lintian-overrides.
- Fix debian/changelog, it assseumed it was still native source
format.
- Update debian version in meta_data.yaml
- Fix typo in debian/control
Story: 2009101
Task: 43076
Signed-off-by: Charles Short <charles.short@windriver.com>
Change-Id: Idda262ade8c801c3bbafa020e37b596a8378f541
Due to a recent change in fm-api's directory structure, unit tests would
fail since the virtualenv would not be able to find fm-api/setup.py.
Adjust the tox.ini to point to the correct directory. Tested locally
by running tox.ini.
Depends-On: https://review.opendev.org/c/starlingx/fault/+/806046
Story: 2009101
Task: 43091
Signed-off-by: Charles Short <charles.short@windriver.com>
Change-Id: If55a6d92dc861b23516637fcb90c52852c3fd92a
A lot of work has gone into making sure that StarlingX is python3
compatible. To ensure future compatibility, enable the python3
portability checks. Disable the checks that are raising errors.
Another set of commits will address the offending code.
Add following suppress warnings in cgcs-patch/pylint.rc:
- W1618: no-absolute-import
- W1619: old-division
- W1630: cmp-method
Add following suppress warnings in patch-alarm/pylint.rc:
- W1618: no-absolute-import
Story: 2006796
Task: 43198
Signed-off-by: Ricardo Alvim <Ricardo.AlvimNetto@windriver.com>
Change-Id: I0cbe384a72792cf123976f0de2020ae6f3fcd208
This commit fixes an issue seen during a k8s upgrade from 1.18.1
to 1.19.13. It was noticed that after upgrading kubelet to 1.19.13,
the sw-patch-controller process would continually restart.
It was found via packet tracing and logging that traffic from the
management interface to the localhost address at port 5489 was being
blocked. This indicated a likely issue in iptables.
Comparing the iptables rules in 1.18.1 to 1.19.13 shows the reason
why:
Chain KUBE-FIREWALL (2 references)
target prot opt source destination
DROP all -- !loopback/8 loopback/8 \
! ctstate RELATED,ESTABLISHED,DNAT
That is, drop all packets _not_ from the loopback interface _to_
the loopback interface that do not have an existing connection
state.
It was found that this rule was added in the following commit:
https://github.com/kubernetes/kubernetes/pull/91569/files
Which was added to address the security concern identified here:
https://github.com/kubernetes/kubernetes/issues/90259
It appears that the PatchMessageHelloAgent periodically sends
messages to both the patch controller's agent address as well
as to the localhost address. Since the outgoing socket used
for all messages is explicitly bound to the management
address, the traffic to the localhost address will hit the
drop rule noted above.
The solution in this commit is to not explicitly bind the
outgoing socket to the management address, so as to have the
kernel choosed the correct outgoing interface for both
messages.
Story: 2008972
Task: 43244
Testing:
AIO-SX (unicast traffic), AIO-DX, Standard (multicast traffic).
Ensure sw-patch-controller stays up after k8s upgrade.
Install a patch on all nodes.
Signed-off-by: Steven Webster <steven.webster@windriver.com>
Change-Id: I93912b934986dc28196c9ba50f2803bf0fe01513
Enable python3.9 in tox.ini and zuul gate. Tested locally
by running tox and running in the zuul gate.
Story: 2009101
Task: 43105
Signed-off-by: Charles Short <charles.short@windriver.com>
Change-Id: I44eaeb134d6b7b54788469fa469b04674aeb90c0
The load-delete function calls sw-patch del-release function
that tries to delete a 'version' key from a dictionary that
does not exists.
This dictionary is populated looping the folders inside
/www/pages/feed. Each folder is a version imported.
After the upgrade the old version folder is deleted,
the version is not include at the dictionary.
The solution was to verify if this key exists before deleting.
The following tests
Tested load-delete action on a SX system after load-import
Tested load-delete action on a SX system after upgrade-complete
Closes-Bug: 1940302
Signed-off-by: João Pedro Alexandroni Cordova de Sousa <JoaoPedroAlexandroni.CordovadeSouza@windriver.com>
Change-Id: I83f8d144edd53523a98402fbee71dce1507fc79c
If a second load is imported with different
pxe-network-installer/platform-kickstarts packages, the
upgrade-start-pkg-extract script will fail preventing the upgrade-start
command from completing.
To address this we clean the dnf cache in the upgrade-start-pkg-extract
script.
Closes-Bug: 1938304
Signed-off-by: David Sullivan <david.sullivan@windriver.com>
Change-Id: I1299619ffb37f6dbaef0a6df34f73397ef528508
Added pycryptodomex as depdendency
needed for packaging cgcs-patch.
When running tox on patch-alarm with py36 env,
it raises an exception, when trying to import
Cryptodome.
There is also a workaround here:
https://review.opendev.org/c/starlingx/config/+/800099
Story: 2008454
Task: 42768
Signed-off-by: Daniel Safta <daniel.safta@windriver.com>
Change-Id: Ic97d7779e601026c362110c706168f328dcd65be
(cherry picked from commit 614e73687e)
Updated the library that we use for
xml manipulation to lxml.
Implemented __le__ to compare
two PackageVersion objects.
Switched to Cryptodome instead of Crypto
and updated requirements.txt
Replaced the old dict.keys() with list(dict)
Use strict=False in configparser when py3
is used
Tested and working:
query, upload, apply
local and host-install,
delete, remove, query patch details
Story: 2008454
Task: 42768
Signed-off-by: Daniel Safta <daniel.safta@windriver.com>
Change-Id: I3c68322603eaaf9a78d101b5b1198e9582497105
(cherry picked from commit 51524c09c7)
Added a new utility in cgcs_patch/patch_functions.py that
extracts only the metadata (and only to stdout) in order to
parse it and provide the software release version.
Signed-off-by: Jessica Castelino <jessica.castelino@windriver.com>
Partial-Bug: 1934674
Change-Id: I39cbc67c5ed8822fe2a21630eec380a43c09de95
The py2.7 jobs need to specify xenial
The py3.6 jobs need to specify bionic
The focal zuul nodes only have python 3.8 installed in them
Zuul targets that invoke a generic python3 interpreter such
as pep8 is not specified.
Also ignore H216 since we still use py2.7.
The copyright date was updated in order to trigger
the zuul jobs, as a no-delta type of change.
Partial-Bug: 1928978
Signed-off-by: Charles Short <charles.short@windriver.com>
Change-Id: I81fd790dfc8a665a4e4e0ff59a013af7921b6e06
Signed-off-by: Charles Short <charles.short@windriver.com>
The EXCLUDE_LIST_FROM_TAR list in cgcs-patch build_srpm.data includes
entries to exclude .tox and other dirs when building the package, but
the path included an extra directory level. This update corrects the
paths.
Change-Id: I8b3641e4e86f52ef7b9fb56f9eb3df289935e188
Closes-Bug: 1908940
Signed-off-by: Don Penney <don.penney@windriver.com>
If a cold reboot occurs in the middle of patch installation, the
system can be left in a state where the patch-agent is unable to
perform its operations properly. The RPM database can be left with
duplicate RPMs due to the incomplete transaction, which can in turn
lead to DNF update installation issues.
This update adds detection of duplicate RPMs to the patch-agent to
avoid attempting installation until the system is recovered.
Additionally, protection is added to the sw-patch init to treat
multiple reboot patch installations as an error, to avoid boot loops.
Closes-Bug: 1904928
Change-Id: Ia06a6f669c45398d7956f2ac2caa76c447bc1b16
Signed-off-by: Don Penney <don.penney@windriver.com>
pylint-2.6.0 introduced a new W0707 warning related to a new
capability in python3. Given cgcs-patch is still running py2, this new
warning is being ignored. When we move to python3, we will review the
pylint warnings at that time.
Change-Id: If59dcc62ce68d7bb7b119018502ba6d318e7746a
Closes-Bug: 1902916
Signed-off-by: Don Penney <don.penney@windriver.com>
The error handling for the patch release check in the setup_patch_repo
utility has an invalid format string. Rather than providing a helpful
error message, an exception occurs due to the format string.
Change-Id: I1a5445f190353e4f37d2a8d0844327942b93cdf3
Closes-Bug: 1900634
Signed-off-by: Don Penney <don.penney@windriver.com>
The parsing of the UNREMOVABLE tag in the patch XML is setting the
wrong metadata flag. It's setting "removable" instead of
"unremovable", meaning the UNREMOVABLE tag has no effect.
Change-Id: I6f497a8a76e1c46b33edf896e71ed1266004d964
Closes-Bug: 1891729
Signed-off-by: Don Penney <don.penney@windriver.com>
This update revises the cgcs-patch and patch-alarm package builds to
drop the old and unneeded custom build_srpm script, in order to use
the PKG_GITREVCOUNT feature for package versions.
Change-Id: If541f55340565cf73213aeac511e69487e53fa11
Story: 2006166
Task: 40137
Signed-off-by: Don Penney <don.penney@windriver.com>
When the patch-agent is notified by the patch-controller of a new
patching operation (patch_op_counter in HELLO is incremented), it will
run a new software query consisting first of running "dnf makecache",
then checking the software repositories for changes. In rare cases,
the metadata returned to dnf when it makes the query could be stale,
resulting in the patch-agent believing the current software is
up-to-date.
In order to protect against this, the patch-agent will now verify the
repository revision id for the updates repo. If the revision id has
not changed when it is reasonable to expect it could have (ie. the
patch_op_counter has increased, indicating the repository may have
been updated), the patch-agent will retry once to allow for any
potential caching to have cleared.
Change-Id: I3a44ed86e16cd9fe67f0b0e763c95a5a7e126cf8
Closes-Bug: 1884094
Signed-off-by: Don Penney <don.penney@windriver.com>
1. Rename TIS to StarlingX for .service file
Test:
After the de-brand change, bootimage.iso has been built in the flock
Layer and installed on the dev machine to validate the changes.
Please note, doing de-brand changes in batches, this is batch12 changes.
Story: 2006387
Task: 39650
Change-Id: I3a4836563836246f90e195bd3f5212af73420fbe
Signed-off-by: Sharath Kumar K <sharath.kumar@intel.com>
This commit replaces the use of "repoquery" in the
upgrade-start-pkg-extract utility with "dnf repoquery".
Story: 2007403
Task: 39059
Change-Id: I8bc86af11cc7adc85b6d5d30f3a7368581c7988c
Signed-off-by: Carmen Rata <carmen.rata@windriver.com>
To prevent the warning about a newer version of pip
available, the following env setting was added
PIP_DISABLE_PIP_VERSION_CHECK=1
To prevent sub-calls to tox from using the default pip
cache location, rather than whatever a user may override
using xdg env variables, the following has been added:
passenv =
XDG_CACHE_HOME
The python patch scripts are now validated by flake8.
All flake8 tox targets are triggered from a single zuul job.
Story: 2004515
Task: 38581
Change-Id: I105512edbd3da15480355e30c0dd14933584a47b
Signed-off-by: albailey <Al.Bailey@windriver.com>
Hugo Brito