This commit fix the deploy precheck <release> command output which
is returning one line with an blank "Error:" when the output is correct.
Test Plan:
PASS: Deploy precheck <release> returning without error.
PASS: Deploy precheck <release> returning expected error.
PASS: Deploy start failed with expected error.
PASS: Deploy start
Note: Exit code 3 was chosen based on exit codes with special meaning
which the 1 is for catchall general errors and 2 for misuse of shell
builtins. The number 3 is not allocated with special meaning so it
was chosen for unhealthy precheck.
Closes-Bug: 2056106
Change-Id: Ifed48157f7810eec2881d8a9e011eae8941f3427
Signed-off-by: Luis Eduardo Bonatti <LuizEduardo.Bonatti@windriver.com>
When using Keystone auth for software cli, only user with 'admin' role
is allowed to run any commands. When using software cli without 'sudo',
all software commands require user with 'admin' role.
This review also update the exception handling and error reporting.
Test Plan:
PASS: A Keystone user in the 'admin' project with 'admin' role should
be able to run ALL 'software' commands WITHOUT SUDO
PASS: A Keystone user in the 'admin' project with only 'member' and/or
'reader' role should NOT be able to run ANY 'software' commands
WITHOUT SUDO
Story: 2010676
Task: 49754
Change-Id: I46653021b1a82bccded5eb870dc0907cd5c2351b
Signed-off-by: Joseph Vazhappilly <joseph.vazhappillypaily@windriver.com>
There are additional steps needed to enable "deploy host"
operation for major release upgrades. This commit adds these
additional steps, which are:
1. Create TO release platform config directory, done during
upgrade-start on legacy upgrade procedure
2. Create TO release rabbitmq directory, done by upgrade
manifest after controller-1 reinstall on legacy upgrade
The commit also fixes some issues:
1. shell-utils logging functions logging nowhere after
being sourced by other scripts
2. sync-controllers-feed script was only syncing the
ostree_repo directory instead of the full feed content
3. major release deployment scripts ran from different
places, now all scripts are executed from the TO release
feed directory, or from checked out TO release ostree
repo in case of chroot
4. change umount command from chroot_mounts to lazy umount
Note: Since the "deploy host" endpoint for major release
deployment is not yet implemented, the test plan will have
test cases that simulate the "deploy host" operation.
Test Plan
PASS: simulate "deploy host" successfully for AIO-SX
PASS: simulate "deploy host" successfully for AIO-DX
Depends-on: https://review.opendev.org/c/starlingx/config/+/913715
Story: 2010676
Task: 49703
Change-Id: Ib6ae49b3590a1e50acb305ac7482e28bcc4de403
Signed-off-by: Heitor Matsui <heitorvieira.matsui@windriver.com>
Insert ostree commit id into metadata file after apt-ostree
runs. "previous_commit" tag is for commit id before apt-ostree
is executed and "commit" tag is for latest commit id after it
is executed.
Test Plan:
PASS: Run software deploy start for patch and check commit id
is inserted in metadata file
Story: 2010676
Task: 49753
Change-Id: I8fc3c33430188449d852770824e3ecd765583dd7
Signed-off-by: Charles Short <charles.short@windriver.com>
Signed-off-by: sshathee <shunmugam.shatheesh@windriver.com>
This commit is to fix 'software upload-dir' not having
respond that contains uploaded release info.
Test Plan:
PASS: upload files using 'software upload-dir'
Task: 49634
Story: 2010676
Change-Id: I635554fdbdb80fe31a38d1170202405fe6f32d3a
Signed-off-by: junfeng-li <junfeng.li@windriver.com>
This change adds checks before deleting software releases:
1. software release is available or unavailable
2. When it is on a system controller, the release is not being used by a
subcloud
This change also update the following:
1. removed the exception handling in controller level, moved to
exception hook
2. CLI code to display HTTP error, only handles 500 status code, by
displaying message from API, all other 4xx, 5xx status code display
HTTP error directly.
3. ensure CLI return 1 for unsuccessful requets (status code 500)
4. fixed some minor issues
Story: 2010676
Task: 49657
TCs:
passed: observe delection rejected because of release not found,
release is not in available or unavailable state.
passed: delete an available release
passed: on system controller, successfully delete scenarios
passed: (simulated) on system controller with subcloud, delete
release used by subcloud is rejected
Change-Id: I306b1d8604113b92d907384844e8e8107835a463
Signed-off-by: Bin Qian <bin.qian@windriver.com>
Currently upload-dir logic was uploading only the files from
the last directory passed as argument to the command, and it's
help text and output was divergent from the similar "software
upload" command.
This commit fixes the logic, allowing uploading from multiple
directories correctly, and fixes the help text and output to
align with "software upload".
Test Plan
PASS: show help text for "software upload-dir"
PASS: upload one directory containing patches
PASS: upload one directory containing iso + patches
PASS: upload multiple directories containing patches
PASS: upload multiple directories containing iso + patches
Story: 2010676
Task: 49693
Change-Id: I5886e8c5c55355e24ec471c4ae47e91ec3c84dfd
Signed-off-by: Heitor Matsui <heitorvieira.matsui@windriver.com>
When doing load-import operation, the /var/www/pages/feed/ files must
be replicated in both controllers. The files were being created in
controller-0, but they were not copied to controller-1.
This fix added a way to get all the folders from one controller to
another and sync them with ostree the explanation is in the following
commit: I42c274079631a3c197015e636e03de1bc96de28b
Test-plan:
PASS: After a load-import, the inactive controller should have the
feed repo created.
Closes-bug: 2045321
Change-Id: I260951461d2c19550e9f57ad7ab9ec66a25de5bb
Signed-off-by: Lindley Werner <Lindley.Vieira@windriver.com>
'software deploy host <in-service-patch-release>' should NOT fail if
there is no restart-script (post-deploy script).
If the rsync does not find the scripts, does not raise an exception.
Test-plan:
PASS: software deploy host <in-service-patch-release> successful
Closes-bug: 2058393
Change-Id: I1b8cac9e0401c3f64c7334139c62bf272e9aeb56
Signed-off-by: Lindley Werner <Lindley.Vieira@windriver.com>
This commit is to include a versioned deploy precheck script into
/opt/software/scripts/rel-<sw_version> to be able to run the correct
precheck code for a specific release.
Along with this commit, the precheck api is changed to use the new
location of the script, and the precheck script is changed to add
support to patch-only prechecks, and as a consequence, minor wording
changes were done to return more accurate messages to the user.
1. For the iso upload scenario:
The upload process will copy all scripts under
<iso_root>/upgrades/software-deploy to /opt/software/rel-<ver>/scripts
2. For the patch upload scenario:
The upload process will check if patch contains the deploy-precheck
script. If it does, then the script is copied to
/opt/software/rel-<ver>/scripts, if not then a symlink will be created
to the patch 'required patch' versioned precheck script.
Notes:
- iso (prepatched or not) will always come with deploy-precheck script
- <ver> assumes the format MM.mm.pp
Test Plan:
PASS: Upload multiples patches, both with and without precheck
scripts, and verify the versioned directories are created
and the precheck script is created as expected
PASS: Run deploy precheck for an iso release and verify the upgrade
precheck output is returned as expected
PASS: Run deploy precheck for a patch release and verify the patch
precheck output is returned as expected
Depends-on: https://review.opendev.org/c/starlingx/metal/+/911595
Story: 2010676
Task: 49263
Change-Id: I04ff89d43579fd71592f7ec534db57a1ead79483
Signed-off-by: Luis Eduardo Bonatti <LuizEduardo.Bonatti@windriver.com>
Co-signed-off-by: Heitor Matsui <heitorvieira.matsui@windriver.com>
This change add support for https with SSL protocol and certificate.
The USM client can work with either insecure (disable SSL/TLS
certificate verification) or with SSL certificate. The client is
also modified to support sessions and versions. These changes are
adapted from cgtsclient.
This adds three authorization modes, [token, keystone & local-root].
In token mode, a keystone token and software-url is used for auth.
Eg: $ software \
--software-url "http://192.168.204.1:5497" \
--os-auth-token "${TOKEN}" list
In keystone mode, sourced keystone configs in env is used for auth.
Eg: $ source /etc/platform/openrc; software list
In local-root mode, authorization is by privileged user (root/sudo)
of the controller where software application is running.
Eg: $ sudo software list
Optional arguments specific to https:
-k, --insecure
--cert-file CERT_FILE
--key-file KEY_FILE
--ca-file CA_FILE
Example usage for insecure connection:
software -k list
Story: 2010676
Task: 49666
Test Plan:
PASS: Verify software cli output for http endpoints
PASS: Verify software cli output for https endpoints
Change-Id: I2e2ff115b8d03cddb02e026da84f389918238dab
Signed-off-by: Joseph Vazhappilly <joseph.vazhappillypaily@windriver.com>
This commit is to allow active controller periodically sending deploy
state message to the software agent on its peer controller. The
interval is set to 30 seconds.
Test Plan:
PASS: build and deploy the iso
PASS: start new deployment, file is synced in both controllers
Task: 49655
Story: 2010676
Change-Id: Ie95c5a7d45b3d88331569ca52d64d40a4f39d6c3
Signed-off-by: junfeng-li <junfeng.li@windriver.com>
This commit fixes an issue that was preventing load import
script from copying the TO release pxe files correctly from
the load, and also gives pxe-update-*.sh script execution
permission to avoid permission denied errors during host-unlock.
Test Plan
PASS: upload TO release load, verify the TO release pxe.cfg.linux
files are copied to /var and that /etc/pxe-update script is
updated to 755
Story: 2010676
Task: 49698
Signed-off-by: Heitor Matsui <heitorvieira.matsui@windriver.com>
Change-Id: I222b484d3a28c603ed8d7c42d0405481086735f0
This commit add some changes on deploy host-list.
Adds a function to query the hostnames from sysinv
to deploy host-list entities during deploy start.
Changes endpoint to GET verb, the endpoint return in case
of no deployment in progress it will an empty list
and at CLI will print "No deploy in progress." In case
there is a deployment in progress the CLI will behave
the same but the endpoint will return the data below:
[{'hostname': '<hostname>',
'software_release': '<sw_version>',
'target_release': '<sw_version>',
'reboot_required': 'str<true/false>',
'host_state': '<host_deploy_state>'}]
This commit also changes the wait_for_install_complete function
to follow the new state logic.
Note: Software deploy host is affected by this change related
to states and will need a future commit regarding state changes
during deploy start and deploy host itself.
Test Plan:
PASS: Software deploy host-list with/without deployment in progress.
PASS: Deploy_host creation/update/get/delete.
PASS: Collect hostnames to deploy host entities during deploy start.
Story: 2010676
Task: 49586
Change-Id: I7b03df30fd8e326637a3ffc031e0fdf543cb6356
Signed-off-by: Luis Eduardo Bonatti <LuizEduardo.Bonatti@windriver.com>
This commit add some changes to deploy show endpoint, the name was
changed to just deploy with GET verb and also changes the deploy to
be saved as a list of dict to attend the API requirements. Now the
api accepts from_release and to_release as optional params, in case
it is provided the endpoint will return a dict otherwise will return
a list of dict.
Test Plan:
PASS: Create deploy
PASS: Update deploy
PASS: Software deploy start
PASS: Software deploy show
Story: 2010676
Task: 49645
Change-Id: I68d243c05da88c7eecf2d866c7202c3c7be51a2b
Signed-off-by: Luis Eduardo Bonatti <LuizEduardo.Bonatti@windriver.com>
This is to remove the unnecessary routing based on request
method in which the respond can't be decoded properly by
the sm-api
Test Plan:
PASS: run system host-swact
Task: 49661
Story: 2010676
Change-Id: Ife8862b7d5666de3a3dafff582dbb4c27e1adafa
Signed-off-by: junfeng-li <junfeng.li@windriver.com>
The constraints file used for tox.ini was removed. We need to
update the file to use the StarlingX Debian constraints file.
Test Plan:
PASS - Run tox command
Closes-bug: 2055734
Change-Id: I306be11f6edc4538cbb3f7a164bac9e1ad08501f
Signed-off-by: Hugo Brito <hugo.brito@windriver.com>
This change is to create 2nd thread to provide concurrent service. In a
different commit [1], the haproxy is to be configured to distribute the
slow requests to the 2nd thread, and the fast requests to the primiary
thread.
TCs:
passed: concurrent keystone requests of "software upload/
deploy precheck/deploy start" and "software list/deploy show/
deploy host-list"
passed: keystone authenticated "software deploy precheck"
request completed.
Story: 2010676
Task: 49647
[1] https://review.opendev.org/c/starlingx/stx-puppet/+/910644
Change-Id: I0e8e8ac1b5177f1bbf40e047335c075b0a471fc1
Signed-off-by: Bin Qian <bin.qian@windriver.com>
The usm_load_import script is currently being copied to two
different places needlessly during the upload process for a
major release: (/etc/software and /opt/software/rel-<ver>/bin).
This commit keeps only the copy to the versioned directory under
/opt/software/rel-<ver> and also changes the code to use the
script from this location accordingly.
Test Plan
PASS: run 'software upload' successfully
Story: 2010676
Task: 49624
Relates-to: https://review.opendev.org/c/starlingx/update/+/910027
Change-Id: I0178007721505ff8e2c2244964692b79e8aafea9
Signed-off-by: Heitor Matsui <heitorvieira.matsui@windriver.com>
This commit improves logging during deploy start by:
1. Creating a common module to be sourced by shell scripts to
load general-use functions, thus reducing code duplication
between the scripts
2. Replacing plain "echo" commands on the scripts by logging
functions present on the common module
3. Adding timestamps to the log messages
4. Centralizing all scripts logs into software.log, favouring
the troubleshooting, now that log lines contain timestamps
and the process/script that generated them
This commit also deletes the ostree_mounts.yaml file since
it would be used by apt-ostree integration, which was dropped.
Test Plan
PASS: run deploy start successfully and verify that deploy start
log messages are logged with the expected format
Story: 2010676
Task: 49607
Change-Id: I0bdebde8147faa5b29a642e35bfaf26e9862ed0a
Signed-off-by: Heitor Matsui <heitorvieira.matsui@windriver.com>
This commit improves logging during "deploy start" by:
1. Creating a function on upgrade_utils to configure logging
2. Replacing basic log configuration with the function from
previous item
3. Configuring software/utilities/migrate.py module to
effectively log in a file
4. Suppressing meaningless SQL commands output generated
during databases migration
Test Plan
PASS: run deploy start/upload, verify the logs from the
scripts are created as expected
Story: 2010676
Task: 49620
Depends-on: https://review.opendev.org/c/starlingx/update/+/910262
Signed-off-by: Heitor Matsui <heitorvieira.matsui@windriver.com>
Change-Id: Ib78bdcd1dd98b0f221985a429b6e9eeddeba837b
This commit create a generic method to be used when a deploy state
is changed. For every modification on the file it will call the message
class to send the deploy state dataset to peer controller.
Test Plan:
PASS: Deploy state change sent to peer controller on deploy start
Depends-on: https://review.opendev.org/c/starlingx/update/+/904367
Story: 2010676
Task: 49335
Change-Id: Idc06074f059c2f0d683b5f291e594f6a1555d6d5
Signed-off-by: Luis Eduardo Bonatti <LuizEduardo.Bonatti@windriver.com>
The deploy-precheck script is now stored in a versioned
directory under /opt/software/rel-<ver>, so there is no
need to check if the feed directory exists.
This commit removes deprecated variables and checks that are
not needed as result of the versioning of the release scripts
directory.
Test Plan
PASS: run "software deploy precheck" successfully
Story: 2010676
Task: 49633
Change-Id: I42b00143234f4aa5a3c871bbd6900f3bfce43c8d
Signed-off-by: Heitor Matsui <heitorvieira.matsui@windriver.com>
This commit changes the wording on software upload command
to improve the feedback to the user.
Test Plan
PASS: run "software upload -h" and verify the output
Story: 2010676
Task: 49589
Change-Id: I8df2e67ef20627e96399d797c998e00c38eab849
Signed-off-by: Heitor Matsui <heitorvieira.matsui@windriver.com>
This commit adds a new USM REST API endpoint that
provides the controllers deployment sync status during
the upgrade.
Depends-on: https://review.opendev.org/c/starlingx/update/+/904367
Test Plan:
PASS: sent request to get the response of the sync status
Task: 49426
Story: 2010676
Change-Id: Ib661f707686f13cbb52a5dbb67f636a9504ce8c9
Signed-off-by: junfeng-li <junfeng.li@windriver.com>
After major release data migration completes, update deploy state to
start-done for successful migration or start-failed if not.
Ths deploy state update from sub process is sent to software_controller
via udp message. Only the updates from a valid agent will be accepted.
story: 2010676
Task: 49590
TCs:
pass: Update deploy state to start-done after data migration
completes.
pass: Update deploy state to start-failed after data migration
fails.
Change-Id: If4bfdfea168374bd11185243cffc20aa3af9c160
Signed-off-by: Bin Qian <bin.qian@windriver.com>
This commit creates classes to handle the communication and sync
of deploy state between controllers and also add a operation
counter of deploy states.
This commit also synced with peer controller when deploy starts.
Test Plan:
PASS: Software.json of peer controller updated on deploy start.
PASS: Software.json of synced folder on peer controller matches
active controller on deploy start.
PASS: Software.json of peer controller updated with deploy state
change.
PASS: deploy state synced to peer in DX system when software deploy
start succeed.
Depends-on: https://review.opendev.org/c/starlingx/update/+/904362
Story: 2010676
Task: 49325
Change-Id: Id69b15e38402b5314657de963f5b69f164e2c351
Signed-off-by: Luis Eduardo Bonatti <LuizEduardo.Bonatti@windriver.com>
The apt-ostree command expect to receive only the name of the
package to install/delete from the package archive repository,
and due to [1] the packages are stored with name+version,
leading to apt-ostree failure.
This commit splits name from version from the package list so
that apt-ostree failures are fixed.
[1] https://review.opendev.org/c/starlingx/update/+/909046
Test Plan
PASS: install a patch release successfully
PASS: delete a patch release and verify the package is deleted
from apt-ostree package repository successfully
Story: 2010676
Task: TBD
Change-Id: Id7f8b8379aa0e870f1fa136cb2e629f020dde7f3
Signed-off-by: Heitor Matsui <heitorvieira.matsui@windriver.com>
When using 'software upload-dir' to upload files ,
this commit will only allow one iso file or one signature
to be uploaded in the directory. Multiple iso/signature
files in the directory to be uploaded won't be permitted
in order to prevent not enough disk space in /scratch
directory in the active controller and wrong signature
files being used.
Test Plan:
PASS: upload multiple iso files in the directory.
PASS: upload one iso file in the directory.
PASS: upload multiple sig files in the directory.
PASS: upload one sig file in the directory.
Task: 49576
Story: 2010676
Change-Id: Iba5af49ca50b4c836157027e140095a68ee5341b
Signed-off-by: junfeng-li <junfeng.li@windriver.com>