Commit Graph

27 Commits

Author SHA1 Message Date
Hugo Brito da7b5b99bb Fix tox constraints and bandit
This changes the upper-constraints to point to our internal,
since the OpenStack Stein file doesn't exist anymore. Also,
a small fix detected by the Bandit tool.

Test Plan:
- Execute: tox -e bandit

Closes-bug: 2033294

Change-Id: I8735bcd2ad23f4269d83a3ffc880a8588e72b5a3
Signed-off-by: Hugo Brito <hugo.brito@windriver.com>
2024-01-22 15:07:43 -03:00
Li Zhu 4b876fa82c Decontainerizing rvmc.py and Modular Integration in DC Repo
Make the following changes for decontainerizing rvmc.py:
1. Copy rvmc.py from the Metal repo to the DC repo. Convert it into a
   common module that can be imported either from an independent script
   (rvmc_install.py), which is invoked from the Ansible installation
   playbook, or from other DC Python modules, such as the
   subcloud_install module. This module is used to shut down the host
   when subcloud deployment is aborted.
2. Create a Redfish Debian package called python3-redfish to install
   the Redfish Python library for the rvmc module.
3. Install rvmc_install.py to /usr/local/bin using the
   distributedcloud-dccommon package.

Make the following changes in the rvmc module/script:
1. Create LoggingUtil and ExitHandler classes to differentiate the way
   of logging and exit handling when called from the module or an
   independent script.
2. Implement HTTP request logging when the debug log level is set to 0.
3. Introduce a timeout mechanism to restrict the total execution time
   of the script during installation.
4. Add a signal handler to ensure the script exits properly when
   aborting a subcloud installation. Implement process ID tracking to
   allow only one RVMC process to run at a time for a subcloud
   installation, regardless of the termination status of previous
   installations.
5. Incorporate a power-off function to be called from the DC
   subcloud_install module, which is used in "subcloud deploy abort."
6. Include "subcloud_name" and "config_file" parameters/arguments to
   enhance functionality and flexibility.

Test Plan:

PASS: Verify successful subcloud installation.
PASS: Verify successful "subcloud deploy abort/resume".
PASS: Verify the playbook execution is halted, and the subcloud is
      shut down when an abort is triggered during subcloud installation
PASS: Verify the successful generation of the Redfish Debian package
      using the 'build-pkgs' command.
PASS: Verify that the image is successfully generated, along with
      the required Redfish package and its dependencies,
      using the 'build-image' command.
PASS: Verify that 'rvmc_install.py' is located in '/usr/local/bin'
      after the successful installation of the system controller with
      the new image.
PASS: Verify HTTP request and response logging are generated regardless
      of debug log level for failure conditions.
PASS: Verify the retry attempts to be applied only when the response
      status code is 5XX.
PASS: Verify that the subcloud installation command fails with
      the expected error message when the value of 'rvmc_debug_level'
      is set outside of the range [0..4].
PASS: Verify the success of the subcloud installation command by setting
      the 'rvmc_debug_level' value within the range [0..4].
PASS: Verify that the RVMC script exits when the timeout is reached.
PASS: Verify that only one RVMC process is running at a time for
      a subcloud installation, regardless of whether the previous
      subcloud installation terminated properly or not.

Story: 2010144
Task: 48897

Depends-On: https://review.opendev.org/c/starlingx/tools/+/900206

Change-Id: Ic35da91a35594ede668cef6875c752b93a19d6d5
Signed-off-by: lzhu1 <li.zhu@windriver.com>
2023-11-10 18:18:23 -05:00
Hugo Brito 80e07ef733 Update tox configuration for Distributed Cloud
Cleanup the tox configuration and requirements.

Test Plan:
Verify that all Tox tests pass without errors.
1. Successfully executed on the distributedcloud dir
- tox -e docs,releasenotes,api-ref,bandit,linters
- tox -e newnote -- new-note
2. Successfully executed on the distributedcloud/distributedcloud dir
- tox -e py39,pylint,pep8

Partial-Bug: 2033294

Change-Id: Ie7b2acb03c379708e8ef57b52d3c83560414f007
Signed-off-by: Hugo Brito <hugo.brito@windriver.com>
2023-08-29 17:29:17 -03:00
Al Bailey 4b05f5dcc0 Update tox.ini to work with tox 4
Tox 4 deprecated whitelist_externals.
Replace whitelist_externals with allowlist_externals

Partial-Bug: #2000399

Signed-off-by: Al Bailey <al.bailey@windriver.com>
Change-Id: I128a047f569fc3e07fdb0b78e6b57a074c056e15
2022-12-26 18:27:41 +00:00
Al Bailey ede5de8f33 Fix failing zuul jobs due to tox showconfig
As part of how we nest the tox.ini files in STX,
the environments in the sub directories need to also
be defined in the tox.ini in the base file.

I suspect the tox_extra_args change here may have exposed
the problem in some of our tox files.
https://review.opendev.org/c/zuul/zuul-jobs/+/866935

Partial-Bug: #1997255

Signed-off-by: Al Bailey <al.bailey@windriver.com>
Change-Id: Ia963897be5f844599d2f6145f472f5eac1df6805
2022-12-09 22:46:36 +00:00
Al Bailey d8b1a08d1e Fix openstack-tox jobs for DC repo
Bindep updates are required to get cffi to install on
ubuntu-jammy.

This update adds a bindep.txt to ensure Zuul will install
the required third party entries

The Zuul upper-constraints env variable declaration needed
to be added to tox.ini otherwise an older constraints
was being used which does not work with newer
versions of python.

Partial-Bug: #1997255

Signed-off-by: Al Bailey <al.bailey@windriver.com>
Change-Id: I7e4b032f9c91bfaab33fec49f23613004101f44b
2022-11-21 16:58:49 +00:00
Bart Wensley 527a48513a Turn off the legacy pip resolver for distcloud
The requirements that had conflicts have been updated.
The local upper-constraints is no longer being used by tox
since two constraints files with different values for the same
requirements are not supported. To enable this change, some
unnecessary code that required an older version of
oslo.messaging is removed.

Bugbear is also being disabled due to brokeness that can cause
test failures.

Change-Id: If007b7e0ad3c04b78b619542162267cc7d502460
Story: 2004515
Task: 41670
Signed-off-by: Bart Wensley <barton.wensley@windriver.com>
2021-01-22 16:00:28 -06:00
albailey 9f0c5426cc Fix tox failures from new resolver in pip
New pip was released on Dec 3, 2020 which breaks downloads
from pypi for some URLs.

New virtualenv was released Dec 7 which includes the new pip.

The new pip uses a different resolver which fails with many
of the components installed by pip.  These components
may need to be fixed by later commits, since the legacy
resolver is being removed completely in early 2021.

Change-Id: Ifcbd8965e2ef0b53dae0c878c2a0f2bd91b572c2
Closes-Bug: 1907125
Signed-off-by: albailey <Al.Bailey@windriver.com>
2020-12-09 08:03:19 -06:00
Sharath Kumar K c1da0762d5 Tox and Zuul job for the bandit code scan in stx/distributedcloud
Setting up the bandit tool for the scanning of HIGH severity issues
in the python codes under Starlingx/distributedcloud folder.
Expecting this merge will enable zuul job for CI/CD of bandit scan.

Configuration files:
1. tox.ini for adding bandit environment and command.
2. test-requirements.txt for adding bandit version.
3. .zuul.yaml file for adding bandit job and configuring under
   check job to run code scan every time before code commit.

Test:
Run tox -e bandit command inside the fault folder to validate the
bandit scan and result.

Please note:
Changes will be implemented in batches and  this is Batch6 change.

Story: 2007541
Task: 39648
Depends-On: https://review.opendev.org/#/c/721294/

Change-Id: I6fdfeacdb000faafc73c44eef352bf2abd38a9a2
Signed-off-by: Sharath Kumar K <sharath.kumar@intel.com>
2020-06-16 04:02:59 +00:00
Al Bailey 501fa35af2 Update pylint for distributedcloud
The pylint that was specified as an upper constraint was not
as strict as the newer versions.

- Fixed or suppressed any new error codes being reported.
Suppressed error codes should be fixed by additional reviews.

- added a zuul target for pylint

- Added dcdbsync to the folders being processed by pylint.

- Removed the env:UPPER_CONSTRAINTS_FILE from tox.ini so that zuul
does not override the upper constraints with an newer version.

- Needed to add greenlet to the extension-pkg-whitelist since
runtime introspection of that component is not possible.

- oslo-messaging get_transport deprecated 'aliases' in pike, and
removed it in rocky. The packaged version in STX is 5.30.6
which still supports aliases. This requires us to explicitly
run tox with the version that we ship (pre-stein). This is
controlled by a local upper-constraints.txt file.
Related-Bug: 1865054

- Paste file handling reports a pylint issue which has been suppressed
Related-Bug: 1865085

Story: 2004515
Task: 38882
Change-Id: Ie7c8c2adab1eeb9100159a2aa29968a0b557e2e4
Signed-off-by: Al Bailey <Al.Bailey@windriver.com>
2020-02-27 15:02:43 -06:00
Al Bailey bde6982ded Updating flake8 version for distributed cloud
This also fixes the linters target which fails when yamllint
does not find any inputs.

Most new failures are being suppressed for now, and will be
cleaned up by future submissions.

Story: 2004515
Task: 38739
Change-Id: I8bddba83408be19a487837e326c86432579c50db
Signed-off-by: Al Bailey <Al.Bailey@windriver.com>
2020-02-26 20:09:38 +00:00
Don Penney 657df7be76 Update files for relocated repo
Change-Id: I92e73dda13d1135a1ecc60007c8c1447bb01ad7a
Story: 2006166
Task: 37337
Signed-off-by: Don Penney <don.penney@windriver.com>
2019-11-04 14:36:51 -05:00
Don Penney 9531b76af1 Move content to subdir to support relocated packaging
Change-Id: Id4f645de23b9f0aec1914edbaef11cdb6e6dc0af
Story: 2006166
Task: 37337
Depends-On: https://review.opendev.org/692861
Signed-off-by: Don Penney <don.penney@windriver.com>
2019-11-04 13:57:02 -05:00
Scott Little 2c538b9e52 tox.ini changes to reflect new location of tsconfig
Story: 2006166
Task: 35687
Depends-On: I6c62895f8dda5b8dc4ff56680c73c49f3f3d7935
Change-Id: I9ac723963b51f76168b76de2caf366e5678c765d
Signed-off-by: Scott Little <scott.little@windriver.com>
2019-09-05 14:52:14 -04:00
Don Penney ff13bcf60a Update tox.ini files to use stein constraints
This updates the upper-constraints.txt used by tox for
installing python modules to reference the stable/stein
constraints.

Change-Id: I5029186ae1ccee719da17f6a36665f81e207f882
Partial-Bug: 1834216
Signed-off-by: Don Penney <don.penney@windriver.com>
2019-06-25 13:00:50 -04:00
Dean Troyer 042d172876 Followup opendev cleanup and test jobs
Change-Id: Ide29dfb0f650b5e53b43a9e06413c3c68dad9b3a
Signed-off-by: Dean Troyer <dtroyer@gmail.com>
2019-04-21 15:44:19 -05:00
Kristine Bujold 9b3af14ec5 Remove wrs-configutilities SDK Module
Some of the APIs were moved to controllerconfig.

With the StarlingX move to supporting pure upstream OpenStack, the
majority of the SDK Modules are related to functionality no longer
supported. The remaining SDK Modules will be moved to StarlingX
documentation.

Story: 2005275
Task: 30265
Depends-On: https://review.openstack.org/#/c/649175

Change-Id: I2044761b3e653b6862ece2cf82dd8135b059b60d
Signed-off-by: Kristine Bujold <kristine.bujold@windriver.com>
2019-04-01 16:08:51 -04:00
Zuul 330cac2691 Merge "Fix coverage reporting for stx-distcloud" 2018-11-23 19:29:55 +00:00
98k 9a976a61c3 fix tox python3 overrides
We want to default to running all tox environments under python 3, so
set the basepython value in each environment.

We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.

We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.

Change-Id: I24b87db5bd80876ff631698350606aa4e23919ee
Closes-Bug:  #1802032
Signed-off-by:  98k <18552437190@163.com>
2018-11-21 02:22:44 +00:00
Al Bailey 83c91fb67f Fix coverage reporting for stx-distcloud
tox -e cover  was not working, and now it is

Currently coverage is 28% based on py27 unit tests

Story: 2003862
Task: 27992
Change-Id: I1a2c3371dfb02d95181306d15012f5c752f91513
Signed-off-by: Al Bailey <Al.Bailey@windriver.com>
2018-11-19 10:08:13 -06:00
Lachlan Plant e815932fdb Enable tox tests for stx-distcloud
py27 tests fixed and passing
py27 added to gate and check
pep8 added to gate and check

Story: 2003862
Task: 26691

Change-Id: I0469978534f9b50d33c65484f0b4c07b3314cb67
Signed-off-by: Lachlan Plant <lachlan.plant@windriver.com>
2018-11-02 10:27:02 -05:00
Dean Troyer d020d8270e Docs need to build without upper-constraaints
We are using openstackdocstheme >= 1.26.0 which is well beyong
what is in stable/pike upper-constraints.txt.

Change-Id: I2285d7dab0f82ce3a875cff7a8e6612a9c47cf6a
Signed-off-by: Dean Troyer <dtroyer@gmail.com>
2018-10-22 16:22:24 -05:00
Abraham Arce 0c19a79fbd [Doc] OpenStack API Reference Guide
Baseline changes to comply with OpenStack API documentation
from OpenStack Documentation Contributor Guide [0]:

- [1] How to document your OpenStack API service

[0] https://docs.openstack.org/doc-contrib-guide
[1] https://docs.openstack.org/doc-contrib-guide/api-guides.html

Story: 2002712
Task: 26621

Change-Id: Ie2bc6b177a8cdff1c844696b89f954cead76e830
Signed-off-by: Abraham Arce <abraham.arce.moreno@intel.com>
2018-09-25 08:18:08 -07:00
Abraham Arce 09599bb049 [Doc] Release Notes Management
Baseline changes to comply with Release Notes Management
based in Reno [0]:

* Use build-openstack-releasenotes job for release notes
  (We can't use the OpenStack releasenotes template as it includes
  publish jobs, stx needs its own)
* Add newnote tox environment as convenience for creating new release
  notes, re-using the releasenotes venv.
* Remove the unused 'true' tox environment.

[0] https://docs.openstack.org/reno/latest/

Story: 2003101
Task: 26620

Change-Id: Ie0eac726983f8cdfb6bade20a2b5445559a962e0
Signed-off-by: Abraham Arce <abraham.arce.moreno@intel.com>
2018-09-25 08:17:47 -07:00
Abraham Arce e536aba329 [Doc] Building docs following Docs Contrib Guide
Baseline changes to comply with OpenStack Documentation
Contributor Guide [0] starting with the following sections:

- Project guide setup
  - [1] sphinx-quickstart
  - [2] doc/source/ layout
- Building documentation
  - [3] tox -e docs
- Using documentation tools
  - [4] openstackdocstheme
- Use build-openstack-docs-pti job template for docs

[0] https://docs.openstack.org/doc-contrib-guide
[1] http://www.sphinx-doc.org/en/master/usage/quickstart.html
[2] https://docs.openstack.org/doc-contrib-guide/project-guides.html
[3] https://docs.openstack.org/doc-contrib-guide/docs-builds.html
[4] https://docs.openstack.org/openstackdocstheme/

Story: 2002708
Task: 26618

Story: 2002813
Task: 26619

Change-Id: Id37ae25bf0e6bfef594da97d0a2f3747934316f0
Signed-off-by: Abraham Arce <abraham.arce.moreno@intel.com>
2018-09-25 08:16:59 -07:00
Dean Troyer be7da964fd Set up initial Gerrit requirements
* Add .gitreview
* Add .zuul.yaml with basic linters and pep8 jobs
* Fix up tox.ini to run linters and pep8 environments.  This
  changed some leagacy config in the default testenv that will
  affect the more specific environments.

Change-Id: Ieca279a240b8705dcb73d9d195e1658a40e199e1
Signed-off-by: Dean Troyer <dtroyer@gmail.com>
2018-09-06 15:38:41 -05:00
Scott Little e82c7b4336 StarlingX open source release updates
Signed-off-by: Scott Little <scott.little@windriver.com>
2018-08-07 11:51:21 -04:00