Commit Graph

130 Commits

Author SHA1 Message Date
OpenDev Sysadmins fb854f1f5f OpenDev Migration Patch
This commit was bulk generated and pushed by the OpenDev sysadmins
as a part of the Git hosting and code review systems migration
detailed in these mailing list posts:

http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.html
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004920.html

Attempts have been made to correct repository namespaces and
hostnames based on simple pattern matching, but it's possible some
were updated incorrectly or missed entirely. Please reach out to us
via the contact information listed at https://opendev.org/ with any
questions you may have.
2019-04-19 19:52:23 +00:00
Dean Troyer 0429fe8e37 Update .gitreview for f/ceph_mimic_upgrade
Change-Id: I845e125c6966bb78b32dafa01a31bf795abed85e
Signed-off-by: Dean Troyer <dtroyer@gmail.com>
2019-01-30 08:09:12 -06:00
SidneyAn 729a8e7f36 Remove retired codes populated “V1” tables
the old source code which populate "V1" tables are no longer being used
in the current stx-ha design architecture.

Remove list:
  service-mgmt/sm-db-1.0.0/upgrades/
  service-mgmt/sm-db-1.0.0/scripts/sm-db-populate.script
  service-mgmt/sm-db-1.0.0/patches/*install.patch
  service-mgmt/sm-db-1.0.0/database/sm_database.xlsb
  service-mgmt-tools/sm-tools/sm-tools/sm-patch.py

Story: 2004752
Task: 28842

Change-Id: I9f230f900baafcb8495d8f7ed739bf8e182abae5
Signed-off-by: SidneyAn <ran1.an@intel.com>
2019-01-29 00:12:11 +08:00
Zuul 6306f76223 Merge "Clean up and standardize landing pages" 2019-01-23 14:43:22 +00:00
Angie Wang 27113df5e8 Helm repository replication
This commit adds a new resource helmrepository-fs and its dependency
in SM.

Story: 2004520
Task: 28343
Change-Id: Ia2be180a15b27f08f8fa7a8ac32f8612be8a258d
Signed-off-by: Angie Wang <angie.wang@windriver.com>
2019-01-16 09:29:44 -05:00
Luis Botello 196c036013 Improve security by avoiding buffer overflows
This patch adds compiler flags to improve the security of STX code.
Flags added:
Format string vulnerabilities: CFLAGS="-Wformat -Wformat-security"
Compiler will treat string format warnings as errors,
so at compiling level, buffer overflow is avoided.

Story: 2004380
Task: 28823

Signed-off-by: Luis Botello  <luis.botello.ortega@intel.com>
Reviewed-by: Erich Cordoba <erich.cordoba.malibran@intel.com>
             Victor Rodriguez <vm.rod25@gmail.com>
Suggested-by: Victor Rodriguez <vm.rod25@gmail.com>
             Erich Cordoba <erich.cordoba.malibran@intel.com>

Change-Id: I45a0002288db434bc79c477c231f900e477347a1
2019-01-09 05:34:07 -06:00
Kristal Dale cf8a2fffc9 Clean up and standardize landing pages
doc index.rst:
1. Update intro sentence to read as a complete sentence
2. Remove unused toctree
3. Correct heading levels (impacting side nav and correct rendering
of content)
4. Remove "Indices and Tables" section: genindex page not used,
search searches only index (not useful here)

api-ref index.rst:
1. Update intro sentence to read as a complete sentence
2. Update text around search link for consistency (move
to follow intro)
3. Add heading before toctree for consistency with other pages

releasenotes index.rst:
1. Standardize page title reST markup
2. Remove search (make consistent with other openstack release note
pages)

Story: 2004737
Task: 28801

Change-Id: I83d023c57881e03639ff6bb85b256e53f4205817
Signed-off-by: Kristal Dale <kristal.dale@intel.com>
2019-01-08 15:20:30 -08:00
Zuul ff1b69302e Merge "Fix AIO-DX/DC no controller active issue" 2019-01-07 22:57:50 +00:00
Bin Qian 6c93e74230 Fix AIO-DX/DC no controller active issue
When controller-1 reboot in an AIO-DX/DC setup, mgmt/infra network will
temporarily go down. This is expected. However, SM couldn't determine
the interface going up again when the controller-1 reboot after unlock
the first time.

Add code to reverify the state of down interfaces when heartbeat
message is received.

Closes-Bug: 1809315
Change-Id: I02c9b6bf35539df2d36ad6b665b0a5ce8f2a1c75
Signed-off-by: Bin Qian <bin.qian@windriver.com>
2019-01-07 14:33:40 -05:00
zhipengl e406346c98 Refactor patches for rabbitmq-server package
After using rabbitmq-server-config package to package config files
for rabbitmq-server package, we can remove related patches of it. 
We need to let SM to call our custom ocf file instead of default one
installed by rabbitmq-server package.

Story: 2003768
Task: 28042

Change-Id: I12e132831d064daa00f26d44df505870a71c45ef
Signed-off-by: zhipengl <zhipengs.liu@intel.com>
2018-12-18 02:38:50 +00:00
Zuul 9cc9ce0a2d Merge "Change openstack-dev to openstack-discuss" 2018-12-17 18:49:58 +00:00
Tao Liu 787c7c192d Change compute node to worker node personality
The compute personality & subfunction has been changed to
worker, and compute_reserved.conf has been rename to
worker_reserved.conf.

This update changes sm dependencies to compute
personality and compute_reserved.conf.

Tests Performed:
Non-containerized deployment
AIO-SX: Sanity and Nightly automated test suite
AIO-DX: Sanity and Nightly automated test suite
2+2 System: Sanity and Nightly automated test suite
2+2 System: Horizon Patch Orchestration

Kubernetes deployment:
AIO-SX: Create, delete, reboot and rebuild instances
2+2+2 System: worker nodes are unlock enable and no alarms

Story: 2004022
Task: 27013

Depends-On: https://review.openstack.org/#/c/624452/

Change-Id: I0d6fac17cb5b7fa1e2cc757ea82b7c0385b9a6ee
Signed-off-by: Tao Liu <tao.liu@windriver.com>
2018-12-11 20:55:40 -05:00
Zuul 549d54a262 Merge "Use hbs cluster info to determine best survivor" 2018-12-11 20:07:43 +00:00
Bin Qian ad8665a1b7 Use hbs cluster info to determine best survivor
Uses cluster hbs info to determine which controller to be the survivor when
communication lost between 2 controllers with the new rules:

1. If a controller is the only controller to connect to storage-0,
it is choosen to be the survivor
2. A controller that can reach more nodes is choosen to be the survivor.
3. A controller is choosen to be failed if it cannot reach any nodes.

Story: 2003577
Task: 27704

Change-Id: I79659e1a788b865536500fc125fd65ae2f34123d
Signed-off-by: Bin Qian <bin.qian@windriver.com>
2018-12-11 11:12:16 -05:00
ZhijunWei 2694a20004 Change openstack-dev to openstack-discuss
Story: 2004582
Task: 28382
Change-Id: Ia27d6a4ffd2d00c271ea54274ba362df78dcc084
2018-12-10 14:46:38 +00:00
Zuul e4a8db70a9 Merge "CEPH support for 2 node configuration" 2018-12-06 14:50:30 +00:00
Zuul 847bce4edd Merge "SM management for Barbican in HA environment." 2018-12-06 04:26:30 +00:00
Stefan Dinescu 358cece3f4 CEPH support for 2 node configuration
In order to enable Openstack's helm charts on StarlingX we need
a distributed persistent storage for Kubernetes that leverages
our existing configurations.

Changes made:
- add to SM entries for monitoring ceph-mon, ceph-osd
  and drbd-ceph
- the ceph monitor is a floating monitor between the
  two controllers backed by a DRBD replicated filesystem

Change-Id: I8f9ea4798070e08171ad73da39821bc20b7af231
Co-Authored-By: Ovidiu Poncea <ovidiu.poncea@windriver.com>
Story: 2002844
Task: 26878
Signed-off-by: Stefan Dinescu <stefan.dinescu@windriver.com>
2018-12-04 18:13:05 +02:00
Alex Kozyrev 2ef5e82555 SM management for Barbican in HA environment.
Add Barbican processes into SM database for its management in HA.
SM will be able to manage Barbican processes via an OCF scripts.
 Barbican processes are: api, worker and keystone-listener).

Change-Id: I1d037b2051dd9febf3d73efe4032b957ab32c187
Story: 2003108
Task: 27700
Depends-On: I6b0b0c90456627bebde2b834b339bc968100b6f9
Signed-off-by: Alex Kozyrev <alex.kozyrev@windriver.com>
2018-12-01 14:45:53 -05:00
Bin Qian 28e293bda5 Retrieve hbs cluster info
This change includes:
1. adds code to receive cluster info update from hbsAgent.
2. support of ondemand hbs cluster info query (asynchronous).

Depends-On: I7d294d40e84469df6b6a6f6dd490cf3c4557b711

Story: 2003577
Task: 27816

Change-Id: Idb65abc58b4afe9649aba442f0798c24d9fffb10
Signed-off-by: Bin Qian <bin.qian@windriver.com>
2018-11-30 11:59:23 -05:00
Zuul d7ba498da9 Merge "Implement Active-Active Heartbeat as HA Improvement" 2018-11-21 16:46:23 +00:00
Zuul 0c142d12b7 Merge "Fix flake8 errors and disable ignore case" 2018-11-16 18:35:28 +00:00
Don Penney 0a606feba3 Rename centos_wheels.inc to centos_pike_wheels.inc
Update wheels inc file to include openstack release prefix

Change-Id: Ifcae39ffcb7f0cd9a4ecee98cfc3b6ef3a5ba6df
Story: 2003907
Task: 27527
Signed-off-by: Don Penney <don.penney@windriver.com>
2018-11-14 15:07:51 -06:00
Eric MacDonald b616bfb83d Implement Active-Active Heartbeat as HA Improvement
This update removes the hbsAgent from the SM database so
that it is no longer an SM managed/monitored process.

Story: 2003576
Task: 24907

Depends-On: https://review.openstack.org/#/c/617835
Change-Id: I12a8776d45c680dfe2fa9b8e2bf7cf3897ca9e1b
Signed-off-by: Eric MacDonald <eric.macdonald@windriver.com>
2018-11-14 20:18:39 +00:00
Zuul b1aad31b96 Merge "split-brain avoidance improvement" 2018-11-09 14:48:57 +00:00
Bin Qian 133da10b08 split-brain avoidance improvement
This change enables one way communication via BMC (if configured)
through mtce.
when 2 controllers lost all communications to each other.
The algorithm is:
when communications all lost,
both active and standby controllers, verify its interfaces (mgmt,
infra, and oam)
if active controller is healthy, it will request a bmc reset
thorugh mtce, against standby controller.
if standby controller is healthy, it will active itself and wait
a total 45 seconds before requesting a bmc reset through mtce,
against the active controller.

Changes also include:
1. adding new initial failover state.
   initial state is a state before the node is enabled
2. remove failover thread.
   using worker thread action to perform time consuming operations
3. remove entire failover action table

Story: 2003577
Task:  24901
Change-Id: I7d294d40e84469df6b6a6f6dd490cf3c4557b711
Signed-off-by: Bin Qian <bin.qian@windriver.com>
2018-11-08 20:18:43 +00:00
Sun Austin da53f143ee Fix flake8 errors and disable ignore case
below issures are removed from ignore cases:
    E114 indentation is not a multiple of four (comment)
    E116 unexpected indentation (comment)
    E121 continuation line under-indented for hanging indent
    E122 continuation line missing indentation or outdented
    E123 closing bracket does not match indentation of opening bracket's line
    E124 closing bracket does not match visual indentation
    E125 continuation line with same indent as next logical line
    E126 continuation line over-indented for hanging indent
    E127 continuation line over-indented for visual indent
    E128 continuation line under-indented for visual indent
    E129 visually indented line with same indent as next logical line
    E131 continuation line unaligned for hanging indent
    E201 whitespace after '('
    E228 missing whitespace around modulo operator
    E231 missing whitespace after ','
    E241 multiple spaces after ':'
    E251 unexpected spaces around keyword / parameter equals
    E265 block comment should start with '#'
    E271 multiple spaces after keyword
    E302 expected 2 blank lines, found 1
    E303 too many blank lines
    E305 expected 2 blank lines after class or function definition, found 1
    E704 multiple statements on one line (def)
    E713 test for membership should be 'not in'
    E714 test for object identity should be 'is not'
    E722 do not use bare except'
    E731 do not assign a lambda expression, use a def
    E999 SyntaxError: invalid syntax (this is likely python3)
    F401 <foo> imported but unused
    F841 local variable 'foo' is assigned to but never used
    H201: no 'except:'
    H233: Python 3.x incompatible use of print operator
    B001 Do not use bare `except:`
    B004 Using `hasattr(x, '__call__')` to test
         if `x` is callable is unreliable.
    B305 `.next()` is not a thing on Python 3. Use the `next()` builtin.
    B306 `BaseException.message` has been deprecated as of Python 2.6
          and is removed in Python 3.
    B007 Loop control variable 'key' not used within the loop body.
remain below issues in ignores:
    E402 module level import not at top of file
    ./service-mgmt-api/sm-api/sm_api/cmd/__init__.py:25

    Hxxx since which are related with document format

    F811 redefinition of unused '<foo>' from line <x>
    ./service-mgmt-tools/sm-tools/sm_tools/sm_configure.py:18

    F821 undefined name 'e'
    ./service-mgmt-api/sm-api/sm_api/common/utils.py:448

    B006 Do not use mutable data structures for argument defaults.
    ./service-mgmt-api/sm-api/sm_api/common/service.py:59

    B008 Do not perform calls in argument defaults.
    ./service-mgmt-api/sm-api/sm_api/openstack/common/timeutils.py:117

Test have been done:Build,Deploy,some smc command,such as smc service-list,
smc service-show, sm-dump, etc

Story: 2003430
Task: 26524

Change-Id: I3e2a4a31f87e3ff66cfce86f54285e830ee1c3dc
Signed-off-by: Sun Austin <austin.sun@intel.com>
2018-11-07 08:12:37 +08:00
Zuul fa29551d9e Merge "Add new configuration for hbs cluster info" 2018-11-05 19:54:49 +00:00
Bin Qian 01e3a2be4b Add new configuration for hbs cluster info
Add default of 2 new system configurations to sm.db.
Add configuration code to enable configure the options.

Story: 2003577
Task: 27727

Change-Id: I1a3dbaec78de591ef0de75ef9cafab48bcbb4dbf
Signed-off-by: Bin Qian <bin.qian@windriver.com>
2018-11-02 19:27:10 -04:00
Sun Austin ec02214db7 Fix: others issues for Python 2/3 compatible code.
Fix: "reduce" issue in
   service-mgmt-api/sm-api/sm_api/openstack/common/rpc/impl_zmq.py
     - from functools import reduce to support python 3
Fix: "long" issue in
   service-mgmt-api/sm-api/sm_api/openstack/common/jsonutils.py
     -  remove long type if for python 3
     -  change to six.integer_types
Fix: "funcattrs" issue in
   service-mgmt-api/sm-api/sm_api/openstack/common/db/sqlalchemy/session.py
     - change f.func_name to f.__name__
Fix: "zip" and "methodattrs" issues in
   service-mgmt-api/sm-api/sm_api/common/safe_utils.py
     - change zip() to list(zip())
     - change function.im_self to function.__self__
Fix: "itertools" issue in
   service-mgmt-api/sm-api/sm_api/openstack/common/log.py
     - from six import moves and use moves.filter

Story: 2003430
Task: 26523

Change-Id: I37b47a12a24ef05fb4d57823b2a91086b6ca537f
Signed-off-by: Sun Austin <austin.sun@intel.com>
2018-10-27 09:30:19 +08:00
Sun Austin 6753274b28 Fix: "metaclass" issues for Python 2/3 compatible code.
remove "__metaclass__ = abc.ABCMeta"
and adding decorator "@six.add_metaclass(abc.ABCMeta)" on class

and removing pep8 ignore case

Story: 2003430
Task: 27476

Change-Id: If60b8d5662d9a87b217a9b1d48a2c3e9b037884b
Signed-off-by: Sun Austin <austin.sun@intel.com>
2018-10-26 11:49:16 +08:00
Sun Austin d9c7f0bc2f Fix types issue for Python 2/3 compatible code.
replace types.NoneType  with type(None)
replace types.DictType with dict

Story: 2003430
Task: 27481

Change-Id: I835b1c086267b05bedb454a0d2eaaeef58254dae
Signed-off-by: Sun Austin <austin.sun@intel.com>
2018-10-26 11:49:16 +08:00
Don Penney b4049622ec Add hooks for python wheel generation
This update adds hooks to the spec files for the following packages
to generate wheels for the python modules:
- sm-api
- sm-client
- sm-tools

Change-Id: If6ebbdaaad5ee7623cbbaa07b48d216461a16dec
Story: 2003907
Task: 27527
Signed-off-by: Don Penney <don.penney@windriver.com>
2018-10-24 17:13:06 +00:00
Zuul c3050c7fc1 Merge "force flake8 < 3.6.0 to avoid break current gate" 2018-10-24 16:48:01 +00:00
Sun Austin d8bf005235 force flake8 < 3.6.0 to avoid break current gate
the version flake8 3.6.0 will cause below error
 W504 line break after binary operator
 W605 invalid escape sequence '\w'
 F901 'raise NotImplemented' should be 'raise NotImplementedError'

Change-Id: If9a8c678aa1b573ec7b4e5ad324ba86c5d90b959
Signed-off-by: Sun Austin <austin.sun@intel.com>
Partial-Bug: 1799721
2018-10-24 09:50:33 -05:00
Sun Austin de91fb2ac1 Fix urllib import issue for Python2/3 compatible code.
change urlparse to 'from six.moves.urllib.parse import urlparse'
and remove obsolete code

Story:2003430
Task:27479

Change-Id: I38e75f5faa55046517f1e479b18ebcef0304e471
Signed-off-by: Sun Austin <austin.sun@intel.com>
2018-10-24 08:39:08 +08:00
Sun Austin 1477d25d55 Fix: "raise" issues for Python 2/3 compatible code.
use from six import reraise as raise_
to re-implement raise

Story:2003430
Task:27478

Change-Id: I34373ce7f233f7f908da3d92bcc000a9e924375d
Signed-off-by: Sun Austin <austin.sun@intel.com>
2018-10-24 08:39:08 +08:00
Zuul 28ad64fdbb Merge "close File point when it was opened in C files" 2018-10-23 16:05:14 +00:00
Zuul afd09544e8 Merge "Fix: "except" issues for Python 2/3 compatible code." 2018-10-23 13:08:46 +00:00
Sun Austin 424a3c4463 close File point when it was opened in C files
Closes-Bug:1791880

Change-Id: I392bfa8b864d18b00ba18958fce8233f950b00b9
Signed-off-by: Sun Austin <austin.sun@intel.com>
2018-10-23 08:56:55 +08:00
Zuul accf67c475 Merge "[Doc] openstackdocstheme starlingxdocs theme" 2018-10-23 00:44:25 +00:00
Abraham Arce eb86e7e885 [Doc] openstackdocstheme starlingxdocs theme
Enable starlingxdocs theme support for:

- Documentation
- Release Notes
- API Reference

Change-Id: I37051b4bd80d87a2282f5eec65e849d8ac1e77d7
Signed-off-by: Abraham Arce <abraham.arce.moreno@intel.com>
2018-10-22 14:36:04 +00:00
Sun Austin 4fe73f515d Fix: "except" issues for Python 2/3 compatible code.
change "except ValueError, e:"
to "except ValueError as e:"

and remove tox H231 from ignore for pep8 check
Story:2003430
Task: 26522

Change-Id: I06c7646c375ada7f8f06f870a7bdb0d7fc282e5a
Signed-off-by: Sun Austin <austin.sun@intel.com>
2018-10-22 11:49:35 +08:00
Sun Austin f2a6d355a9 Fix next issue for Python 2/3 compatible code.
change x.next() to next(x)
and
add __next__() for class

Story: 2003430
Task: 26519

Change-Id: I4cda6f1d679e05b7f533b94840e0eb561213aa76
Signed-off-by: Sun Austin <austin.sun@intel.com>
2018-10-22 09:51:19 +08:00
Sun Austin eb6a4cae4e Initial variable before read
Closes-Bug:1795120

Change-Id: I116d489da785e50d3a37157079bf2509c03dca43
Signed-off-by: Sun Austin <austin.sun@intel.com>
2018-10-22 08:46:31 +08:00
Zuul d987f1bcdf Merge "Fix service groups on controller-1 stuck in initial" 2018-10-17 12:15:13 +00:00
Zuul 9a3650ea95 Merge "Add api-ref and relnotes publish jobs" 2018-10-16 14:15:34 +00:00
Bin Qian 5f9731c47e Fix service groups on controller-1 stuck in initial
Initialize fail-pending timer id.
Also deregister the timer when program exits.

Closes-Bug: 1797567

Change-Id: Ief278dfff1185a6acea718b683da11934a192161
Signed-off-by: Bin Qian <bin.qian@windriver.com>
2018-10-16 10:14:29 -04:00
Zuul 5a88ecc2a7 Merge "Add idle time to worker thread" 2018-10-15 20:37:37 +00:00
Bin Qian 3eb68cc231 Add idle time to worker thread
Adding 50ms idle time in worker thread loop.
This is to fix sm taking 100% cpu for a busy loop.

Closes-Bug 1797438

Change-Id: Ia41acfab86c0188ceb5c80822010376977c6fc74
Signed-off-by: Bin Qian <bin.qian@windriver.com>
2018-10-15 17:51:46 +00:00