Commit Graph

30 Commits

Author SHA1 Message Date
Don Penney 9531b76af1 Move content to subdir to support relocated packaging
Change-Id: Id4f645de23b9f0aec1914edbaef11cdb6e6dc0af
Story: 2006166
Task: 37337
Depends-On: https://review.opendev.org/692861
Signed-off-by: Don Penney <don.penney@windriver.com>
2019-11-04 13:57:02 -05:00
Tyler Smith c0e54ee4de Request forwarding and version matching in dcorch proxy
Need to pass in the application to be the forwarder so when
the version doesn't match the request can continue down the
pipeline.  It was 'mostly' working before since the version
matching was registering all get requests as version requests
and just forwarding them, this commit also restricts that
version regex so that is no longer the case.

Change-Id: I887027a043e2a686770d4ece0ae511e00814be61
Closes-Bug: 1849831
Signed-off-by: Tyler Smith <tyler.smith@windriver.com>
2019-10-25 17:45:02 -04:00
Andy Ning eb572c47f8 Check ids instead of names for DC assignment synchronization
In distributed cloud, subcloud's user ids, project ids and role ids
are synced with System Controller. But project role assignment
functions still use names to check if master resources and subcloud
resources has the same id, and if user, project and role exist before
POST call to grant project role to user. This will cause an assignment
PUT job created and identity sync status flip from "in-sync" to
"out-of-sync" and back to "in-sync" again for every audit cycle.

A more detailed explanation, at the very first audit, roles are queued
for sync but the job doesn't run and their ids don't changed at the
subcloud yet. At the same audit dcorch finds the project role assignment
actually exist (since it check names in has_same_ids()), so it maps the
the assginment of center cloud to the assignment of the subcloud with
the current ids. Once the roles sync job queued get executed, roles ids
are changed. At this point the assignment mappings becomes invalid. The
next audit can no longer find the mapped assignment from subcloud so the
logic falls into audit_discrepancy() where the has_same_ids() return
TRUE again and a PUT job is queued for the assignment. The sync endpoint
type becomes "out-of-sync" since there is a job for it. Once the PUT
function return, its status returns to "in-sync" again.

This change updated project role assignment functions to use ids
instead of names.

Change-Id: I024f2c2f97aaf9670d7b2c5c70a2dae7d6d08d38
Closes-Bug: 1847661
Signed-off-by: Andy Ning <andy.ning@windriver.com>
2019-10-25 14:36:19 -04:00
Tyler Smith 96db1d7ee0 Distributed Cloud enhancements for Horizon
- Adding endpoints for the subcloud's platform services to the central
  keystone.  This was done so horizon can reach all subclouds
- Allowing version requests to bypass the authtoken validator in the
  dcorch proxy.  version requests do not require authentication and
  they are required by horizon to work in the SystemController region

Change-Id: I508e0168e77d1f46b8f5720fd16047177b4920c2
Partial-Bug: 1846239
Signed-off-by: Tyler Smith <tyler.smith@windriver.com>
2019-10-18 16:57:16 -04:00
Zuul b020fc86c8 Merge "keystone-api-proxy for containerized openstack services - service" 2019-10-17 15:08:45 +00:00
Andy Ning 3e2eccea4a Moved eventlet monkey_patch before app is imported
In cmd/api.py the eventlet monkey_patch has been moved to be before api
app import. This is because if it's called too late, the api app and
db api module will be loaded without awareness of eventlet, the
threading local context in db api won't be eventlet compatible, causing
DB parallel operation errors.

Change-Id: I294657fc910c6a4696f91308d60697d005dc53b0
Closes-Bug: 1846411
Signed-off-by: Andy Ning <andy.ning@windriver.com>
2019-10-03 13:29:30 -04:00
Andy Ning 95eac8c932 keystone-api-proxy for containerized openstack services - service
This update enhanced keystone-api-proxy to take a sync_endpoint
parameter from its configuration file and enqueue job for dcorch with
that sync_endpoint type. If sync_endpoint doesn't present in its
configuration file, it will use the default endpoint type to enqueue
job.

Change-Id: I85698638cee2598955c4deb41a6b8033b0ace9fd
Story: 2004766
Task: 36156
Depends-On: https://review.opendev.org/#/c/682062/
Signed-off-by: Andy Ning <andy.ning@windriver.com>
2019-09-26 15:12:19 -04:00
Zuul c3241d0a41 Merge "DC remove firewallrules audit from dcorch" 2019-09-19 16:47:30 +00:00
Andy Ning a7d91e2961 DC remove firewallrules audit from dcorch
OAM firewallrules are now managed by Calico GlobalNetworkPolicy configuration
via k8s API (not by sysinv anymore). This update removed firewallrules
audit from dcorch.

Change-Id: I9fab73c016bb4af760c7d78f0db18dcc8bb77057
Closes-Bug: 1844147
Signed-off-by: Andy Ning <andy.ning@windriver.com>
2019-09-18 10:08:13 -04:00
Andy Ning 4fc9543820 Fix dcorch audit_discrepancy() failure
In a Distributed Cloud system, when dcorch audit platform, it will
failed at audit_discrepancy() function call. This is because sysinv
audit_discrepancy() missed the 4th parameter. This update fixed this by
adding it in.

Change-Id: I72057b3406b4b362808d241fbc2e43bf07d7b677
Closes-Bug: 1843770
Signed-off-by: Andy Ning <andy.ning@windriver.com>
2019-09-12 13:32:20 -04:00
fpxie 1f4769dafd Use triple double-quoted strings for docstrings
Change-Id: I1bc612866b9fc98be7e702565ba249bda4a54785
Story: 2006136
Task: 35632
2019-07-01 15:10:59 +08:00
Andy Ning 828cdcf2bb Keystone DB sync - enhance dcorch first audit
This commit enhanced dcorch find_missing algorithm so that the first
audit after subcloud becomes managed won't try to create resources that
have matches in the subcloud (otherwise the creation will fail for DB
duplication error). This is neccessary for resources that are created
at deployment time and existing resources not yet tracked by dcorch.

Story: 2002842
Task: 22787

Change-Id: I60f94057caf71265942f3b37b400eeba4f368fed
Signed-off-by: Andy Ning <andy.ning@windriver.com>
2019-04-30 14:51:17 -04:00
Andy Ning 67215f30fe Keystone DB sync - update dcorch to use dcdbsync
This commit updates dcorch to use the newly introduced dbsync service
APIs to synchronize identity resources from central cloud to subclouds.
The following identity resources are synced:

- users (local users only)
- user passwords
- projects
- roles
- project role assignments
- token revocation events

Story: 2002842
Task: 22787

Signed-off-by: Andy Ning <andy.ning@windriver.com>
(cherry picked from commit e9096c7a23)

Depends-On: https://review.opendev.org/#/c/655921
Depends-On: https://review.opendev.org/#/c/655773
Depends-On: https://review.opendev.org/#/c/655776
Depends-On: https://review.opendev.org/#/c/655927
Change-Id: I77c2cc712a1c3dc8a228883c3fea1423e5207dea
2019-04-30 14:50:55 -04:00
Tao Liu 4139b42efe Remove oslo transport aliases
Aliases were first deprecated in oslo.messaging 5.20.0
during Pike, and they have been removed in Stein.

This update removes oslo transport aliases in both
dcorch and dcmanager messaging to support the
containerized keystone-api-proxy that uses Stein.

Story: 2004766
Task: 30450

Change-Id: I015e23575d56ab031a7a94efa4ec5464fcd3f543
Signed-off-by: Tao Liu <tao.liu@windriver.com>
2019-04-10 11:36:33 -04:00
Tao Liu 3ac45e3281 Integrate Distributed Cloud with containerized services
Disable nova, cinder and neutron api proxy services
Disable nova, cinder and neutron sync threads
Add cluster IP support to generated subcloud
configuration file
Remove openstack users from subcloud user list

Story: 2004766
Task: 28884

Change-Id: I683ba05ee74b159716924f08814a7473e7053d4d
Signed-off-by: Tao Liu <tao.liu@windriver.com>
2019-03-14 15:13:21 -04:00
Sun Austin 6022c0420d FIX other issues for Python 2/3 compatible code
change func.func_name to func.__name__
change itertools.izip_longest to six.moves.zip_longest
change self.assertRaisesRegexp(xxx) to six.assertRaisesRegex(self,xx)
use absolute import path

Story: 2004585
Task: 28447

Change-Id: I2b4814fb6f96feb90ad0a081ffcab5c72160b0f7
Signed-off-by: Sun Austin <austin.sun@intel.com>
2019-01-15 00:33:54 +00:00
Sun Austin 5e9dc48384 FIX: "dict" issue for Python 2/3 compatible code
Replace dict.iteritems() with dict.items()
Change dict.keys() to list(dict.keys())

Story: 2004585
Task: 28449

Change-Id: I674c499fd575800fa307a835d8f7f272f2b2cd03
Signed-off-by: Sun Austin <austin.sun@intel.com>
2019-01-15 00:33:38 +00:00
Sun Austin bb65ec36e6 FIX: module rename issue for Python 2/3 compatible code
import six.moves to fix rename of urlparse and Queue

Change-Id: Iee1b6053e4c1a174c40cfa858bf62557d308194a
Story: 2004585
Task: 28446
Signed-off-by: Sun Austin <austin.sun@intel.com>
2019-01-15 00:33:13 +00:00
Alex Kozyrev 7fe134264b Convert NTP/PTP enabled flag to string implicitly in DC sync.
NTP/PTP enabled flag is not propagated properly to subclouds.
The root cause is the wrong boolean/string comparison of the enabled
flag. Need to to convert it to string to simplify the logic.

Change-Id: Ie9d67c567732caf5edba751fd1310b94d2c084ca
Closes-Bug: 1802530
Signed-off-by: Alex Kozyrev <alex.kozyrev@windriver.com>
2018-12-14 17:04:51 +00:00
Kristine Bujold 149979407e Adding TODO comment
Adding a clarification to earlier commit 7c1c082b.

Related-Bug: 1802397

Change-Id: I9c008db783fa32c5747e416844f76315bd4dc3ba
Signed-off-by: Kristine Bujold <kristine.bujold@windriver.com>
2018-11-30 13:56:16 -05:00
Kristine Bujold 7c1c082be0 DC fix audit sync failure
During the sync Security Group needs to be compared with the tenant
names and not tenant ids. The tenant id which is the uuid are different
between regions. The compare does not match and an attempt to create
the Security Group fails since it already exists. This commit changes
the compare to use tenant names.

Closes-Bug: 1802397

Change-Id: I9363945868b857802e137dcf05757ec3691230c1
Signed-off-by: Kristine Bujold <kristine.bujold@windriver.com>
2018-11-09 10:41:01 -05:00
Zuul 736b7c3779 Merge "Fix pep8 error when running tox" 2018-11-02 15:18:27 +00:00
Tao Liu 3dceb62d37 Fernet key synchronization
This update contains the following changes for Distributed
Cloud Fernet Key Synching & Management:

1.Distribute the fernet keys when the subcloud is managed
2.Setup a periodic task to rotate and re-distribute keys
3.Support fernet repo audit

Story: 2002842
Task: 22786

Depends-On: https://review.openstack.org/#/c/613620/

Change-Id: I203c937e9c2334da3f4766c0a49f32f71f7fd39e
Signed-off-by: Tao Liu <tao.liu@windriver.com>
2018-10-29 20:08:34 -05:00
junboli 0c34a4901d Fix pep8 error when running tox
openstack-tox-pep8 now is failure, this patch is to fix
this.

Closes-bug: #1799930
Change-Id: Ia1b7f3395e92014e0f5373d81788dc55f7815002
2018-10-29 06:23:49 +08:00
Andy Ning 95da742946 Fix role synchronization in distributed cloud
There is a typo error in role deletion that prevents deleting the role
in subcloud during role synchronization. This update fixed this typo
error and made role synchronization work.

Closes-Bug: 1797960

Change-Id: Iff78ceffdd95b2676854d986126c6c2d001866de
Signed-off-by: Andy Ning <andy.ning@windriver.com>
2018-10-17 13:34:32 -04:00
Alex Kozyrev 27654d679c Fix for PTP sync failure n a Distributed Cloud system.
PTP failed to sync because of "enabled" parameter passed.
It was passed as a boolean and ingnored by SysInv API.
Need to convert it to string before passing to SysInv.

Change-Id: Ice439bcb46bc901390c562f4ca5a8af0a73b738e
Closes-Bug: 1791997
Signed-off-by: Alex Kozyrev <alex.kozyrev@windriver.com>
2018-10-03 14:34:00 -04:00
melissaml 2ca33d3e42 fix a typo
Change-Id: Ica2307f82c6c98d47d64b4b8f740efca95f94b5a
2018-09-20 10:57:29 +08:00
Alex Kozyrev 2e3f520442 Distributed Cloud: PTP parameters synchronization.
There are 3 new parameters added to PTP configuration:
timestamping mode, transport protocol and delay mechanism.
Need to synchronize these parameters in the Distibuted Cloud.

Signed-off-by: Alex Kozyrev <alex.kozyrev@windriver.com>
2018-08-28 15:51:39 -04:00
Tao Liu 38f08a0d98 Decouple Fault Management from stx-config
The fault management (FM) APIs has been removed from the syinv API
service and a new FM API service has been introduced. This update adds
a new fm openstack driver for retrieving each region's alarm summary,
and it also modifies the alarm aggregate manager to use the fm driver.
In addition, it removes get alarm summary routine from sysinv and
adds the  fm user to the subcloud user list

Story: 2002828
Task: 22747

Signed-off-by: Tao Liu <tao.liu@windriver.com>
2018-08-23 14:59:38 -04:00
Scott Little e82c7b4336 StarlingX open source release updates
Signed-off-by: Scott Little <scott.little@windriver.com>
2018-08-07 11:51:21 -04:00