summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorJohn Kung <john.kung@windriver.com>2018-10-17 10:14:34 -0400
committerJohn Kung <john.kung@windriver.com>2018-10-30 11:21:18 -0400
commit756973392675b45bd8d501163ce13b92badaf4bd (patch)
treeb198086f9a681d8871eab0c9aacda8fb7bff019c
parent8317f38ade5528fe4008d2852efa1e4c3ca3ea4a (diff)
Create Spec: Decouple System/Service Configuration from System Inventory
stx-config project sysinv currently consists of Inventory and Configuration components. Inventory includes Host Management. This creates the spec to decouple Inventory components from System, Storage, Service Configuration and management. Story: 2002950 Task: 22952 Change-Id: I1a4d3b4e2191a14a7919425efe4fcec58de096b3 Signed-off-by: John Kung <john.kung@windriver.com>
Notes
Notes (review): Code-Review+2: Brent Rowsell <brent.rowsell@windriver.com> Workflow+1: Ian Jolliffe <ian.jolliffe@windriver.com> Verified+2: Zuul Submitted-by: Zuul Submitted-at: Wed, 31 Oct 2018 02:05:01 +0000 Reviewed-on: https://review.openstack.org/611331 Project: openstack/stx-specs Branch: refs/heads/master
-rw-r--r--specs/2019.03/approved/sysinv_2002950-decouple-system-configuration-from-inventory.rst425
1 files changed, 425 insertions, 0 deletions
diff --git a/specs/2019.03/approved/sysinv_2002950-decouple-system-configuration-from-inventory.rst b/specs/2019.03/approved/sysinv_2002950-decouple-system-configuration-from-inventory.rst
new file mode 100644
index 0000000..769e9be
--- /dev/null
+++ b/specs/2019.03/approved/sysinv_2002950-decouple-system-configuration-from-inventory.rst
@@ -0,0 +1,425 @@
1.. This work is licensed under a Creative Commons Attribution 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/legalcode
2
3===================================================
4Decouple System Configuration from System Inventory
5===================================================
6
7Storyboard: https://storyboard.openstack.org/#!/story/2002950
8
9This story decouples System Configuration from System Host
10Management and Inventory.
11
12
13Problem description
14===================
15
16SysInv currently consists of Inventory and Configuration components.
17
18The Inventory component performs Host management and physical inventory
19discovery. The Inventory component interacts with stx-metal and stx-nfv
20components for host management.
21
22The Configuration component performs System, Storage configuration and
23management, and generation of configuration data, such as for puppet
24hierdata or helm charts.
25
26Currently, the inventory resources are coupled to configuration resources
27via its relational data model and does not allow external configuration
28services.
29
30
31Use Cases
32=========
33
34The decoupled system provides the same functional capabilities as the
35coupled SysInv. The physical and logical resource management are
36decoupled for a focused metal management and a more flexible logical
37resource management framework.
38
39Allow Inventory Host Management component to interact with the plugin
40Configuration to create the required configuration. The plugin
41allows the bind in of abstract configuration methods required for the
42Inventory host management operations. This increases flexibility to
43provide alternative logical configuration drivers.
44
45Configuration obtains Inventory data via GET request on the
46Inventory REST API resources. This allows Configuration to develop
47with a published API.
48
49Upversion to oslo standard libraries to facilitate Developer integration
50of components.
51
52Allow move of Inventory component to stx-metal from stx-config for
53architectural alignment of bare metal management.
54inventory depends on interfaces with stx-metal; systemconfig does
55not depend upon stx-metal.
56stx-metal is thus focused on physical inventory and bare metal management.
57
58
59Proposed change
60===============
61
62Create separate Inventory database, and Inventory services for REST api,
63conductor and agent. The configuration driver is developed as a plugin
64to Inventory for configuration.
65
66Update Inventory python client to support sessions. This facilitates
67keystone token management, for consumers such as SystemConfiguration
68and Distributed Cloud.
69
70Create separate Configuration database, and Configuration services for api,
71conductor, and agent.
72The high availability aspect of the api and conductor services are managed
73by stx-ha; and agent by stx-metal pmon.
74
75Update datamodels to decouple Inventory from Configuration.
76Relationships between Inventory and Configuration database tables are removed
77and referenced externally via each resource's globally unique identifier.
78Information required between services are obtained via the service's REST API.
79
80Update infrastructure to reference the oslo standard libraries.
81
82
83Alternatives
84============
85
86Alternatively, maintain SysInv as combined Inventory and Config with its
87current built-in puppet and helm configuration drivers. Update to allow plugin
88of alternative Configuration drivers based upon its Sysinv service
89configuration.
90
91Advantages:
92 * Facilitates access to data required for configuration without
93 requiring an external REST API, since internal database reference.
94 * Maintains referential integrity when inventory updates without requiring
95 external notification or audit.
96 * Upgrades: database is not split across different services.
97 * Lower development cost as a separate API, service is not created and
98 data model split could still have some interdependencies.
99
100Disadvantages:
101 * Interdependencies between inventory and configuration hinder
102 introduction of alternative configuration drivers, and raise risk of
103 introducing additional dependencies between the physical inventory and
104 baremetal management and logical configuration management.
105 * Referential integrity only applies to internal configuration
106 implementation with access to data model. Formal contract for API is
107 still required when introducing flexibility of an external
108 configuration driver.
109 * Coupling physical and logical configuration resources lessens flexibility
110 of each component to evolve independently.
111 * The Inventory and Host bare-metal management component remains
112 within stx-config.
113 * Configuration Resources which are populated could result in changes to
114 Inventory Resource data; thus the datamodel split would
115 still be required.
116
117
118Data model impact
119=================
120
121The current datamodel relations which reference the split Inventory into
122Configuration component needs to be removed from the service.
123Configuration will obtain the required reference based upon the resource uuid
124which it GETs from the Inventory API.
125
126The SysInv datamodels are split according to:
127
128Inventory:
129 * Host
130 * System - (Inventory reference) - uuid
131 * Ports
132 * Lldp
133 * Pci_devices
134 * Disks
135 * Cpus
136 * Memory
137 * Sensors,
138 * SensorGroups
139
140Configuration:
141 * System
142 * Host, for configuration host info such as config_status
143 * Interfaces
144 * Networks
145 * Interface_Networks
146 * Addresses
147 * AddressPools
148 * Routes
149 * Helm_overrides
150 * Certificates
151 * Community
152 * ControllerFS
153 * DNS
154 * DRBDConfig
155 * NTP
156 * PTP
157 * RemoteLogging
158 * ServiceParameter
159 * Storage: lvgs, pvs, clusters, peers, partition, ceph_mon, journal
160 * Storage: storage_backend, _ceph, _external, _lvm, _file, _tiers,
161 * TpmConfig, Tpmdevices
162 * User
163
164
165REST API impact
166===============
167
168Existing APIs from SysInv are migrated to Inventory or Configuration.
169 * New Configuration APIs are introduced and migrated APIs from Inventory
170 are deprecated.
171 * Remove the ā€˜iā€™ prefix in URL resource, where applicable.
172
173There are no policy changes. admin was required to access the SysInv API,
174and is required for the Inventory and Configuration APIs.
175
176New Configuration APIs to support the generation of configuration for the host:
177 PUT v1/hosts/<host_uuid>/<action>
178 The following actions are required:
179 * configure
180 * configure_check
181 * check whether config is sufficient (e.g. for host-unlock)
182 * update_operational
183 * For storage host, config performs update_add_ceph_state
184 disable_check
185 * Determines whether host config may be disabled
186 (i.e. pre host-lock)
187
188
189Security impact
190===============
191
192Security of the new Configuration service is equivalent to Sysinv:
193 * systemconfig-api REST API service with keystone authentication and
194 haproxy for https configured oam interface.
195 * api service requires admin keystone policy and runs under
196 systemconfig user privilege.
197 * database, amqp/rabbit access are protected by username and password.
198
199
200Other end user impact
201=====================
202
203A new python-client, python-systemconfigclient is introduced for the
204Configuration component. The systemconfig cli will retain 'system',
205whereas the inventory cli will be under 'inventory'.
206
207The interface will no longer be automatically created. Previously, in the
208case of AE config, interfaces need to be configured to 'none' before being
209configured again. This transition is no longer required in this case.
210
211
212Performance Impact
213==================
214
215With Sysinv Decoupling, additional REST API calls are required between the
216independent Inventory and Configuration components.
217
218In particular, the following additional REST API interactions:
219* Inventory notifies Configuration via REST API to perform
220host configuration action
221* Configuration requests up-to-date view on configuration action,
222the scope is generally limited to the amount of data to be transferred
223for the host inventory resource.
224* Periodic audits from system config is required to ensure the Inventory
225Hosts view is accurate and up-to-date.
226
227
228Other deployer impact
229=====================
230
231Initial Bootstrap (config_controller) now initializes both Inventory and
232Configuration services and populates the services with the required host
233and configuration data. This will be transparent to the users.
234
235The association of interfaces to ethernet_ports was performed by default
236to a single interface previously. With the separation of the ports and
237interfaces data model, the admin must now associate the interface to
238the port as required (ie. via the system host-if-add).
239
240Support for profiles is removed. The current implementation requires
241reference between inventory and configuration resources.
242
243
244Developer impact
245=================
246
247* A new driver API for Configuration is introduced.
248
249
250Upgrade impact
251===============
252
253Upgrades from N-1 are not supported for this update.
254
255
256Implementation
257==============
258
259Assignee(s)
260===========
261
262Primary assignee:
263 john.kung@windriver.com
264 louie.kwan@windriver.com
265
266Other contributors:
267
268
269Repos Impacted
270==============
271
272stx-config: systemconfig
273 python-systemconfigclient
274 stx-metal pmon is configured to manage systemconfig-agent.
275
276stx-ha: SM management of systemconfig and inventory.
277
278stx-metal: mtce integration
279 wrsroot user update via systemconfig-api rather than
280 inventory-api.
281
282stx-nfv: vim interacts with the inventory and configuration components
283 and will need a new client to access the host information
284 from both the inventory and configuration components
285
286distributedcloud:
287 update to api-proxy, dcorch engine to bind to systemconfig
288 optional: simplify framework to utilize keystone sessions
289 as per other resources managed by dcorch.
290
291stx-gui: update to reference inventory and configuration apis.
292 datamodels within api/sysinv.py need to be refactored to
293 fill in the data from the respective service.
294
295stx-clients: add python-systemconfigclient to remoteclients
296
297stx-integ: ceph-manager deprecate
298 restapi-doc updates
299
300
301Work Items
302==========
303
304Phase 1: Create new configuration services and update required
305infrastructure
306
307* systemconfig:
308 * data model: decouple from internal inventory resource references
309 * create systemconfig-api
310 * REST API for new config actions (see REST API section)
311 * propagate sysinv-api to systemconfig-api
312 * obtain inventory resources as required from inventory api
313 * remove 'i' prefix from URL
314 * create systemconfig-conductor
315 * update framework to use standard libraries rather than the internal
316 libraries in sysinv (e.g. openstack.common is migrated to oslo_service,
317 paste, keystoneauth1, oslo_db, oslo_messaging, oslo_log)
318
319* sysinv:
320 * data model: decouple from configuration resource references
321 * update REST API
322
323* ha management:
324 * stx-ha service-management of systemconfig-api, systemconfig-conductor
325 * stx-metal pmon service management of systemconfig-agent(via configuration
326 * implemented in stx-config)
327
328* vim:
329 * update VIM to handle sysinv API changes
330
331* update config_controller bootstrap to systemconfig. Startup services,
332 populate initial configuration.
333
334* python-sysinvclient:
335 * update CLI and client to include keystone session support
336 for token management.
337
338* python-systemconfigclient:
339 * create CLI and client
340
341
342Phase 2: Decouple Major focus areas
343
344Major decoupling focus areas:
345 * interface decoupling
346 ethernet_ports in inventory and interfaces in systemconfig
347
348 * remove interface_id from ports table in inventory
349 * remove autoprovisioning of interfaces in systemconfig
350
351 * storage decoupling
352
353 * disk in inventory and partition/lvg/stor in config
354 * ceph_manager
355 * deprecate ceph-manager: move rpc endpoint functionality into
356 systemconfig-conductor, ceph-manger-audit to config-audit,
357 and alarm monitoring into collectd plugin.
358
359Create plugin model in Inventory for configuration. The plugin is
360implemented as a stevedore driver and selection of driver is driven by
361config. The plugin allows the bind in of abstract configuration methods
362required for the Inventory host management operations.
363
364Phase 3: Decoupling of remaining resources and Integration
365
366* decouple remaining configuration resources from sysinv
367
368* distributedcloud
369 * proxy SystemController systemconfig-api requests into dcorch-engine
370 * dcorch-engine to interface systemconfig-api for configuration
371 * dcmananger-api to interface with python-systemconfigclient
372 (network list, address_pools, routes)
373
374* stx-gui inventory dashboard updated to reference the inventory and
375 configuration REST APIs
376
377* tox unit tests (this could be started earlier, however initial focus
378 is verification in lab)
379
380
381Dependencies
382============
383
3842002827 Decouple Service Management REST API from sysinv
385https://storyboard.openstack.org/#!/story/2002827
386
3872002828 Decouple Fault Management from stx-config
388https://storyboard.openstack.org/#!/story/2002828
389
390
391Testing
392=======
393
394* Bootstrap Initialization and Configuration
395* Host Configuration and Management
396* Interface Configuration
397* Storage Configuration
398* Service Parameter Configuration
399* HA verification
400* Distributed Cloud Verification
401* Horizon GUI
402* Devstack
403
404
405Documentation Impact
406====================
407
408systemconfig and sysinv REST API documentation
409End User Guide: installation and configuration
410
411
412References
413==========
414
415
416History
417=======
418
419.. list-table:: Revisions
420 :header-rows: 1
421
422 * - Release Name
423 - Description
424 * - 2019.03
425 - Introduced