From dfaeb38ab727cbae8ad578e144fbfd42b3070089 Mon Sep 17 00:00:00 2001 From: Cole Walker Date: Fri, 25 Mar 2022 15:52:34 -0400 Subject: [PATCH] [PTP Dual NIC] Spec for PTP Dual NIC work The spec document covers the proposed changes to the StarlingX PTP configuration model in order to support multi-instance PTP and enable more flexible configuration. Story: 2009248 Task: 44860 Signed-off-by: Cole Walker Change-Id: I2957b70b6c9874e4750379f4bc7c7a77c3e0c5e2 --- ...networking_2009248_ptp_dual_nic_config.rst | 683 ++++++++++++++++++ 1 file changed, 683 insertions(+) create mode 100644 doc/source/specs/stx-7.0/approved/networking_2009248_ptp_dual_nic_config.rst diff --git a/doc/source/specs/stx-7.0/approved/networking_2009248_ptp_dual_nic_config.rst b/doc/source/specs/stx-7.0/approved/networking_2009248_ptp_dual_nic_config.rst new file mode 100644 index 0000000..5481c7b --- /dev/null +++ b/doc/source/specs/stx-7.0/approved/networking_2009248_ptp_dual_nic_config.rst @@ -0,0 +1,683 @@ +.. + This work is licensed under a Creative Commons Attribution 3.0 Unported + License. http://creativecommons.org/licenses/by/3.0/legalcode + +.. + Many thanks to the OpenStack Nova team for the Example Spec that formed the + basis for this document. + +PTP Dual NIC Boundary Clock Configuration +========================================== + +Storyboard: +https://storyboard.openstack.org/#!/story/2009248 + +The objective of this feature is to rework how StarlingX handles the +configuration and deployment of the PTP userspace applications (ptp4l, phc2sys +and ts2phc). There are a wide variety of applicable configurations that a user +may wish to deploy on their system and this work is required to allow these +diverse configurations to be achieved. The basic solution is for StarlingX +to provide a method to configure multiple instances of each application type, +each with their own independent configs. This kind of approach will provide the +required flexibility for users to achieve any number of configurations. + +Problem description +=================== + +Users wishing to configure a StarlingX host as a T-GM or T-BC PTP node have +very limited configuration options and may not be able to achieve the desired +config due to how PTP is currently implemented in StarlingX. The current +implementation only supports a single instance of ptp4l and phc2sys +on each host. The new ts2phc application is not currently supported at all. +Additionally, many of the config options for these services apply across all +hosts and do not allow for host-specific settings. Finally, some parameters for +the PTP applications are not configurable at all - specifically interface level +parameters and command line flags. + +Use Cases +--------- + +* End user wants to configure multiple instances of ptp4l, phc2sys and ts2phc + to achieve a T-BC (boundary clock) or T-GM (grand leader) configuration. +* End user wants full access to all of the program options for each of these + services and the ability to configure them on a per-instance, per-interface + and per-host basis. + + +Proposed change +=============== + +Updating linuxptp + + +The existing linuxptp package installed in StarlingX is version 2.0, which does +not include the ts2phc utility that users may want for certain configurations. +The installed package will be updated to the latest 3.1.1 version. + +Rework of CLI and Data Model + +The existing CLI and data model is not flexible enough to support multiple +instances of the linuxptp services and also does not provide access to all of +the configuration parameters for these services. It will be necessary to +develop new CLI commands and a new data model which will handle each instance +independently and allow a user to deploy a variety of configurations. + +Alternatives +------------ + +This work is an extension of StarlingX's suppport for PTP. The existing model +has been in place since at least STX 4.0 and has been utilized by many end +users. During this time, new use cases have been identified which can be met +with the linuxptp application, but not with StarlingX because of the +limitations that the platform places on congfiguration options. Alternatives +for linuxptp have not been examined, as it continues to meet the needs of +users. + +Regarding alternatives to the approach described in this spec, the goal of this +work is create a fully flexible and user-defined PTP interface where any +current and future use cases can easily be supported. + +Data model impact +----------------- + +A new group of tables will be introduced to handle the PTP configurations. The +diagram below shows the new table names and relationships. + + +.. + .------------------------. + | PtpParameterOwnerships | ("bridge" table) + .------------------------. + | + | + +---------------+ | +--------------------+ + | |N | N| | + | PtpParameters |---------------| PtpParameterOwners | + | | | | + +---------------+ +--------------------+ + ^ + /_\ + | + +--------+--------+ + | | + +--------------+ +---------------+ + | | | | + | PtpInstances |<------| PtpInterfaces | + | | | | + +--------------+ +---------------+ + N| N| + | | + N| N| + +-------+ +------------+ + | Hosts | | Interfaces | + +-------+ +------------+ + +The Hosts and Interfaces tables are pre-existing and used to map data into +PtpInstances and PtpInterfaces respectively. + +The PtpInstances table contains the instance name, type and host(s) it is +associated with. + +PtpInterfaces contains the PTP interface name, which PtpInstance owns it, and +which physical interface(s) are associated with it. + +PtpParameters contains all of the key/value parameters. To avoid duplicate +entries the PtpParameterOwnerships and PtpParameterOwners tables are used to +map the associaton of parameters to PtpInstances and PtpInterfaces. Given the +hierarchy of the tables, it is not necessary to implement logic to decide upon +the actual owner, leaving such resolution for SQLAlchemy to manage. + +Migrations will be provided to copy existing PTP configurations from +the existing tables to the new ones. The relevant tables that will supply data +for this are the 'ptp' table and the 'service-parameters' table. + +REST API impact +--------------- + +The following REST API endpoints will be deprecated: + * /v1/ptp (GET) + * /v1/ptp/{ptp_id} (PATCH) + * /v1/ptp/apply (POST) + +The new paths for PTP are: + +/v1/ptp_instances +_________________ + +GET /v1/ptp_instances + +RESP: + +.. + { + "ptp_instances": [ + { + "uuid": "45576c98-caf6-4ced-a323-067bdd472bf2", + "service": "ptp4l", + "created_at": "2022-03-22T17:05:46.382740+00:00", + "updated_at": null, + "capabilities": {}, + "hostnames": [ + "controller-0" + ], + "parameters": [ + "BMCA=noop", + "domainNumber=0", + "tx_timestamp_timeout=700", + "slaveOnly=1", + "boundary_clock_jbod=1" + ], + "type": "ptp-instance", + "id": 3, + "name": "ptp1" + }, + ] + } + +POST /v1/ptp_instances + +.. + { + "name": "ptp1", + "service": "ptp4l" + } + + +RESP: + +.. + { + "uuid": "19fa712a-1296-486c-92a8-f84d0887352e", + "service": "ptp4l", + "created_at": "2022-03-25T19:03:19.162668+00:00", + "updated_at": null, + "capabilities": {}, + "hostnames": [], + "parameters": [], + "type": "ptp-instance", + "id": 1, + "name": "ptp1" + } + +/v1/ptp_instances/{ptp_instance_id} +___________________________________ + +GET /v1/ptp_instances/45576c98-caf6-4ced-a323-067bdd472bf2 + +RESP: + +.. + { + "uuid": "45576c98-caf6-4ced-a323-067bdd472bf2", + "service": "ptp4l", + "created_at": "2022-03-22T17:05:46.382740+00:00", + "updated_at": null, + "capabilities": {}, + "hostnames": [ + "controller-0" + ], + "parameters": [ + "domainNumber=0", + "slaveOnly=1", + "boundary_clock_jbod=1", + "BMCA=noop", + "tx_timestamp_timeout=700" + ], + "type": "ptp-instance", + "id": 3, + "name": "ptp1" + } + +PATCH /v1/ptp_instances/19fa712a-1296-486c-92a8-f84d0887352e + +.. + [ + { + "path": "/ptp_parameters/-", + "value": "domainNumber=24", + "op": "add" + } + ] + +RESP: + +.. + { + "uuid": "19fa712a-1296-486c-92a8-f84d0887352e", + "service": "ptp4l", + "created_at": "2022-03-25T19:03:19.162668+00:00", + "updated_at": null, + "capabilities": {}, + "hostnames": [], + "parameters": [ + "domainNumber=24" + ], + "type": "ptp-instance", + "id": 1, + "name": "ptp1" + } + +/v1/ptp_interfaces +__________________ + +GET /v1/ptp_interfaces + +RESP: + +.. + { + "ptp_interfaces": [ + { + "ptp_instance_uuid": "45576c98-caf6-4ced-a323-067bdd472bf2", + "interface_names": [ + "controller-0/ens2f1" + ], + "ptp_instance_id": 3, + "uuid": "b4485c06-abf4-4686-9b1b-e514b48740d4", + "parameters": [], + "created_at": "2022-03-22T17:27:48.060172+00:00", + "updated_at": null, + "capabilities": {}, + "hostnames": [ + "controller-0" + ], + "ptp_instance_name": "ptp1", + "type": "ptp-interface", + "id": 5, + "name": "n2_p2" + } + ] + } + +POST /v1/ptp_interfaces + +.. + { + "ptp_instance_uuid": "19fa712a-1296-486c-92a8-f84d0887352e", + "name": "int1" + } + +RESP: + +.. + { + "ptp_instance_uuid": "19fa712a-1296-486c-92a8-f84d0887352e", + "interface_names": [], + "ptp_instance_id": 1, + "uuid": "53efedee-770f-40ec-8a4b-2c80fcc7e1dc", + "parameters": [], + "created_at": "2022-03-25T19:06:22.567333+00:00", + "updated_at": null, + "capabilities": {}, + "hostnames": [], + "ptp_instance_name": "ptp1", + "type": "ptp-interface", + "id": 2, + "name": "int1" + } + +/v1/ptp_interfaces/{ptp_interface_id} +_____________________________________ + +GET /v1/ptp_interfaces/07258277-e123-4362-95b5-8b7372daf39f + +RESP: + +.. + { + "ptp_instance_uuid": "7323768f-6485-47da-904b-7fba889806fc", + "interface_names": [ + "controller-0/ens2f1" + ], + "ptp_instance_id": 6, + "uuid": "07258277-e123-4362-95b5-8b7372daf39f", + "parameters": [], + "created_at": "2022-03-22T17:35:55.384720+00:00", + "updated_at": null, + "capabilities": {}, + "hostnames": [ + "controller-0" + ], + "ptp_instance_name": "phc1", + "type": "ptp-interface", + "id": 7, + "name": "phc1_if" + } + +DELETE /v1/ptp_interfaces/53efedee-770f-40ec-8a4b-2c80fcc7e1dc + +RESP: Empty + + + +PATCH /v1/ptp_interfaces/cee22189-53cd-463f-aaac-1bf744af1eb8 + +.. + [ + { + "path": "/ptp_parameters/-", + "value": "domainNumber=24", + "op": "add" + } + ] + +RESP: + +.. + { + "ptp_instance_uuid": "19fa712a-1296-486c-92a8-f84d0887352e", + "interface_names": [ + "controller-0/oam0" + ], + "ptp_instance_id": 1, + "uuid": "cee22189-53cd-463f-aaac-1bf744af1eb8", + "parameters": [ + "domainNumber=24" + ], + "created_at": "2022-03-25T19:09:56.625744+00:00", + "updated_at": null, + "capabilities": {}, + "hostnames": [ + "controller-0" + ], + "ptp_instance_name": "ptp1", + "type": "ptp-interface", + "id": 3, + "name": "int1" + } + +/v1/ihosts/{ihost_uuid}/ptp_instances +_____________________________________ + +GET /v1/ihosts/e68ae900-2eef-4481-b865-b8dee686f028/ptp_instances + +RESP: + +.. + { + "ptp_instances": [ + { + "uuid": "2878d2e5-e9d7-489d-823f-1fa01ca6b0a5", + "service": "clock", + "created_at": "2022-03-24T15:05:17.426493+00:00", + "updated_at": null, + "capabilities": {}, + "hostnames": [ + "controller-0" + ], + "parameters": [], + "type": "ptp-instance", + "id": 8, + "name": "clock1" + } + ] + } + + +/v1/ihosts/{host_id}/ptp_interfaces +___________________________________ +GET /v1/ihosts/e68ae900-2eef-4481-b865-b8dee686f028/ptp_interfaces + +RESP: + +.. + { + "ptp_interfaces": [ + { + "ptp_instance_uuid": "45576c98-caf6-4ced-a323-067bdd472bf2", + "interface_names": [ + "controller-0/ens2f1" + ], + "ptp_instance_id": 3, + "uuid": "b4485c06-abf4-4686-9b1b-e514b48740d4", + "parameters": [], + "created_at": "2022-03-22T17:27:48.060172+00:00", + "updated_at": null, + "capabilities": {}, + "hostnames": [ + "controller-0" + ], + "ptp_instance_name": "ptp1", + "type": "ptp-interface", + "id": 5, + "name": "n2_p2" + } + ] + } + +/v1/iinterfaces/{interface_id}/ptp_interfaces +_____________________________________________ +GET /v1/iinterfaces/4c3a3435-7275-4a4d-b52b-2dfa9759bda7/ptp_interfaces + +RESP: + +.. + { + "ptp_interfaces": [ + { + "ptp_instance_uuid": "19fa712a-1296-486c-92a8-f84d0887352e", + "interface_names": [ + "controller-0/oam0" + ], + "ptp_instance_id": 1, + "uuid": "cee22189-53cd-463f-aaac-1bf744af1eb8", + "parameters": [], + "created_at": "2022-03-25T19:09:56.625744+00:00", + "updated_at": null, + "capabilities": {}, + "hostnames": [ + "controller-0" + ], + "ptp_instance_name": "ptp1", + "type": "ptp-interface", + "id": 3, + "name": "int1" + } + ] + } + +/v1/ptp_parameters +__________________ + +GET /v1/ptp_parameters + +RESP: + +.. + { + "ptp_parameters": [ + { + "owners": [ + "45576c98-caf6-4ced-a323-067bdd472bf2" + ], + "name": "BMCA", + "created_at": "2022-03-22T17:06:02.042825+00:00", + "updated_at": null, + "value": "noop", + "uuid": "656b2d30-8f6a-4d7d-abc9-f6c2fa2a2eb0" + }, + { + "owners": [ + "45576c98-caf6-4ced-a323-067bdd472bf2" + ], + "name": "boundary_clock_jbod", + "created_at": "2022-03-22T17:06:00.793710+00:00", + "updated_at": null, + "value": "1", + "uuid": "ab41d9f7-5b95-4673-9cdb-120f73afe32c" + } + ] + } + +/v1/ptp_parameters/{ptp_parameter_id} +_____________________________________ + +GET /v1/ptp_parameters/061dbf29-e38d-4249-8317-f53ca2d74b2b + +RESP: + +.. + { + "owners": [ + "ffd00463-4291-412b-8391-f6100dc84b60" + ], + "uuid": "061dbf29-e38d-4249-8317-f53ca2d74b2b", + "created_at": "2022-03-24T15:25:25.502121+00:00", + "updated_at": null, + "value": "input", + "id": 8, + "name": "sma1" + } + +PATCH /v1/ptp_parameters/1a0615ee-c2db-4b93-857b-88c71f2acf90 + +.. + [ + { + "path": "/value", + "value": "25", + "op": "replace" + } + ] + +RESP: + +.. + { + "owners": [ + "cee22189-53cd-463f-aaac-1bf744af1eb8" + ], + "uuid": "1a0615ee-c2db-4b93-857b-88c71f2acf90", + "created_at": "2022-03-25T19:33:26.074442+00:00", + "updated_at": null, + "value": "25", + "id": 3, + "name": "domainNumber" + } + + +/v1/ptp_instances/apply +_______________________ + +POST /v1/ptp_instances/apply +RESP: None + +Security impact +--------------- + +PTP functionality impacts system time, so could carry with it the risk of +system instability due to poor configuration. The new multi-instance approach +will expose a wide range of configuration options compared to limited amount +currently available. Users will have to take care to read the documentation and +become familiar with the operation of the linuxptp applications in order to set +up PTP. + + +Other end user impact +--------------------- + +For upgrading from a previous release, users should be aware that existing PTP +configuration in the PTP table and in the service-parameters table will be +migrated to the new tables. Systems that have PTP configured will create +ptp4l-legacy and phc2sys-legacy instances to hold this prior configuration. +While the existing database entries will remain present, the CLIs and APIs used +to configure the old single instance PTP will be deprecated. + +Developer impact +---------------- + +None + +Upgrade impact +-------------- + +See "Other end user impact" above for upgrade related details. + +Implementation +============== + +Assignee(s) +----------- + +Primary assignee: + Cole Walker + +Other contributors: + + Douglas Koerich + + Teresa Ho + +Repos Impacted +-------------- + +starlingx/clients + +starlingx/config + +starlingx/gui + +starlingx/integ + +starlingx/monitoring + +starlingx/root + +starlingx/stx-puppet + +Work Items +---------- + +* Create new database entries for PTP tables +* SQLAlchemy handling for tables +* Update REST API / deprecate old PTP +* Update CLI / deprecate old PTP +* Generate Puppet hieradata from database inputs +* Create puppet manifests to apply configurations +* Update stx-platformclients container image +* Alarming and monitoring support + + +Dependencies +============ + +None + +Testing +======= + +Unit tests will be created for APIs, as well as for the hieradata generation. + +Functional testing will be performed on hardware meeting the requirements of +having at least two PTP capable NICs per node in AIO-SX, AIO-DX, Standard +and DC configurations. Scenarios will include testing T-GM, T-BC and OC PTP +configurations in order to test a wide range of linuxptp behaviours. Testing +will be completed on several different NIC types. + +Existing regression testing for the ptp-notification feature should be run +during this period in order to verify that the application still functions as +expected. It is possible that updates to the ptp-notification app may be +required as part of this work. + + +Documentation Impact +==================== + +This work will primarily impact deployers and end users. The End User +documentation will need to be updated to remove old PTP instructions and +provide guidance on how to set up PTP with the new CLI commands. + +The API reference documentation will also have to be updated with new paths and +to indicate the deprecated ones. + +StarlingX release notes describing the PTP changes will also be provided. + +References +========== + +None + +History +======= + +None