Commit Graph

198 Commits

Author SHA1 Message Date
Sachin Gopala Krishna 126b37dfe8 remove support for versions of K8s lower than 1.24
Remove support for k8s versions 1.23 and lower since they are not
supported. This change removes k8s versions 1.21.8,
1.22.5, 1.23.1 from the build

Test Plan:
Pass: Tested by successfully creating and installing ISO on AIO-SX.
Pass: Verify /usr/local/kubernetes/ doesn't contain k8s versions
1.21.8, 1.22.5 and 1.23.1.
Pass: Perform platform upgrade, then k8s upgrade to v1.26.1.

Story: 2010368
Task: 48240

Depends-On: https://review.opendev.org/c/starlingx/config/+/886188

Change-Id: If3b9934937c542074ebbcb23d49a5fd4c7e69898
Signed-off-by: Sachin Gopala Krishna <saching.krishna@windriver.com>
Signed-off-by: Boovan Rajendran <boovan.rajendran@windriver.com>
2023-12-15 03:19:39 +00:00
Scott Little 2d3951f713 Config file changes to remove 'virt/qemu virt/libvirt '
Qemu and libvirt have been relocated to 'cgcs-root/stx/virt'

Required all reviews in the topic:
   https://review.opendev.org/q/topic:virt-repo

Testing:
build-pkgs -c -a: pass
build-pkgs -p libvirt,qemu: pass

Story: 2010317
Task: 47780
Signed-off-by: Scott Little <scott.little@windriver.com>
Change-Id: If81d59698b97e27a8013b2a0b93e2fadc0186205
2023-04-12 15:21:49 -04:00
Rafael Cardoso Pereira 5385bad154 Removing unused package python-ryu
After performing an analysis of the system it was recognized
that the following package: python-ryu; is not being used anymore
by the system. In order to clean it up, it was decided to
remove the package.

Test Plan:
PASS - All pkgs built successfully after removal of python-ryu
PASS - Successfully generated an openstack tarball
PASS - Stx-Openstack tarball successfully applied
PASS - Built CentOS ISO with the change and applied it to a lab

Closes-bug: #1985091

Signed-off-by: Rafael Cardoso Pereira <rafael.cardosopereira@windriver.com>
Change-Id: I399896a24204d618a535e874716eadf8889eec8d
2022-08-12 17:53:49 +00:00
Zuul 758b223090 Merge "Enable puppet-firewall parsing of --random-fully rules" 2022-05-16 20:03:17 +00:00
Jerry Sun d2acaea1ef Add Kubectl Cert Manager
This commit adds the kubernetes plugin kubectl cert manager to the iso.
This is used to convert old v1alpha2 and v1alpha3 cert manager
resources to v1 during a system upgrade. The plugin is not required
for debian because there are no old cert manager resources to convert.

Test Cases:

PASS: Convert our default DC certificates and issuers using
      kubectl cert manager

Change-Id: I59f1b0e4d5d6ece1ccef43fee1acacd7b7e44efd
Story: 2009837
Task: 45372
Signed-off-by: Jerry Sun <jerry.sun@windriver.com>
2022-05-16 11:56:14 -04:00
Steven Webster 5d51ff6dd7 Enable puppet-firewall parsing of --random-fully rules
A problem may occur if puppet attempts to inject a firewall rule
while the underlying iptables/ip6tables has existing rules which
use the --random-fully flag in the NAT table.

The issue occurs because puppet-firewall first makes a call to
iptables-save/ip6tables-save to parse the existing rules
(to determine if the rule already exists).  If it finds a rule
with --random-fully, it will immediately bail out.

The current version(s) of puppet-firewall in StarlingX are old
enough that they don't have parsing logic for the --random-fully
flag that was initially supported in iptables version 1.6.2+.

Now that StarlingX uses iptables 1.8.4, we must account for the
possibility that various components (ie. kubernetes) will make
use of --random-fully rules.

This feature has been implemented upstream in the following commits:

https://github.com/puppetlabs/puppetlabs-firewall/commits/

9a4bc6a81cf0cd4a56ba458fadac830a2c4df529
0ea2b74c0b4a451a37bae8c2ff105b72481ab485

The above commits have been ported back to:

CentOS: puppet-firewall-1.8.2
Debian: puppetlabs-firewall-1.12.0

Since StarlingX does not currently build it's own version
of puppet-firewall in either CentOS or Debian, this commit
also contains the infrastructure to do so.

Testing:

Note: Since the issue is intermittent on unlock, the functional
tests were performed with a custom runtime manifest that installed
a dummy iptables/ip6tables rule when an interface was modified.
At this time, it was guaranteed that there were rules with
the --random-fully flag present.

CentOS:

Package build: PASS
Present in iso: PASS
IPv4 functional test (iptables): PASS
IPv6 functional test (ip6tables): PASS

Debian:

Package build: PASS
Present in iso: PASS
IPv4 functional test (iptables): PASS
IPv6 functional test (ip6tables): PASS

Closes-Bug: #1971900

Signed-off-by: Steven Webster <steven.webster@windriver.com>
Change-Id: I7dbb9e1b99d95df0aa5a7db7aa22c3c314253788
2022-05-10 11:17:08 -04:00
Gleb Aronsky 79db1f3eed Add Kubernetes 1.23.1 pkg
Changes for adding Kubernetes 1.23.1 in
StarlingX, including build environment updates.

The package builds successfully.
Built and installed an iso with K8s 1.23.1 on
AIO-SX.

Depends-On: https://review.opendev.org/c/starlingx/compile/+/825651
Story: 2009830
Task: 44424

Change-Id: I3e2b793d7b88057fc597b2445bddd137bb2b4fcf
Signed-off-by: Gleb Aronsky <gleb.aronsky@windriver.com>
2022-03-11 12:29:41 -05:00
Daniel Safta c66d407614 Remove kubernetes 1.18, 1.19, 1.20 pkgs
The new minimum supported k8s version
will be 1.21. This commit cleans the pkg
files needed to build the old k8s versions.

The pkgs build successfully. Deployed on
AIO-SX and AIO-DX, the k8s services were running ok.

Story: 2009859
Task: 44498
Change-Id: Ib39e9d1522a49c5788240781c8edee2bdffbc97a
Signed-off-by: Daniel Safta <daniel.safta@windriver.com>
2022-02-23 15:24:03 +00:00
Steven Webster d900a5b646 Add bond-cni container network plugin
This commit adds the bonding CNI plugin to StarlingX.

The bonding CNI plugin allows a container to bond multiple
interfaces together to be used in a fail-over or load
balancing configuration.

https://github.com/k8snetworkplumbingwg/bond-cni

Note that this plugin (for now) resides outside of the
main containernetwork project, and is still part of the
k8s network plumbing working group project.  As such,
it is required to build this plugin separately.

v1.0 of the bond-cni was released in 2018.  Since then,
14 commits containing such things as doc clean-ups and
bug fixes have been committed.  We pick up these additional
fixes by clamping down on the latest commit SHA.

Testing:

- Configure bond interface name (ifName)
- Configure miimon value (miimon)
- Configure and verify traffic path for modes (mode):
  - balance-rr (0)
  - active-backup (1)
  - balance-xor (2)
  - broadcast (3)
  - 802.3ad (4)
  - balance-tlb (5)
  - balance-alb (6)
- Configure and verify behaviour for fail-over-mac
  modes (failOverMac):
  - none (0)
  - active (1)
  - follow (2)
- Configure linksInContainer:
  - take lower interfaces existing on host
  - take lower interfaces existing on container
- Links tested:
  - virtual interfaces
  - SR-IOV VF interfaces

Story: 2009800
Task: 44344

Change-Id: I7bffaa272ffe9eba85c3aa0a26b9c4f61428b640
Signed-off-by: Steven Webster <steven.webster@windriver.com>
2022-02-01 09:46:17 -05:00
Daniel Safta a046a7a650 Add kubernetes 1.22.5 pkg
Here are the changes needed for adding k8s v1.22.5
in StarlingX alongside with the changes needed
for the build environment to find and build the package.

The package builds successfully.
Deployed an iso with k8s 1.22.5 on
AIO-SX and AIO-DX. The deployment phase
works and the pods are up and running after
the upgrade completes.

Story: 2009789
Task: 44305
Signed-off-by: Daniel Safta <daniel.safta@windriver.com>
Change-Id: Ibb9be075fa0b1491b9ab1854ebb1fddf4df53461
2022-01-28 11:56:47 +02:00
Chris Friesen df3b902e6b Replace K8s 1.21.3 with 1.21.8
In testing K8s 1.21.8 used less CPU than 1.21.3, so we are moving to
the newer version.

This has been booted in vbox and a basic pod has been started.
A full regression will be performed.

Depends-On: https://review.opendev.org/c/starlingx/compile/+/824802

Partial-Bug: 1957994
Signed-off-by: Chris Friesen <chris.friesen@windriver.com>
Change-Id: I64e4a64c90ef7591aeee52742dfcba9fdd8e5063
2022-01-14 16:57:25 -05:00
Jim Somerville 2b7072eb11 tzdata: upversion to 2021e and start building
We need to install the leap-seconds.list file as
the prebuilts don't.  This file is needed for
later versions of ptp.

Verification:
- tzdata package now builds
- check built package to ensure it contains the
  leap-seconds.list file
- build-iso and make sure it contains the new rpm
- boot the iso and ensure nothing weird observed
  regarding the date
- run "export TZ=/usr/share/zoneinfo/EST5EDT" followed
  by the date command and ensure that it displays the
  correct time for that timezone

Story: 2009130
Task: 44276
Change-Id: I57ce64d49cbf3f6a1de95aa7df462f7ae9daa1ad
Signed-off-by: Jim Somerville <Jim.Somerville@windriver.com>
2022-01-12 17:32:24 -05:00
M. Vefa Bicakci 6da5c5ecbe networking: Update keepalived from 1.3.5 to 2.1.5
This commit updates keepalived from v1.3.5 to v2.1.5 to avoid failures
encountered when building StarlingX flock container images, which
started to fail with the following errors after the recent iptables
update introduced by commit 36673774ee ("iproute-5.12, iptables-1.8.4,
and libnftnl-1.1.5", 2021-10-27):

=== 8< ===
Error: Package: keepalived-1.3.5-19.el7.x86_64 (base)
       Requires: libxtables.so.10()(64bit)
       Available: iptables-1.4.21-35.el7.x86_64 (base)
           libxtables.so.10()(64bit)
       Installing: iptables-1.8.4-21.tis.5.x86_64 (stx-mirror-distro)
           Not found
=== >8 ===

keepalived-2.1.5 was imported from CentOS 8-Stream where it is the
latest version as of this writing. It should be noted that rebuilding
keepalived-1.3.5 (i.e., CentOS 7's version) was not suitable as
keepalived-1.3.5 does not support iptables-nftables, and the CentOS
8-Stream keepalived RPM cannot be used as is due to the specific
versions of some of its dependencies.

During the preparation of this patch, an unexpected build failure had to
be worked around by disabling SNMP support in keepalived, which is
assumed to not have a negative impact on StarlingX according to a
software architect colleague at Wind River. Please see the description
of the patch named "keepalived.spec-Disable-dependency-on-snmp.patch"
for further details regarding the build failure.

Verification:
- Layered and monolithic StarlingX master branch builds succeed.
- StarlingX container builds, which used to fail without this commit,
  succeed as well.

Closes-Bug: #1950513

Signed-off-by: M. Vefa Bicakci <vefa.bicakci@windriver.com>
Change-Id: I59bd7d4f8ed89c73248ecd97e6985f91b88c4623
2021-11-12 12:12:16 -05:00
Zuul 79355056dc Merge "Revert "Add centos package for puppet-postgresql module"" 2021-11-11 15:32:02 +00:00
Daniel Stevens Torres Cardenas b36ef4fab8 Revert "Add centos package for puppet-postgresql module"
This reverts commit d27206ec92.

Reason for revert: We have a red sanity and we believe it this is the commit that is causing ansible to fail.

Change-Id: Ia0ef3bb302be88e04849cd0343fd849895c455f0
2021-11-11 15:04:18 +00:00
Zuul a5674d5141 Merge "Add centos package for puppet-postgresql module" 2021-11-10 17:00:27 +00:00
Roberto Nogueira d27206ec92 Add centos package for puppet-postgresql module
This package is 3rdparty.
All patches were changed and lifted.
Did build puppet-postgresql.

Story: 2009242
Task: 43883

Signed-off-by: Roberto Nogueira <robertoluiz.martinsnogueira@windriver.com>
Change-Id: I1d473e34c703c6355bee4f33daf6ce12b71b4d19
2021-11-09 19:57:09 +00:00
Zuul 4bc0c7dd3d Merge "Implement CNI cache file cleanup for stale files" 2021-11-04 17:22:43 +00:00
Steven Webster 5d1a26b89d Implement CNI cache file cleanup for stale files
It has been observed in systems running for months -> years
that the CNI cache files (representing attributes of
network attachment definitions of pods) can accumulate in
large numbers in the /var/lib/cni/results/ and
/var/lib/cni/multus/ directories.

The cache files in /var/lib/cni/results/ have a naming signature of:

<type>-<pod id>-<interface name>

While the cache files in /var/lib/cni/multus have a naming signature
of:

<pod id>

Normally these files are cleaned up automatically (I believe
this is the responsibility of containerd).  It has been seen
that this happens reliably when one manually deletes a pod.

The issue has been reproduced in the case of a host being manually
rebooted.  In this case, the pods are re-created when the host comes
back up, but with a different pod-id than was used before

In this case, _most_ of the time the cache files from the previous
instantiation of the pod are deleted, but occasionally a few are
missed by the internal garbage collection mechanism.

Once a cache file from the previous instantiation of a pod escapes
garbage collection, it seems to be left as a stale file for all
subsequent reboots.  Over time, this can cause these stale files
to accumulate and take up disk space unnecessarily.

The script will be called once by the k8s-pod-recovery service
on system startup, and then periodically via a cron job installed
by puppet.

The cleanup mechanism analyzes the cache files by name and
compares them with the id(s) of the currently running pods. Any
stale files detected are deleted.

Test Plan:

PASS: Verify existing pods do not have their cache files removed
PASS: Verify files younger than the specified 'olderthan' time
      are not removed
PASS: Verify stale cache files for pods that do not exist anymore
      are removed.
PASS: Verify the script does not run if kubelet is not up yet.

Failure Path:

PASS: Verify files not matching the naming signature (pod id
      embedded in file name) are not processed

Regression:

PASS: Verify system install
PASS: Verify feature logging

Partial-Bug: 1947386

Signed-off-by: Steven Webster <steven.webster@windriver.com>
Change-Id: I0ce06646001e52d1cc6d204b924f41d049264b4c
2021-11-01 10:39:39 -04:00
M. Vefa Bicakci 36673774ee iproute-5.12, iptables-1.8.4, and libnftnl-1.1.5
This commit updates iproute from 5.9 to 5.12, iptables from 1.4.21 to
1.8.4, and libnftnl from 1.0.8 to 1.1.5:

- iproute 5.9 does not make use of libbpf, which causes the 'tc' utility
  (provided by iproute-tc) to report BTF debugging symbol-related
  warnings when eBPF programs are used with tc by the kernel's eBPF
  sample test programs, even though the programs appear to work: "BTF
  debug data section '.BTF' rejected: Invalid argument (22)!".

- iptables 1.4.21 does not support the --object-pinned option, which is
  required to be able to use eBPF programs to match packets.

- libnftnl >= 1.1.5 is a dependency for recent versions of iptables, and
  the version of libnftnl in StarlingX's CentOS 7 is 1.0.8.

The versions which are used by this commit are the latest versions in
CentOS 8-Stream as of this writing.

Notes:
- iptables software package bundles a version of ebtables different than
  the legacy version already included in StarlingX. The legacy version
  supports the broute table and the BROUTING chain and string matching,
  whereas the iptables version does not. The legacy version is
  deprecated by this commit based on feedback received from colleagues,
  mainly to avoid unexpected incompatibilities between ebtables-legacy
  and iptables' netfilter/nft-based versions.

Verification:
- All-in-One simplex installation and bootstrap was carried out
  successfully.
- Installation and bootstrap was successful on two separate systems: One
  system consisting of 2 controller hosts, 4 compute hosts and 2 storage
  hosts, and another system consisting of 2 controller hosts and 2
  compute hosts.
- Configuration of aggregated links (after using ifenslave manually) and
  configuration of virtual function (VF) interfaces (also manually set
  up) were carried out with the iproute tools successfully as basic
  sanity tests.
- The results of basic ebtables commands (insertion and removal of DROP
  rules) were observed in "ebtables -L" output and confirmed to take
  effect in a test bed consisting of two network namespaces connected by
  bridged interfaces, as a basic sanity test.
- Sample eBPF test programs and scripts shipped with the v5.10 kernel
  were executed successfully, with the caveat that there is a need to
  install a recent version of LLVM to compile the eBPF test programs.
  (I built LLVM-13.0 from scratch.)

Partial-Bug: #1949217

Depends-On: I24bb7c60e353643add5e63ae7ea7c6516d07c7bf
Depends-On: I12d20797db91fecdac409b0535632ac97bd6ad47
Depends-On: If95c2d24c98cb2add5e24548bc45f505c94b4b79

Signed-off-by: M. Vefa Bicakci <vefa.bicakci@windriver.com>
Change-Id: I63d557112c653d59b88ac3a4798dee0e89246612
2021-10-29 19:47:27 -04:00
M. Vefa Bicakci ccf50bd652 Add libbpf v0.5.0
This commit adds libbpf-0.5.0, because recent versions of iproute depend
on it. 0.5.0 is the latest released version as of this writing.

Verification:
- Successfully built in a monolithic build environment.
- In conjunction with I63d557112c653d59b88ac3a4798dee0e89246612, the
  sample eBPF test programs and scripts in the v5.10 kernel's source
  tree were successfully executed.

Partial-Bug: #1949217

Signed-off-by: M. Vefa Bicakci <vefa.bicakci@windriver.com>
Change-Id: I24bb7c60e353643add5e63ae7ea7c6516d07c7bf
2021-10-29 17:23:16 -04:00
M. Vefa Bicakci a7760b40a2 iproute: Update from 4.11.0-14 to 5.9.0-4
This commit updates the iproute package from 4.11.0-14 shipped with
CentOS 7.6.1810 (i.e., StarlingX baseline) to 5.9.0-4 shipped with
CentOS 8.4.2105, because the former version does not support the "seg6"
and "seg6local" encapsulation types (used for segment-based routing via
the "ip route" command).

Segment-based routing capability was requested by a user, and the
shortcomings of the pre-existing version of iproute package were noticed
when attempting to test the seg6 encapsulation type with a v5.10-based
kernel.

Note that it was not possible to re-use CentOS 8.4.2105's binary RPM
package on StarlingX, due to the fact that StarlingX's CentOS 7.6.1810
baseline ships with glibc-2.17, whereas the CentOS 8.4.2105 iproute
package depends on glibc-2.27. This requires StarlingX to rebuild the
RPM package.

Verification:
- Installation and bootstrap of an All-in-One simplex system has been
  carried out with this commit as an overall regression test.

- The iproute package's git repository was also browsed for potentially
  non-backwards-compatible changes by searching for keywords "backward"
  and "compat" in the git commit history between versions 4.11.0 and
  5.9.0 using "git log -i --grep".

Story: 2008921
Task: 43663
Depends-On: I5e272dc59b8b69611474706c165644a8dd5d7f52

Signed-off-by: M. Vefa Bicakci <vefa.bicakci@windriver.com>
Change-Id: I6de9659dfec830f954661a0b0f82e69dc9637a5d
2021-10-22 16:37:44 -04:00
M. Vefa Bicakci 7e10236038 Update kexec-tools/makedumpfile to support v5.10 kernel
This patch updates kexec-tools from 2.0.15 to 2.0.21 (and its supporting
software package makedumpfile from 1.6.2 to 1.6.9) for compatibility
with the newer v5.10 kernel.

This commit clones the kexec-tools package's supporting files from
commit 26a7a543427eac59ed39728466f3d95d320f735a in the CentOS RPM
packaging git repository. Links for reference:

- 26a7a54342?branch=c7
- 26a7a54342

Please note that this patch causes the build system to pull in and
extract an SRPM file to acquire:
  kdump-anaconda-addon-003-29-g4c517c5.tar.gz

This is done for security, because the only public reference to commit
4c517c5 is on a Red Hat developer's personal Github account:
  https://github.com/ryncsn/kdump-anaconda-addon/commits/rhel-7

kexec-tools package's supporting files cloned by this commit trigger a
large number of shell script linting errors. Given that the shell
scripts in question are inherited from upstream (i.e., CentOS 7), the
"files" directory of this package is excluded from automated linting via
the changes in tox.ini.

Verification: A kexec-tools RPM package built with this commit was
installed onto an existing StarlingX system. A vmcore file was
succesfully collected from a kernel crash triggered with
/proc/sysrq-trigger. A recent version of the crash utility was found to
succesfully parse the collected vmcore file.

Credits: Thanks to Jiping Ma for helping with cleaning up and publishing
an earlier version of this patch.

Story: 2008921
Task: 43040

Depends-On: https://review.opendev.org/c/starlingx/tools/+/805127

Signed-off-by: Jiping Ma <jiping.ma2@windriver.com>
Signed-off-by: M. Vefa Bicakci <vefa.bicakci@windriver.com>
Change-Id: Idc4e523610e4c09259300c8b67ea5e0fbe59c611
2021-10-19 14:29:42 -04:00
Jim Gauld 872dd513fc Add staged kubernetes version 1.21.3
Multiple versions of kubernetes are required to support upgrade.

This adds staged version of kubernetes 1.21.3, built with a
specific version of golang.

All subpackage versions are included in the iso image without
collisions.

The following patches are ported to specific kubernetes version:
kubelet-cpumanager-disable-CFS-quota-throttling-for-.patch
kubelet-cpumanager-keep-normal-containers-off-reserv.patch
kubelet-cpumanager-infrastructure-pods-use-system-re.patch
kubelet-cpumanager-introduce-concept-of-isolated-CPU.patch
kubeadm-create-platform-pods-with-zero-CPU-resources.patch
enable-support-for-kubernetes-to-ignore-isolcpus.patch

The following changes were made for 1.21.3:
- following upstream commit was reverted:
  Revert-use-subpath-for-coredns-only-for-default-repo.patch

- kubelet-cpumanager-disable-CFS-quota-throttling-for-.patch
  was refactored due to new internal_container_lifecycle framework
  We leverage the same mechanism to set Linux resources as:
  cpu manager: specify the container CPU set during the creation
  (commit 38dc7509f862f081828e7d9167107b8c6e98ea23).

- kubelet-cpumanager-introduce-concept-of-isolated-CPU.patch
  was refactored due to upstream API change:
  node: podresources: make GetDevices() consistent
  (commit ad68f9588c72d6477b5a290c548a9031063ac659).

  The routine podIsolCPUs() was refactored in 1.21.3 since the
  API p.deviceManager.GetDevices() is returning multiple devices
  with a device per cpu. The resultant cpuset needs to be the
  aggregate.

Story: 2008972
Task: 43056

Signed-off-by: Jim Gauld <james.gauld@windriver.com>
Change-Id: I5ba7ff2e6aebb744af265698c0f90256ac5e70f4
2021-09-22 16:31:39 -04:00
Jim Gauld 94517398e4 Add staged kubernetes version 1.20.9
Multiple versions of kubernetes are required to support upgrade.

This adds staged version of kubernetes 1.20.9, built with a
specific version of golang.

All subpackage versions are included in the iso image without
collisions.

The following patches are ported to specific kubernetes version:
kubelet-cpumanager-disable-CFS-quota-throttling-for-.patch
kubelet-cpumanager-keep-normal-containers-off-reserv.patch
kubelet-cpumanager-infrastructure-pods-use-system-re.patch
kubelet-cpumanager-introduce-concept-of-isolated-CPU.patch
kubeadm-create-platform-pods-with-zero-CPU-resources.patch
enable-support-for-kubernetes-to-ignore-isolcpus.patch

Story: 2008972
Task: 43056

Signed-off-by: Jim Gauld <james.gauld@windriver.com>
Change-Id: Ie19612f1980690be073ab2236afbb9ccefe504e5
2021-09-13 14:53:42 -04:00
Jim Gauld 229ecb0d99 Add staged versions of kubernetes 1.18.1 and 1.19.13
Multiple versions of kubernetes are required to support upgrade.

This adds staged versions of kubernetes 1.18.1 and 1.19.13, each are
built with a specific version of golang.

All subpackage versions are included in the iso image without collisions.

The following patches are included upstream in kubernetes 1.19 and are no
longer required:
Patch1: 0001-Fix-pagesize-check-to-allow-for-options-already-endi.patch
Patch3: fix_http2_erringroundtripper_handling.patch
Patch8: Fix-exclusive-CPU-allocations-being-deleted-at-conta.patch

The following patches are ported to specific kubernetes version:
kubelet-cpumanager-disable-CFS-quota-throttling-for-.patch
kubelet-cpumanager-keep-normal-containers-off-reserv.patch
kubelet-cpumanager-infrastructure-pods-use-system-re.patch
kubelet-cpumanager-introduce-concept-of-isolated-CPU.patch
kubeadm-create-platform-pods-with-zero-CPU-resources.patch
enable-support-for-kubernetes-to-ignore-isolcpus.patch

Depends-On: https://review.opendev.org/c/starlingx/ansible-playbooks/+/806912
Story: 2008972
Task: 43055

Signed-off-by: Jim Gauld <james.gauld@windriver.com>
Change-Id: I90871451c361e4d855098adbf0c9f4f0fddcc461
2021-09-01 16:51:45 -04:00
Kyle MacLeod e2ab5cc2c8 Patch watch.py in python-kubernetes package
Patch the python2-kubernetes-8.0.0-8.el7.noarch.rpm with recent
bug fix commits required for proper kubernetes watch functionality.

Patches watch.py up to commit 10ae476 in the 'base' repo
(kubernetes-client/python-base).

Commits are taken from the cloned github repo, saved in patch format,
and applied as a patch to the source RPM.

Reference:
https://github.com/kubernetes-client/python-base/commits/master/watch/watch.py

This patch includes commits beginning with d56fdbc, up to and including 10ae476

Testing:
- Built and testing on local distributed cloud system
- Similar testing to this patch but  ased on locally modified package
  has been done on 1000 subcloud system
- Examine/compare contents of installed package vs. expected
- Generating events which trigger the watch conditions
- Monitor watches for proper behaviour on expiry

Story: 2008960
Task: 43053

Signed-off-by: Kyle MacLeod <kyle.macleod@windriver.com>
Change-Id: I7ad78957b6ef61e7204c45f482f201d5c281385b
2021-08-25 17:05:03 -04:00
Zuul 0a6dd52d96 Merge "uprev the containernetwork-plugins" 2021-08-17 20:15:05 +00:00
Jim Gauld 8e7ce96e0e Split kubernetes into versioned stages and unversioned package
This packages kubernetes in versioned subdirectories to be able to
support upgrading multiple versions of kubernetes without collisions.
Common configuration/environment files are moved to the new
kubernetes-unversioned package.

This creates directories:
/usr/local/kubernetes/<version>/stage1
/usr/local/kubernetes/<version>/stage2

The binaries and configuration or kubernetes-node, kubernetes-kubeadm,
and kube-client are placed in new locations, e.g.,
/usr/local/kubernetes/1.18/stage1/usr/bin/kubeadm
/usr/local/kubernetes/1.18/stage2/usr/bin/{kubelet,kubectl}

A new package kubernetes-unversioned contains directories:
/usr/local/kubernetes/current/stage1
/usr/local/kubernetes/current/stage2

This package contains symlinks of all the expected binaries and
configuration needed to run kubernetes. For example, /usr/bin/kubelet
is a symlink to /usr/local/kubernetes/current/stage2/user/bin/kubelet.

At ansible install time and during subsequent boots, there is code in
place to bind-mount /usr/local/kubernetes/<version>/stageX onto
/usr/local/kubernetes/current/stageX .

This removes redundant doc and man pages from the production rpms.

Story: 2008972
Task: 43002
Depends-On: https://review.opendev.org/c/starlingx/ansible-playbooks/+/802898
Signed-off-by: Jim Gauld <james.gauld@windriver.com>
Change-Id: I74dc867faea6759906a687cef0b0ebf9555829ee
2021-08-17 11:19:21 -04:00
Steven Webster a59d1553d5 uprev the containernetwork-plugins
This commit upgrades the containernetwork plugins to version 0.9.1

As there is no existing upstream package for this version for
StarlingX, it is being built based on the downloaded source and
centos7 spec file.

Note: previous commit 303ed35 was reverted because of an issue
with the centos_tarball-dl.lst and the autosetup command of the
spec file.  The second parameter of the centos_tarball-dl.lst
specifies the top level directory name of the package. If the
dl_tarball script detects that this TLD differs from that of
the downloaded tarball, the TLD is changed to that of the
2nd parameter.  In this case, the TLD of the downloaded package
was 'plugins-0.9.1', while the centos_tarball-dl.lst had
specified 'containernetworking-plugins-v0.9.1'.  The two options
to fix this incompatibility are to either change the TLD in the
centos_tarball-dl.lst to match the downloaded version, or to
modify the .spec file to run the autosetup command against the
re-named TLD.  This commit chooses the 2nd option, which allows
for the package to built against any mirror dl that already has
the containernetworking-plugins download present in it.

Plugins tested:

bridge
vlan
host-device
dhcp
host-local
static
tuning
portmap
bandwidth

Story: 2008972
Task: 42977

Signed-off-by: Steven Webster <steven.webster@windriver.com>
Change-Id: Ice12cbeacaeadc8beaa22152ca2a6104d31eec8b
2021-08-13 11:35:07 -04:00
Steven Webster d801ea657d Revert "uprev the containernetwork-plugins"
This reverts commit 303ed359c7.

Reason for revert: The v0.9.1 tarball referenced in the centos_tarball-dl.lst is extracting to 'plugins-0.9.1' rather than the previous 'containernetworking-plugins-0.9.1'.  This seems to have happened in the last couple of days and will need to be investigated.

Change-Id: I9116cfa133d8e582740c7a9dbee873f3be939b13
2021-08-12 21:34:32 +00:00
Steven Webster 303ed359c7 uprev the containernetwork-plugins
This commit upgrades the containernetwork plugins to version 0.9.1

As there is no existing upstream package for this version for
StarlingX, it is being built based on the downloaded source and
centos7 spec file.

Plugins tested:

bridge
vlan
host-device
dhcp
host-local
static
tuning
portmap
bandwidth

Story: 2008972
Task: 42977
Signed-off-by: Steven Webster <steven.webster@windriver.com>
Change-Id: Ia29df16aacec35dbda79a2f10a44eab90192dd6f
2021-08-10 11:41:04 -04:00
Cole Walker 1f4538df28 Add collection of linuxptp patches
This commit applies several patches to the linuxptp srpm in order to
address an issue syncing multiple interfaces on a ptp node. The srpm
used is linuxptp-2.0-2.el7.src.rpm.

Patch descriptions:
base/linuxptp/centos/meta_patches:
0001 updates the srpm spec file to apply the patches during build
0002 updates the package versioning to comply with the STX format

base/linuxptp/centos/patches:
Patches 0001-0005 combine to correct a fault present when a ptp node is
configured with multiple clocks in jbod mode which results in the client
port getting stuck in the UNCALIBRATED state and unable to lock to the
Grandmaster clock. The root of the issue lies in the sanity check where
checking timestamps recieved on multiple ports will result in the
sanity_freq_limit threshold constantly being reached and the servo for
that port is repeatedly reset, preventing it from ever syncing.

The changes in patches 0001-0005 have been written by Miroslav Lichvar
on the linuxptp-devel mailing list. They are currently under review and
testing by the upstream linuxptp maintainers prior to merging. I was
able to apply them as-is to linuxptp v2.0. I have chosen to keep them as
individual patches, as that is how they will appear upstream.

Patch 0006 is my work and serves to address an issue in phc2sys
where the local ptp clocks are not synced together properly if the local
time is far behind the reference time. This issue ocurrs when phc2sys
starts and there is no client port currently synced to a grandmaster. In
the original behaviour, phc2sys selects the first configured port and
proceeds to sync all of the other clocks to it by performing the
first_step operation.

Then ptp4l will evenually lock to the Grandmaster clock, and that
single port will have its time updated to the correct value, but
phc2sys has already performed the first_step operation and will not
step the other clocks again.

My solution is to provide an option to disable the selection of a
default port by phc2sys. When no default port is selected, phc2sys waits
for ptp4l to sync to the Grandmaster before bringing the other clocks
into sync with the first_step operation.

This option is configured via the default_sync
parameter or the -D flag. The default_sync parameter is set to on by
default to in order to keep the behaviour the same as upstream linuxptp
but can be configured by users via
system service-parameter-add ptp global default_sync=0

Closes-Bug: 1930607

Signed-off-by: Cole Walker <cole.walker@windriver.com>
Change-Id: I2f660787c6753dcd4fc4c51da7b08ab9e6f197f4
2021-06-28 14:15:22 -04:00
Zuul 43ffd243ca Merge "integ: add nvidia gpu-operator helm charts" 2021-04-07 11:49:01 +00:00
Chris Friesen 859e8eb730 add isolcpus device plugin for kubernetes
In order to minimize latency as much as possible, we want to allow
kubernetes containers to make use of CPUs which have been specified
as "isolated" via the kernel boot args.

This commit creates an isolcpus device plugin, which detects the isolated
CPUs and exports them to kubelet via the device plugin API.

See kubernetes/plugins/isolcpus-device-plugin/files/README.md for
more information on the behaviour and design choices for this commit.

When we move to a newer version of the Intel device plugin manager we
may be able to simplify some of this.  See the above README.md file
for details.

Change-Id: I3bfe04ab6e7fbafefa63f6dc43cb2ed79a52579f
Story: 2008760
Task: 42165
Signed-off-by: Chris Friesen <chris.friesen@windriver.com>
2021-04-01 11:10:09 -06:00
Babak Sarashki 7badc1dad1 integ: add nvidia gpu-operator helm charts
This commit adds nvidia gpu-operator helm charts use case for
custom container runtime feature. To load nvidia-gpu-operator
on starlingx:

system service-parameter-add platform container_runtime \
custom_container_runtime=\
nvidia:/usr/local/nvidia/toolkit/nvidia-container-runtime

And define  runtimeClass for nvidia gpu  pods:

kind: RuntimeClass
apiVersion: node.k8s.io/v1beta1
metadata:
  name: nvidia
handler: nvidia

The above will direct all containerd creations of pods with nvidia
runtimeClass to nvidia-container-runtime -- where the nvidia-conta
iner-runtime is installed by the operator onto a hostMount.

Story: 2008434
Task: 41978

Signed-off-by: Babak Sarashki <babak.sarashki@windriver.com>
Change-Id: Ifea8cdf6eb89a159f446c53566279e72fcf0e45e
2021-03-31 17:33:41 +00:00
Jim Gauld f161f7f18e Revert "integ: gpu-operator helm charts"
This reverts commit 41bdf53f65.

Reason for revert: gpu operator patch is breaking stx-master build.

e.g.,
08:06:44 Failed to build packages:  gpu-operator-1.6.0-0.tis.1.src.rpm; problem with:
Patch #2 (enablement-support-on-starlingx-cloud-platform.patch):
. .
Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file deployments/gpu-operator/templates/operator.yaml.rej
patching file deployments/gpu-operator/values.yaml
error: Bad exit status from /var/tmp/rpm-tmp.VQuqLh (%prep)

Change-Id: Id7a05987586582c940d605874d1e0f813333f2c3
2021-03-29 12:31:25 +00:00
Babak Sarashki 41bdf53f65 integ: gpu-operator helm charts
This commit adds nvidia gpu-operator helm charts use case for
custom container runtime feature. To load nvidia-gpu-operator
on starlingx:

system service-parameter-add platform container_runtime \
custom_container_runtime=\
nvidia:/usr/local/nvidia/toolkit/nvidia-container-runtime

And define  runtimeClass for nvidia gpu  pods:

kind: RuntimeClass
apiVersion: node.k8s.io/v1beta1
metadata:
  name: nvidia
handler: nvidia

The above will direct all containerd creations of pods with nvidia
runtimeClass to nvidia-container-runtime -- where the nvidia-conta
iner-runtime is installed by the operator onto a hostMount.

Story: 2008434
Task: 41978

Signed-off-by: Babak Sarashki <babak.sarashki@windriver.com>
Change-Id: I999804d4697349bc0966d0a6e653d7bce15e18fc
2021-03-25 01:10:04 +00:00
Babak Sarashki 8a33372bee Add: PF Baseband Device config application for ACC100
This introduces PF BBDEV (baseband device) Configuration Application
"pf_bb_config" and inih. PF BBDEV program accesses the configuration
space and sets the various parameters through memory-mapped IO
read/writes. This is needed for Intel ACC100 (Mt Bryce) configuration
and QMGR related settings.

PF BBDEV requires inih for parsing .INI configuration file. This
commit adds the inih for static linkage with PF BBDEV.

Story: 2008440
Task: 41472
Signed-off-by: Babak Sarashki <zbsarashki@gmail.com>
Change-Id: Idaebcac5d0021d5c11c7ab27e13176139ba66c3b
2021-02-11 23:18:51 +00:00
Zuul 46a26e251c Merge "Uninstall SNMP RPM Host-Based from STX." 2021-01-22 15:54:34 +00:00
Nicolas Alvarez d815cfe2f2 Uninstall SNMP RPM Host-Based from STX.
Uninstall SNMP RPM Host-Based from starlingx/integ repo because it
will be containerized.
Also disable snmp from networking/lldpd/centos/lldpd.spec file.

Story: 2008132
Task: 41322
Depends-On: https://review.opendev.org/761792
Signed-off-by: Nicolas Alvarez <nicolas.alvarez@windriver.com>

Change-Id: Ifda06a5eb3bd0ec9683823b643e6d9cc0e7c97e2
2020-12-14 11:45:30 -03:00
Carmen Rata 53bbe86567 swtpm rpms cleanup
Software-based TPM for openstack VMs is not supported
anymore. As a result we are removing all the swtpm
related rpms configuration and existing implementation.

Story: 2008037
Task: 40694

Change-Id: Icc4809e02c4cd790641ac20692809e93ffddf782
Signed-off-by: Carmen Rata <carmen.rata@windriver.com>
2020-12-11 09:21:33 -05:00
Robert Church 17c1b8894d Introduce k8s pod recovery service
Add a recovery service, started by systemd on a host boot, that waits
for pod transitions to stabilize and then takes corrective action for
the following set of conditions:
- Delete to restart pods stuck in an Unknown or Init:Unknown state for
  the 'openstack' and 'monitor' namespaces.
- Delete to restart Failed pods stuck in a NodeAffinity state that occur
  in any namespace.
- Delete to restart the libvirt pod in the 'openstack' namespace when
  any of its conditions (Initialized, Ready, ContainersReady,
  PodScheduled) are not True.

This will only recover pods specific to the host where the service is
installed.

This service is installed on all controller types. There is currently no
evidence that we need this on dedicated worker nodes.

Each of these conditions should to be evaluated after the next k8s
component rebase to determine if any of these recovery action can be
removed.

Change-Id: I0e304d1a2b0425624881f3b2d9c77f6568844196
Closes-Bug: #1893977
Signed-off-by: Robert Church <robert.church@windriver.com>
2020-09-03 23:38:41 -04:00
Jim Gauld 8ceb938222 Build helm-toolkit for armada to decouple distro from flock
This provides helm-toolkit identically to how openstack-helm-infra
is built in repo stx/openstack-armada-app. This version of
helm-toolkit is used to build armada chart. This decouples distro
build from flock.

Story: 2007000
Task: 38893

Change-Id: I537625236fb05200e5380c4f23e3e144e24c8675
Signed-off-by: Jim Gauld <james.gauld@windriver.com>
2020-06-13 16:34:13 -04:00
Jim Gauld e0bf31f63f Add support for Helm v3, chartmuseum, and armada chart
This adds support for Helm v3:
- 'helm init' and 'helm serv' were removed in v3, and helm
  initialization was simplified so that is not required in build.
- chart validation and version checking is enforced with 'helm lint',
  so all Charts require the tag: apiVersion: v1 (or v2).
- 'chartmuseum' is a drop-in replacement for 'helm serv', and is
  currently used for building charts only. It is not part of ISO
  image.
- armada chart is built and installed to /opt/extracharts. This
  provides a Kubernetes pod with armada-api and tiller containers.

This provides a Helm v2 client (i.e., helmv2-cli) that gives access
to containerized armada/tiller managed charts. This can be used as
an interactive shell, or as a wrapper for single helm v2 commands.

Change-Id: Iff2b219ea765cf9278c6e80c6aeb5b98cc9a0626
Depends-On: https://review.opendev.org/732731
Story: 2007000
Task: 38893
Signed-off-by: Jim Gauld <james.gauld@windriver.com>
2020-06-10 03:44:59 -04:00
Ran An e5996667f5 Add python3-daemon required by logmgmt
This reverts commit fc125a7a24.

pkg logmgmt upgraded to python3 requires python3 model "daemon",
and no pkgs in Centos7 offical repo provide it.

this patch refer to the python3-daemon pkg build by rdo
for CentOS 8: python-daemon-2.2.3-7.el8.src.rpm

disable the rpm check part which is not required in stx to
reduce python3 dependencies that not supported by CentOS 7

Depends-on: https://review.opendev.org/#/c/728324/
Depends-on: https://review.opendev.org/#/c/729635/
Depends-on: https://review.opendev.org/#/c/728326/
Change-Id: Iad2e4bb2f2087f46b7c27e80a9423cd5cc1e0517
Story: 2007106
Task: 39291
Signed-off-by: SidneyAn <ran1.an@intel.com>
2020-05-27 06:32:15 +00:00
Ran An fc125a7a24 Revert "Add python3-daemon required by logmgmt"
This reverts commit 97cd7ea5c1.

Change-Id: I3f09054c1546252493f8eb29dc70806829324a52
2020-05-14 11:41:50 +00:00
SidneyAn 97cd7ea5c1 Add python3-daemon required by logmgmt
pkg logmgmt upgraded to python3 requires python3 model "daemon",
and no pkgs in Centos7 offical repo provide it.

this patch refer to the python3-daemon pkg build by rdo
for CentOS 8: python-daemon-2.2.3-7.el8.src.rpm

disable the rpm check part which is not required in stx to
reduce python3 dependencies that not supported by CentOS 7

Depends-on: https://review.opendev.org/#/c/727657/
Depends-on: https://review.opendev.org/#/c/727662/
Change-Id: Ie08ea9c7adf830ad4e8e924fa69352fb2a923a6f
Story: 2007106
Task: 39291
Signed-off-by: SidneyAn <ran1.an@intel.com>
2020-05-13 21:14:03 +08:00
Jim Somerville 02dd02d591 Move mellanox userspace from integ repo
libibverbs and rdma-core are part of the mellanox driver
software package and to facilitate layering, shouldn't
be split across different repos.

Move them from the integ repo to the kernel repo so they
can co-reside with the kernel module portion.

Change-Id: I4c8582dadbae52b342d7178313d8b62beeb61148
Closes-Bug: 1877160
Depends-On: I41153feceef4eb26a41ff634c60dc3adaaf238b1
Signed-off-by: Jim Somerville <Jim.Somerville@windriver.com>
2020-05-06 19:58:38 -04:00
Davlet Panech bceb6c148c Subdirectory kernel relocated to new repo starlingx/kernel
* Moved subdirectories from kernel/ into a new repo
* Removed references to kernel from this repo's file lists

Change-Id: I386418f51169dd9b8c977bae328060077ac44b93
Depends on: I4b171accd8b489c92f6d2c69cb7aa5c63e75f336
Signed-off-by: Davlet Panech <davlet.panech@windriver.com>
2020-04-11 13:08:18 -04:00