This update introduces a new maintenance group alarm ; 200.003
This new alarm is minor and management affecting if asserted.
It is considered management affecting for the upgrades case because
the pxeboot network is needed to upgrade a node.
The alarm represents a communication/messaging failure between the
active controller mtcAgent process and the mtcClient that runs
on each node.
Test Plan:
PASS: Verify alarm attributes
PASS: - code of 200.003
PASS: - assertion cause text
PASS: - proposed repair action text
PASS: - suppression option
PASS: - does not inhibit other alarms
PASS: - affect of assertion on upgrade healthcheck
PASS: Verify ability to assert and clear
PASS: Verify fm logging for the above assertion and clear
Story: 2010940
Task: 49789
Change-Id: I507d30213674c5b1e24fcfebe15c6a87bad74358
Signed-off-by: Eric MacDonald <eric.macdonald@windriver.com>
This new alarm is raised when the controllers deploy
state is out of sync during the deployment.
Test Plan:
PASS: alarm raised when the deploy state is out of sync
PASS: alarm cleared when the deploy state is in sync
Task: 49732
Story: 2010676
Change-Id: Ibdcc54f02c9e156b2b78313b527cd273a62425f1
Signed-off-by: junfeng-li <junfeng.li@windriver.com>
This change added alarm 250.004, "IPsec certificates renewal failed".
This alarm will be raised by the ipsec-cert-renew cron job when the
renewal failed, and will be cleared when cron job script is re-run,
either manually or by cron, after the error is fixed.
Test Plan:
PASS: Simulate a failure condition (eg, ipsec-client return non zero),
run the cron job script, verify the IPsec renewal fails, and
alarm 250.004 is raised.
PASS: Run the script with IPsec cert not being about to expire, verify
the script finish successfully and alarm 250.004 is cleared.
Story: 2010940
Task: 49706
Change-Id: Ie4d3970ca32173939c1df55a2e59241ac214b2ae
Signed-off-by: Andy Ning <andy.ning@windriver.com>
Configuring new alarm 850.002 which will be used when K8s
periodic audit happens and any of the endpoint health check
fails.
Test Plan:
PASS: Performed tox test locally and packages were built
successfully
PASS: Verify that k8s orchestrated upgrade will get blocked,
if it is tried while alarm is set. And k8s orchestrated
upgrade will complete, if it is tried when alarm is
cleared.
Story: 2011037
Task: 49535
Change-Id: I335179ea98ef63d7c35c89d82328a52ab2391f5c
Signed-off-by: rakshith mr <rakshith.mr@windriver.com>
Currently there is no alarm for node taint.
This new alarm 900.701 describes the attributes
of the node taint.
Test Plan:
PASSED: Verified the details of the alarm
using fm alarm-list.
Partial-Bug: 2046273
Change-Id: I929ddb45b75f1e4b097b84919f703d458d8fa39e
Signed-off-by: Vanathi.Selvaraju <vanathi.selvaraju@windriver.com>
Maintenance service raises an alarm with ID 200.016 if luks-fs-mgr
service is inactive. This change adds the description of 200.016 alarm.
Test Plan:
PASS: build-pkgs -c -p fm-doc
PASS: build-image
PASS: AIO-SX bootstrap with LUKS service status inactive. A critical
alarm with ID 200.016 should be displayed while listing the alarm
using 'fm alarm-list'
Story: 2010872
Task: 49125
Depends-On: https://review.opendev.org/c/starlingx/metal/+/901455
Change-Id: Iadee64bffbb37cfd94aa735f7eeb12ba0fa86fbd
Signed-off-by: Jagatguru Prasad Mishra <jagatguruprasad.mishra@windriver.com>
The alarm 280.004 is added and will be raised when the system peer
connection failure has been detected, and will be cleared when the
system peer connection has been restored.
The alarm 280.005 is added and will be raised when the a subcloud peer
group is being managed by a remote system with a lower priority, and
will be cleared when the subclouds belonging to the subcloud peer group
have been migrated back to current system.
Test Plan:
PASS - Verify successful tox test and package build
Story: 2010852
Task: 48492
Change-Id: I3068676933c0446a88bd4290277456cd0962f941
Signed-off-by: Zhang Rong(Jon) <rong.zhang@windriver.com>
400.001 and 400.002 alarms are tagged for openstack but should be
starlingx.
This change tags them to starlingx so the documentation scripts are
able to classify them correctly.
Test plan
PASS: Check the parsing scripts end successfully.
Closes-bug: 2028379
Signed-off-by: Agustin Carranza <agustin.carranza@windriver.com>
Change-Id: I8f5966d5b0a7b82198e4bc2e735fa4536a4cdd0a
Since the alarm documentation has been automated and the events.yaml
file is taken as source of truth for it, it is required to link
the alarms proposed repair action with a direct link to the
documentation for the users.
This change modifies the mentions of documentation to a proper link,
using Sphinx placeholder that are interpreted by the documentation
language.
Test plan
PASS:
* Build fm-doc package. Check that all parsing checks were run and
package was built successfully.
Closes-bug: 2022104
Change-Id: Iccb34e42ed80634d73cf7549e9230976579deef7
Signed-off-by: Agustin Carranza <agustin.carranza@windriver.com>
The 250.002 alarm has been deprecated long time ago.
This change deletes it from the alarm list.
Test plan:
PASS:
* Build fm-doc and fm-api packages.
* Check that all parsing checks were run and package was built
successfully.
Depends-on: https://review.opendev.org/c/starlingx/distcloud/+/886001
Closes-bug: 2024010
Signed-off-by: Agustin Carranza <agustin.carranza@windriver.com>
Change-Id: I0180bd5addc1feae6e3e45edd24c1f50d6622e2c
Some alarms reference to "System Administration Manual" but this
document does not exist. It was changed to a generic documentation
reference.
The 800.103 alarm has been deprecated so it is deleted from the
events.yaml file.
Test plan
Pass:
* Build fm-doc package. Check that all parsing checks were run and
package was built successfully.
Closes-bug: 2022104
Signed-off-by: Agustin Carranza <agustin.carranza@windriver.com>
Change-Id: I4723d05e77983796a0f64c7242f5c2bcf4699763
This change adds a parsing check to ERROR if Context field is Empty.
Until now there had not been a requirement of non empty fields, so in
case this is needed in the future for other key/values, a collection is
created.
Test plan
PASS: * Add/modify an alarm/log in events.yaml file with Context field
set to <Empty>.
* Run the checkEventYaml script and check it fails.
PASS: * Check that all the events in events.yaml file have the Context
field set to a non empty value.
* Run the checkEventYaml script and check it ends successfully.
Closes-bug: 2020381
Signed-off-by: Agustin Carranza <agustin.carranza@windriver.com>
Change-Id: Ia267886dd49099525751165975fb5d291c0c6f82
System config update alarms are 900.6xx series
The new alarms are originated by a new type of vim strategy
orchestrating configuration update.
The new alarms are similar in numbering and wording as the
kube upgrade auto apply 900.4xx series alarms and logs.
System config update in-progress alarm is 900.010.
System config update aborted alarm is 900.011.
Story: 2010719
Task: 47947
Change-Id: Ieb6e68adf359ac7b0489d15bb33cb4b4a9f3ef3f
Signed-off-by: Yuxing Jiang <Yuxing.Jiang@windriver.com>
Product Documentation is missing the alarm 900.007 'Kubernetes upgrade
in progress.'
That alarm has the Context field set to none. In order to be included
in stx documentation, it has to be set to Context: starlingx.
Test plan:
PASS: Run documentation generating scripts and check the alarm is now
included.
Closes-bug: 2019146
Signed-off-by: Agustin Carranza <agustin.carranza@windriver.com>
Change-Id: I4d4867e5299e3fb1eb37c9bcd3e53447d4f08ba5
This commit is intended to update the 260.002 alarm. As the 'severity'
is set to 'minor', it is desired to classify it as
non-management-affecting by adjusting its Management_Affecting_Severity
value to 'none'.
Test Plan:
PASS: Build and install Debian package.
Story: 2010719
Task: 47938
Change-Id: Ie228191ebdda5f2651dab1309b929ae06bc1f7f6
Signed-off-by: Enzo Candotti <enzo.candotti@windriver.com>
This commit adds a new alarm id and definition for resources
that has INSYNC=False.
The alarm will be raised when a resource is not
synchronized during a process of update. It will be cleared when
the resource is synchronized again.
Test Plan:
- Verify successful tox test and package build
- Verify the alarm can be raised using FmClientCli
Story: 2010719
Task: 47910
Change-Id: I24a976ed4beaa8248df25fd97eeee27f5754b969
Signed-off-by: Enzo Candotti <enzo.candotti@windriver.com>
Some documentation generating scripts were introduced in order to avoid
manual intervention every time an alarm/log is changed/added/removed.
Those scripts required a way to know where the alarm/log belongs to.
For that requirement, the field Context was introduced in previous
commits. During that development, it was taken the current
classification at that time in the docs as source of truth, but it was
outdated.
This commits modifies the values that were detected as wrong/outdated.
The scripts also require the value 'none' in the Context field for when
an alarm/log should not be included in the documentation but still be
defined in the events.yaml file. So the Context value is updated for
that case too.
Context incorrectly tagged as openstack and changed to starlingx:
* 900.006
Context incorrectly tagged as starlingx and changed to openstack:
* 100.105
* 100.112
* 100.113
* 300.001
* 300.002
Closes-bug: 2012981
Test plan
PASS: Since the Context field does not have impact in functionality,
build and install fm-doc package successfully.
Check the file in the filesystem contains this change.
PASS: Trigger random alarms and check FM functionality.
Signed-off-by: Agustin Carranza <agustin.carranza@windriver.com>
Change-Id: I16f858bbb712349f08b2ceca33152e365b0ed733
Currently, there is no alarm for Restore in progress.
Because of this, the system is shown as healthy,
before restore has been completed.
This new alarm will prevent the system from being healthy
until restore has properly been completed.
TEST PLAN
PASS: On any available system, the following commands can
be triggered at anytime:
* Run "system restore-start" to trigger alarm
* Run "system restore-complete" to clear alarm
Story: 2010117
Task: 47689
Signed-off-by: Joshua Kraitberg <joshua.kraitberg@windriver.com>
Change-Id: I292b5c8083c08b68ac757fe5a650989178eb819f
When a 800-Series alarm occurs, users refer to the documentation to
know what kind of error is shown. But sometimes that is not enough
information.
The output of some commands can be useful information and could
save time when solving issues related to the storage alarms.
Closes-bug: 2004601
Test plan
PASS: * Build fm packages and deploy an ISO containing new fm
packages.
* Trigger alarms that were modified by this commit,
(e.g. shutdown a controller).
* Run fm alarm-list --uuid and copy the uuid of a 800-series
alarm.
* Run fm alarm-show <uuid> and check that the field
has changed.
Signed-off-by: Agustin Carranza <agustin.carranza@windriver.com>
Change-Id: I94e2719b55b4fc14b692439526b5b47204460ac7
The checkEventYaml script verifies if all contents
are properly populated for the events.yaml file.
This change ensures that check is done by zuul, rather
than during the build.
yaml.load after version 5.1 requires a Loader argument.
The yaml.load in fm-doc are now updated to use safe_load
intead
Test Plan:
PASS: tox -e linters
PASS: remove 'context' field from an alarm and observe
that tox -e linters reports a failure.
PASS: build-pkgs -p fm-doc
Story: 2010531
Task: 47549
Signed-off-by: Al Bailey <al.bailey@windriver.com>
Change-Id: I369ffe4c74fcaf5fe4a916822fed18a78ead8ff8
Removal of stale alarm 270.001(Host compute service failure)
is raised by the vim. This might be an old reference to nova.
It’s likely not in use since stx.
Test Plan:
PASS: Verify with a load without the changes (removal of alarm)
and the event log in platform.log shows an entry for 270.001 alarm.
PASS: Verify with a load with changes of alarm removal and
the event log in platform.log does not show an entry for 270.001 alarm.
Closes-Bug: 2004744
Change-Id: I47a9f5cede2cfade4a16c63a2dc1bcfd563e88cf
Signed-off-by: Vanathi.Selvaraju <vanathi.selvaraju@windriver.com>
The Debian packaging has been changed to reflect all the
git commits under the directory, and not just the commits
to the metadata folder.
This ensures that any new code submissions under those
directories will increment the versions.
All packages have a higher version than before the change.
Test Plan:
PASS: build-pkgs -c -p fm-api
PASS: build-pkgs -c -p fm-common
PASS: build-pkgs -c -p fm-doc
PASS: build-pkgs -c -p fm-mgr
PASS: build-pkgs -c -p fm-rest-api
PASS: build-pkgs -c -p python-fmclient
Story: 2010550
Task: 47226
Signed-off-by: Al Bailey <al.bailey@windriver.com>
Change-Id: I65e881ba96512d2eaba25c44332d5ae82efea502
The events.yaml file contains every alarm and log used by platform and
openstack. There is no way to know which one relates to one or
the other.
In order to know that, it is required an additional field as part of
each record to differential between platform and openstack.
Story: 2010143
Task: 46723
Test plan
PASS: Build the fm-api and fm-doc packages.
Install fm-api first and then fm-doc.
No errors are found during build and installation process.
Signed-off-by: Agustin Carranza <agustin.carranza@windriver.com>
Change-Id: I8598afc77d27d107c4f9a108dd46b2ebc79b30a1
When compute services remain healthy:
- listing alarms shall not refer to the below Obsoleted alarm
- 200.012 alarm hostname controller function has an in-service failure
This update deletes definition of the obsoleted alarm and any references
200.012 is removed in events.yaml file
Also updated any reference to this alarm definition.
Need to also raise a Bug to track the Doc change.
Test Plan:
Verify on a Standard configuration no alarms are listed for hostname
controller in-service failure
Code (removal) changes exercised with fix prior to ansible bootstrap
and host-unlock and verify no unexpected alarms
Regression:
There is no need to test the alarm referred here as they are obsolete
Closes-Bug: 1991531
Signed-off-by: Girish Subramanya <girish.subramanya@windriver.com>
Change-Id: I255af68155c5392ea42244b931516f742fa838c3
This commit adds a new alarm id and definition for subcloud backup.
The alarm will be raised and cleared by dcmanager upon subcloud backup
failure and successful retry respectively.
Test Plan:
- Verify successful tox test and package build
Story: 2010116
Task: 46162
Change-Id: I9a34a827e484bb691c8ce0da63e1a7d26735c289
Signed-off-by: Tee Ngo <Tee.Ngo@windriver.com>
The following alarms need to be clearer for users.
Some information was gathered in order to improve the
'description' and 'Proposed repair action' fields.
500.200
500.210
750.002
900.002
900.003
900.004
900.009
900.103
900.203
900.303
900.503
Test plan:
Build and install Debian package.
Story: 2010143
Task: 45785
Signed-off-by: Agustin Carranza <agustin.carranza@windriver.com>
Change-Id: I55f81ffa4159284b730e2572e76dfd9867b348d5
This reverts commit baead557fb.
Reason for revert: Change breaks the build. 270.001 alarm has to remain defined because it is still being referenced.
Change-Id: I136928c90634e05e3026dddc9443cc2c59203320
The following alarms need to be clearer for users. Some information
was gathered in order to improve the 'description' and 'Proposed
repair action' fields.
Alarm that has been deprecated:
270.001
The list of alarms that were modified is the following:
500.210
500.200
750.002
750.006
800.003
800.102
900.002
900.003
900.004
900.009
900.103
900.203
900.303
900.503
Test plan:
There is no need to test the alarms affected by the changes.
Story: 2010143
Task: 45785
Signed-off-by: Agustin Carranza <agustin.carranza@windriver.com>
Change-Id: I57b86548a36da66119cb04779ce1f9147254316c
800.102 alarm is not necessary anymore so it must be removed.
Test Plan: No test needed
Story: 2010104
Task: 45668
Signed-off-by: Gabriel de Araújo Cabral <gabriel.cabral@windriver.com>
Change-Id: I6f47dadb28c78781e95f479739a4a846df8431fc
The 900.002 and 900.003 alarm IDs were reversed.
Now corrected to be what the constants and code
are generating.
- 900.002 is Patch host install failure
- 900.003 is Obsolete patch in system
Story: 2009969
Task: 45582
Signed-off-by: Al Bailey <al.bailey@windriver.com>
Change-Id: I0d13cfce39de15b34265012d1e26fd7fedcee81d
This commit updates the description and entity instance for the
PTP alarm for SyncE.
Story: 2009130
Task: 44497
Change-Id: Iefae1e75003ff2a45501e45d850f9556256d580f
Signed-off-by: Teresa Ho <teresa.ho@windriver.com>
Constant FM_ALARM_ID_STORAGE_CEPH_FREE_SPACE was being used only in
legacy code by ceph-manager. The code was updated to remove this
unused constant and its respective alarm.
Test plan:
PASS: Verify that no alarms are raised related to quotas.
PASS: Confirm that no quota messages are present in the logs.
Depends-On: https://review.opendev.org/c/starlingx/config/+/827545
Depends-On: https://review.opendev.org/c/starlingx/utilities/+/820933
Closes-Bug: 1959894
Signed-off-by: Daian Cardoso Sganderlla <Daian.CardosoSganderlla@windriver.com>
Change-Id: I606d0e6dca102189a26c8cd4d587a1e772e610f0
Add debian packaging infrastructure for fm-doc to build
a debian package.
Test Plan:
PASS: Package installed and ISO built successfully
Story: 2009101
Task: 44088
Signed-off-by: Ramon Gazoni Lacerda <Ramon.GazoniLacerda@windriver.com>
Change-Id: If98aa5c561012e31ab67589e7d62486c58be8e4c
Add a new alarm for open FD approaching limit (major) or
limit is reached (critical).
Partial-bug: 1952126
Change-Id: Ifaece0e1d7a335f980cfebc3a591a90edbc35742
Signed-off-by: Bin Qian <bin.qian@windriver.com>
Recent commit to add Expiring-Soon and Expired alarm
IDs and description resulted in failure in Openstack
app with following error.
UnicodeEncodeError: 'ascii' codec can't encode character
u'\u2018' in position 12: ordinal not in range(128)
The problem was traced to incorrect quotation marks
in Description field. Tested with a fix of the quotes
and fm-rest-api pod was able to come up successfully.
Story: 2008946
Task: 43049
Signed-off-by: Sabeel Ansari <Sabeel.Ansari@windriver.com>
Change-Id: Iacdf8c18daf412b0ac6d9d8aa89de1c4026356b7
Added two new alarm IDs and details for expiring-soon and
expired certificates.
Story: 2008946
Task: 43049
Signed-off-by: Sabeel Ansari <Sabeel.Ansari@windriver.com>
Change-Id: I61cb835c38473cef61238de8cf4a21361e123da4
This change added a KUBE_ROOTCA_UPDATE_ABORTED alarm for kube
rootca update abort.
Story: 2008675
Task: 43068
Signed-off-by: Andy Ning <andy.ning@windriver.com>
Change-Id: I3096b57697d2c6a548f5c54002705cb1797ae97b
Zuul