From 47b391782f1f9fb864176fcb0cf2307ac13b607c Mon Sep 17 00:00:00 2001 From: albailey Date: Tue, 25 May 2021 13:26:46 -0500 Subject: [PATCH] Calling an additional shell lint command from zuul The '.sh' files are examined by bashate but now the linters command will also run a shellcheck on them. The majority of the shellcheck errors that are failing have been suppressed, and can be updated and fixed by subsequent submissions. Shell scripts that do not end in .sh will also be examined by bashate and shellcheck. Story: 2008943 Task: 42566 Signed-off-by: albailey Change-Id: Ie5c58ec9391ed76bfe34b3544972306341fafcc4 --- test-requirements.txt | 1 + tox.ini | 113 +++++++++++++++++++++++++++++++++++++++++- 2 files changed, 112 insertions(+), 2 deletions(-) diff --git a/test-requirements.txt b/test-requirements.txt index 7e49a722c6..a6141f0802 100644 --- a/test-requirements.txt +++ b/test-requirements.txt @@ -1,4 +1,5 @@ hacking>=1.1.0,<=2.0.0 # Apache-2.0 bashate >= 0.2 PyYAML >= 3.1.0 +shellcheck-py;python_version>="3.0" # MIT yamllint<1.26.1;python_version>="3.0" # GPLv2 diff --git a/tox.ini b/tox.ini index 3227174027..4b5d79086b 100644 --- a/tox.ini +++ b/tox.ini @@ -14,9 +14,23 @@ setenv = VIRTUAL_ENV={envdir} deps = -r{toxinidir}/test-requirements.txt whitelist_externals = reno -[testenv:linters] + +[testenv:bashate] basepython = python3 whitelist_externals = bash +# The following are suppressed: +# E006 Line too long +# E010 The "do" should be on same line as for +# +# Run bashate twice to handle shell scripts that do not end in .sh +# grep for ' shell' and not 'shell' to exclude files with shell in their name +# Several of the shell scripts that do not end in .sh have more suppressions +# E001 Trailing Whitespace +# E002 Tab indents +# E003 Indent not multiple of 4 +# E011 Then keyword is not on same line as if or elif keyword +# E020 Function declaration not in format ^function name {$ +# E042 local declaration hides errors commands = bash -c "find {toxinidir} \ -not \( -type d -name .?\* -prune \) \ @@ -24,15 +38,110 @@ commands = -not -name \*~ \ -not -name \*.md \ -name \*.sh \ - -print0 | xargs -r -n 1 -0 bashate -v -e E* \ + -print0 | xargs -r -n 1 -0 bashate -v -e E* \ -i E006,E010" + bash -c "find {toxinidir} \ + -not \( -type d -name .?\* -prune \) \ + -type f \ + -not -name \*~ \ + -not -name \*.md \ + \( -exec bash -c 'file \{\} | grep -q :.*shell' \; \ + -a ! -name '*.sh' \) \ + -print0 | xargs -r -n 1 -0 bashate -v -e E* \ + -i E001,E002,E003,E006,E010,E011,E020,E042" +[testenv:yamllint] +basepython = python3 +whitelist_externals = bash +commands = bash -c "find {toxinidir} \ -name .tox -prune \ -o -type f -name '*.yaml' \ -print0 | xargs -0 yamllint -f parsable \ -c {toxinidir}/.yamllint" +[testenv:shellcheck] +basepython = python3 +whitelist_externals = bash +# The following shellcheck errors are suppressed: +# SC1091: Not following: ./bin/activate was not specified as input (see shellcheck -x). +# SC2001: See if you can use ${variable//search/replace} instead. +# SC2002: Useless cat. Consider 'cmd < file | ..' or 'cmd file | ..' instead. +# SC2004: $/${} is unnecessary on arithmetic variables. +# SC2006: Use $(...) notation instead of legacy backticked `...`. +# SC2034: variable appears unused. Verify use (or export if used externally). +# SC2046: Quote this to prevent word splitting. +# SC2064: Use single quotes, otherwise this expands now rather than when signalled. +# SC2086: Double quote to prevent globbing and word splitting. +# SC2119: Use remove_tmp "$@" if function's $1 should mean script's $1. +# SC2120: remove_tmp references arguments, but none are ever passed. +# SC2154: variable is referenced but not assigned. +# SC2166: Prefer [ p ] && [ q ] as [ p -a q ] is not well defined. +# SC2181: Check exit code directly with e.g. 'if mycmd;', not indirectly with $?. +# SC2207: Prefer mapfile or read -a to split command output (or quote to avoid splitting). +# SC2231: Quote expansions in this for loop glob to prevent wordsplitting, e.g. "$dir"/*.txt . +# SC2236: Use -n instead of ! -z +# SC2268: Avoid x-prefix in comparisons as it no longer serves a purpose. +# +# Run bashate twice to handle shell scripts that do not end in .sh +# grep for ' shell' and not 'shell' to exclude files with shell in their name +# The non .sh files have these additional suppressions: +# SC1001: This \= will be a regular '=' in this context. +# SC1009: The mentioned syntax error was in this if expression. +# SC1035: You need a space after the [[ and before the ]]. +# SC1072: Expected test to end here (don't wrap commands in []/[[]]). +# SC1073: Couldn't parse this test expression. Fix to allow more checks. +# SC1090: ShellCheck can't follow non-constant source. Use a directive to specify location. +# SC2003: expr is antiquated. Consider rewriting this using $((..)), ${} or [[ ]]. +# SC2009: Consider using pgrep instead of grepping ps output. +# SC2015: Note that A && B || C is not if-then-else. C may run when A is true. +# SC2069: To redirect stdout+stderr, 2>&1 must be last +# SC2112: 'function' keyword is non-standard. Delete it. +# SC2155: Declare and assign separately to avoid masking return values +# SC2168: 'local' is only valid in functions. +# SC2219: Instead of 'let expr', prefer (( expr )) . +# SC2223: This default assignment may cause DoS due to globbing. Quote it. +# SC3005: In POSIX sh, arithmetic for loops are undefined. +# SC3018: In POSIX sh, ++ is undefined. +# SC3020: In POSIX sh, &> is undefined. +# SC3037: In POSIX sh, echo flags are undefined. +# SC3039: In POSIX sh, 'let' is undefined. +# SC3043: In POSIX sh, 'local' is undefined. +commands = + bash -c "find {toxinidir} \ + -not \( -type d -name .?\* -prune \) \ + -type f \ + -not -name \*~ \ + -not -name \*.md \ + -name \*.sh \ + -print0 | xargs -r -n 1 -0 shellcheck \ + -eSC1091 -eSC2001 -eSC2002 -eSC2004 -eSC2006 -eSC2034 \ + -eSC2046 -eSC2064 -eSC2086 -eSC2119 -eSC2120 -eSC2154 \ + -eSC2166 -eSC2181 -eSC2207 -eSC2231 -eSC2236 -eSC2268" + bash -c "find {toxinidir} \ + -not \( -type d -name .?\* -prune \) \ + -type f \ + -not -name \*~ \ + -not -name \*.md \ + \( -exec bash -c 'file \{\} | grep -q :.*shell' \; \ + -a ! -name '*.sh' \) \ + -print0 | xargs -r -n 1 -0 shellcheck \ + -eSC1091 -eSC2001 -eSC2002 -eSC2004 -eSC2006 -eSC2034 \ + -eSC2046 -eSC2064 -eSC2086 -eSC2119 -eSC2120 -eSC2154 \ + -eSC2166 -eSC2181 -eSC2207 -eSC2231 -eSC2236 -eSC2268 \ + -eSC1001 -eSC1009 -eSC1035 -eSC1072 -eSC1073 -eSC1090 \ + -eSC2003 -eSC2009 -eSC2015 -eSC2069 -eSC2112 -eSC2155 \ + -eSC2168 -eSC2219 -eSC2223 -eSC3005 -eSC3018 -eSC3020 \ + -eSC3037 -eSC3039 -eSC3043" + +[testenv:linters] +basepython = python3 +whitelist_externals = bash +commands = + {[testenv:bashate]commands} + {[testenv:yamllint]commands} + {[testenv:shellcheck]commands} + [testenv:pep8] basepython = python3 usedevelop = False