Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

style(shell): Address various shellcheck issues and formatting #3548

Open
wants to merge 52 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
8f192a3
CI(windows): Address shellcheck warnings in build_osgeo4w.sh
echoix Mar 30, 2024
7306341
test: Quote variables in test_thorough.sh
echoix Mar 30, 2024
bba2c8c
CI(ubuntu): Create makecmd variable as an array instead of a string t…
echoix Mar 30, 2024
b83fddf
CI(ubuntu): Sync structure of build_ubuntu-22.04_without_x.sh with bu…
echoix Mar 30, 2024
5368efb
docker(alpine): Incomplete shebang in docker/alpine/grass_tests.sh
echoix Mar 30, 2024
182478b
CI(macOS): Add missing quotes in make command
echoix Mar 30, 2024
865b79d
CI(macOS): Remove double quotes sandwiched in quoted CONFIGURE_FLAGS.
echoix Mar 30, 2024
ba3b9cd
docker: Uniformize shell scripts formatting
echoix Mar 30, 2024
9594172
binder: Double quote variables in postBuild
echoix Mar 30, 2024
882ea97
binder: Uniformize shell script formatting
echoix Mar 30, 2024
f2c4bdf
utils(module_synopsis): Use $(...) notation instead of legacy backtic…
echoix Mar 30, 2024
9089e1e
utils(module_synopsis): Exit if cd fails
echoix Mar 30, 2024
eb93acc
utils(module_synopsis): Double quote $(...) command substitutions
echoix Mar 30, 2024
9ceeaa4
utils(module_synopsis): Disable some shellcheck errors as false posit…
echoix Mar 30, 2024
00e99ae
utils(module_synopsis): Check exit code directly instead of with `$?`…
echoix Mar 30, 2024
556efc7
utils(module_synopsis): Uniformize shell script formatting
echoix Mar 30, 2024
cfd93f6
utils(module_synopsis): Disable SC2016 when single-quoting $GISBASE i…
echoix Mar 30, 2024
22e9007
Sort configure flags
echoix Nov 16, 2024
284d8d8
Merge branch 'main' into shellcheck-fix
echoix Nov 16, 2024
8c60635
travis: Format shell scripts with `shfmt -w -s -i 4 -ci -bn -sr`
echoix Nov 16, 2024
a7fe797
g.parser: Format shell script with `shfmt -w -s -i 4 -ci -bn -sr`
echoix Nov 16, 2024
12e92fd
g.parser: Add double quote to prevent globbing and word splitting (SC…
echoix Nov 17, 2024
ee39a20
utils(module_synopsis): Revert SC2016 exclusions
echoix Nov 17, 2024
ec1f378
i.topo.corr: Address shellcheck warnings and format with shfmt
echoix Nov 17, 2024
f5085ac
locale(transifex): Address shellcheck warnings and format with shfmt
echoix Nov 17, 2024
465b78d
mswindows(mklibs): Address shellcheck warnings
echoix Nov 17, 2024
4aa7694
mswindows(mklibs): Format with shfmt
echoix Nov 17, 2024
7f69537
vector: Address shellcheck warnings and format with shfmt
echoix Nov 17, 2024
9de6688
utils(vagrant): Address shellcheck warnings and format with shfmt
echoix Nov 17, 2024
3d3b773
utils(fix_typos): Address shellcheck warnings and format with shfmt
echoix Nov 17, 2024
8ada2c3
utils: Address shellcheck warnings and format with shfmt
echoix Nov 17, 2024
91e2773
install-sh: Address shellcheck warnings
echoix Nov 17, 2024
503bcee
install-sh: Format with shfmt
echoix Nov 17, 2024
f6d374f
install-sh: Address more shellcheck warnings
echoix Nov 17, 2024
a89c65d
testsuite: Format with shfmt
echoix Nov 17, 2024
95d06b5
r.colors: Address shellcheck warnings and format with shfmt
echoix Nov 17, 2024
859d6ed
r.sim.water: Address shellcheck warnings and format with shfmt
echoix Nov 17, 2024
b476ab8
raster: Address shellcheck warnings and format with shfmt
echoix Nov 17, 2024
46870ae
raster3d: Address shellcheck warnings and format with shfmt
echoix Nov 17, 2024
0b1ab69
scripts: Address shellcheck warnings and format with shfmt
echoix Nov 17, 2024
ee1721a
temporal: Address shellcheck warnings and format with shfmt
echoix Nov 17, 2024
bb905b7
python: Address shellcheck warnings and format with shfmt
echoix Nov 17, 2024
27415a7
mswindows: Address shellcheck warnings
echoix Nov 17, 2024
2176756
mswindows: Address shellcheck warnings and format with shfmt
echoix Nov 17, 2024
5b1e1b4
gui: Address shellcheck warnings and format with shfmt
echoix Nov 17, 2024
d4fcd17
lib: Address shellcheck warnings and format with shfmt
echoix Nov 17, 2024
d5b0f3b
macosx: Address shellcheck warnings and format with shfmt
echoix Nov 17, 2024
7b6f4b6
v.category: Address shellcheck warnings and format with shfmt
echoix Nov 17, 2024
1abbb07
Merge branch 'main' into shellcheck-fix
echoix Nov 23, 2024
0da3c1d
mswindows: Convert indentation to spaces in DLL list
echoix Nov 23, 2024
08a4fd8
mswindows: Sort DLL list
echoix Nov 23, 2024
4ddb7d0
mswindows: Revert some quoting of DLL list
echoix Nov 23, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions .github/workflows/build_osgeo4w.sh
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,9 @@ set -e
test -d "$1" && cd "$1"

export OSGEO4W_ROOT_MSYS=/c/OSGeo4W
export SRC=$(pwd)
SRC="$(pwd)"
export SRC
export UNITTEST=1

# Build according to OSGeo4W recipe
${SRC}/mswindows/osgeo4w/build_osgeo4w.sh
"${SRC}"/mswindows/osgeo4w/build_osgeo4w.sh
8 changes: 4 additions & 4 deletions .github/workflows/build_ubuntu-22.04.sh
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,10 @@ fi
# Adding -Werror to make's CFLAGS is a workaround for configuring with
# an old version of configure, which issues compiler warnings and
# errors out. This may be removed with upgraded configure.in file.
makecmd="make"
if [[ "$#" -ge 2 ]]; then
makecmd=("make")
if [[ $# -ge 2 ]]; then
ARGS=("$@")
makecmd="make CFLAGS='$CFLAGS ${ARGS[@]:1}' CXXFLAGS='$CXXFLAGS ${ARGS[@]:1}'"
makecmd=("make" "CFLAGS=$CFLAGS ${ARGS[@]:1}" "CXXFLAGS=$CXXFLAGS ${ARGS[@]:1}")
Comment on lines -22 to +25
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here, I'd suggest to refactor, to make things more clear:

# concatenate arguments, from the second up till the last
function get_args_str() {
    local args=("$@");
    for arg in "${args[@]:1}"; do a+="${arg} "; done; echo "${a% }"
}

# Adding -Werror to make's CFLAGS is a workaround for configuring with
# an old version of configure, which issues compiler warnings and
# errors out. This may be removed with upgraded configure.in file.
makecmd="make"
if [[ "$#" -ge 2 ]]; then
    args_str=$(get_args_str "$@")
    makecmd+=" CFLAGS='${CFLAGS} ${args_str}' CXXFLAGS='${CXXFLAGS} ${args_str}'"
fi

fi

# non-existent variables as an errors
Expand Down Expand Up @@ -52,5 +52,5 @@ export INSTALL_PREFIX=$1
--with-tiff \
--with-zstd

eval $makecmd
"${makecmd[@]}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"${makecmd[@]}"
eval "$makecmd"

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't quite the same. When having the quoted $makecmd, the arguments aren't split. Having unquoted $makecmd is problematic for globs, or all reasons why you'd want to quote variables.

Double quoted array with [@], is equivalent of double quoting each element. That would allow to properly handle a path with spaces as a single argument. (https://www.shellcheck.net/wiki/SC2068)

See https://www.shellcheck.net/wiki/SC2086 for general quoting tips (near the end), and https://www.shellcheck.net/wiki/SC2068 and https://www.shellcheck.net/wiki/SC2048 and
https://www.shellcheck.net/wiki/SC2294

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This goes with above suggestion.

make install
9 changes: 5 additions & 4 deletions .github/workflows/build_ubuntu-22.04_without_x.sh
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,10 @@ fi
# Adding -Werror to make's CFLAGS is a workaround for configuring with
# an old version of configure, which issues compiler warnings and
# errors out. This may be removed with upgraded configure.in file.
makecmd="make"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See build_ubuntu-22.04.sh

if [[ "$#" -eq 2 ]]; then
makecmd="make CFLAGS='$CFLAGS $2' CXXFLAGS='$CXXFLAGS $2'"
makecmd=("make")
if [[ $# -ge 2 ]]; then
ARGS=("$@")
makecmd=("make" "CFLAGS=$CFLAGS ${ARGS[@]:1}" "CXXFLAGS=$CXXFLAGS ${ARGS[@]:1}")
fi

# non-existent variables as an errors
Expand Down Expand Up @@ -50,5 +51,5 @@ export INSTALL_PREFIX=$1
--without-openmp \
--without-pthread

eval $makecmd
"${makecmd[@]}"
make install
11 changes: 5 additions & 6 deletions .github/workflows/macos_install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ CONFIGURE_FLAGS="\
--with-bzlib-libs=${CONDA_PREFIX}/lib \
--with-cairo \
--with-cairo-includes=${CONDA_PREFIX}/include/cairo \
--with-cairo-ldflags="-lcairo" \
--with-cairo-ldflags=-lcairo \
--with-cairo-libs=${CONDA_PREFIX}/lib \
--with-cxx \
--with-fftw-includes=${CONDA_PREFIX}/include \
Expand All @@ -33,7 +33,6 @@ CONFIGURE_FLAGS="\
--with-libpng=${CONDA_PREFIX}/bin/libpng-config \
--with-libs=${CONDA_PREFIX}/lib \
--with-netcdf=${CONDA_PREFIX}/bin/nc-config \
--with-netcdf=${CONDA_PREFIX}/bin/nc-config \
--with-nls \
--with-opengl=aqua \
--with-openmp \
Expand All @@ -46,7 +45,7 @@ CONFIGURE_FLAGS="\
--with-proj-share=${CONDA_PREFIX}/share/proj \
--with-readline \
--with-readline-includes=${CONDA_PREFIX}/include/readline \
--with-readline-libs=${CONDA_PREFIX}/lib
--with-readline-libs=${CONDA_PREFIX}/lib \
--with-sqlite \
--with-sqlite-includes=${CONDA_PREFIX}/include \
--with-sqlite-libs=${CONDA_PREFIX}/lib \
Expand All @@ -56,7 +55,7 @@ CONFIGURE_FLAGS="\
--with-zstd-includes=${CONDA_PREFIX}/include \
--with-zstd-libs=${CONDA_PREFIX}/lib \
--without-mysql \
--without-x \
--without-x
"

export CFLAGS="-O2 -pipe -arch ${CONDA_ARCH} -DGL_SILENCE_DEPRECATION -Wall -Wextra -Wpedantic -Wvla"
Expand All @@ -66,7 +65,7 @@ export CPPFLAGS="-isystem${CONDA_PREFIX}/include"
./configure $CONFIGURE_FLAGS
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
./configure $CONFIGURE_FLAGS
./configure "$CONFIGURE_FLAGS"

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This I wasn't sure that it would work on posix shell on Mac, and didn't know the complete side effects

Copy link
Contributor

@nilason nilason Apr 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What shellcheck suggest is the same on Mac.


EXEMPT=""
make -j$(sysctl -n hw.ncpu) CFLAGS="$CFLAGS -Werror $EXEMPT" \
CXXFLAGS="$CXXFLAGS -Werror $EXEMPT"
make -j"$(sysctl -n hw.ncpu)" CFLAGS="$CFLAGS -Werror $EXEMPT" \
CXXFLAGS="$CXXFLAGS -Werror $EXEMPT"

make install
6 changes: 3 additions & 3 deletions .github/workflows/test_thorough.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@
set -e

grass --tmp-project XY --exec \
g.download.project url=https://grass.osgeo.org/sampledata/north_carolina/nc_spm_full_v2alpha2.tar.gz path=$HOME
g.download.project url=https://grass.osgeo.org/sampledata/north_carolina/nc_spm_full_v2alpha2.tar.gz path="$HOME"

grass --tmp-project XY --exec \
python3 -m grass.gunittest.main \
--grassdata $HOME --location nc_spm_full_v2alpha2 --location-type nc \
--min-success 100 $@
--grassdata "$HOME" --location nc_spm_full_v2alpha2 --location-type nc \
--min-success 100 "$@"
6 changes: 3 additions & 3 deletions .travis/linux.install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,6 @@
set -e

sudo apt-get update -y
sudo apt-get install -y wget git gawk findutils
xargs -a <(awk '! /^ *(#|$)/' ".github/workflows/apt.txt") -r -- \
sudo apt-get install -y --no-install-recommends --no-install-suggests
sudo apt-get install -y wget git gawk findutils
xargs -a <(awk '! /^ *(#|$)/' ".github/workflows/apt.txt") -r -- \
sudo apt-get install -y --no-install-recommends --no-install-suggests
48 changes: 24 additions & 24 deletions .travis/linux.script.sh
Original file line number Diff line number Diff line change
Expand Up @@ -10,29 +10,29 @@ export MAKEFLAGS="-j $(nproc) --no-keep-going"
echo "MAKEFLAGS is '$MAKEFLAGS'"

./configure --host=x86_64-linux-gnu \
--build=x86_64-linux-gnu \
--enable-largefile \
--enable-shared \
--prefix=/usr/lib \
--sharedstatedir=/var \
--sysconfdir=/etc \
--with-blas \
--with-cairo \
--with-cxx \
--with-freetype \
--with-freetype-includes=/usr/include/freetype2/ \
--with-gdal \
--with-geos \
--with-lapack \
--with-netcdf \
--with-nls \
--with-odbc \
--with-pdal \
--with-postgres \
--with-postgres-includes=/usr/include/postgresql/ \
--with-proj-share=/usr/share/proj \
--with-readline \
--with-sqlite \
--with-zstd
--build=x86_64-linux-gnu \
--enable-largefile \
--enable-shared \
--prefix=/usr/lib \
--sharedstatedir=/var \
--sysconfdir=/etc \
--with-blas \
--with-cairo \
--with-cxx \
--with-freetype \
--with-freetype-includes=/usr/include/freetype2/ \
--with-gdal \
--with-geos \
--with-lapack \
--with-netcdf \
--with-nls \
--with-odbc \
--with-pdal \
--with-postgres \
--with-postgres-includes=/usr/include/postgresql/ \
--with-proj-share=/usr/share/proj \
--with-readline \
--with-sqlite \
--with-zstd

make CFLAGS="$CFLAGS $GRASS_EXTRA_CFLAGS" CXXFLAGS="$CXXFLAGS $GRASS_EXTRA_CXXFLAGS"
15 changes: 8 additions & 7 deletions binder/postBuild
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@ set -e
--with-bzlib \
--with-cairo \
--with-cxx \
--with-freetype=yes --with-freetype-includes="/usr/include/freetype2/" \
--with-freetype-includes="/usr/include/freetype2/" \
--with-freetype=yes \
--with-geos=/usr/bin/geos-config \
--with-nls \
--with-opengl-libs=/usr/include/GL \
Expand All @@ -20,13 +21,13 @@ make

# put command on path
# ensure the user specific bin dir exists (already on path)
mkdir -p $HOME/.local/bin/
mkdir -p "$HOME/.local/bin/"
# create link to build
ln -s $HOME/bin.*/grass* $HOME/.local/bin/grass
ln -s "$HOME"/bin.*/grass* "$HOME/.local/bin/grass"

# download a sample dataset
mkdir -p data/grassdata \
&& curl -SL https://grass.osgeo.org/sampledata/north_carolina/nc_basic_spm_grass7.zip > nc_basic_spm_grass7.zip \
&& unzip -qq nc_basic_spm_grass7.zip \
&& mv nc_basic_spm_grass7 data/grassdata \
&& rm nc_basic_spm_grass7.zip
&& curl -SL https://grass.osgeo.org/sampledata/north_carolina/nc_basic_spm_grass7.zip > nc_basic_spm_grass7.zip \
&& unzip -qq nc_basic_spm_grass7.zip \
&& mv nc_basic_spm_grass7 data/grassdata \
&& rm nc_basic_spm_grass7.zip
14 changes: 7 additions & 7 deletions docker/alpine/grass_tests.sh
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#/bin/bash
#!/bin/bash

# to be used in alpine Dockerfile

Expand All @@ -11,14 +11,14 @@ apk add --no-cache py3-scikit-learn
grass --tmp-project EPSG:25832 --exec r.in.pdal input="/tmp/simple.laz" output="count_1" method="n" resolution=1 -g

# Test GRASS GIS Python-addon installation
grass --tmp-project XY --exec g.extension extension=r.learn.ml2 operation=add && \
grass --tmp-project XY --exec g.extension extension=r.learn.ml2 operation=remove -f
grass --tmp-project XY --exec g.extension extension=r.learn.ml2 operation=add \
&& grass --tmp-project XY --exec g.extension extension=r.learn.ml2 operation=remove -f

# Test GRASS GIS C-addon installation: raster and vector
grass --tmp-project XY --exec g.extension extension=r.gwr operation=add && \
grass --tmp-project XY --exec g.extension extension=r.gwr operation=remove -f
grass --tmp-project XY --exec g.extension extension=v.centerpoint operation=add && \
grass --tmp-project XY --exec g.extension extension=v.centerpoint operation=remove -f
grass --tmp-project XY --exec g.extension extension=r.gwr operation=add \
&& grass --tmp-project XY --exec g.extension extension=r.gwr operation=remove -f
grass --tmp-project XY --exec g.extension extension=v.centerpoint operation=add \
&& grass --tmp-project XY --exec g.extension extension=v.centerpoint operation=remove -f

# cleanup dependency
apk del py3-scikit-learn
24 changes: 12 additions & 12 deletions docker/testdata/test_docker_image.sh
Original file line number Diff line number Diff line change
Expand Up @@ -36,31 +36,31 @@ grass --tmp-project EPSG:25832 --exec r.in.pdal input="/tmp/simple.laz" output="
printf "\n############\nTesting GRASS GIS Python-addon installation:\n############\n"
/usr/bin/python3 -m pip install --no-cache-dir scikit-learn

grass --tmp-project XY --exec g.extension extension=r.learn.ml2 operation=add && \
grass --tmp-project XY --exec g.extension extension=r.learn.ml2 operation=remove -f
grass --tmp-project XY --exec g.extension extension=r.learn.ml2 operation=add \
&& grass --tmp-project XY --exec g.extension extension=r.learn.ml2 operation=remove -f

# cleanup dependency
/usr/bin/python3 -m pip uninstall -y scikit-learn

# Test GRASS GIS C-addon installation: raster and vector
printf "\n############\nTesting GRASS GIS C-addon installation:\n############\n"
grass --tmp-project XY --exec g.extension extension=r.gwr operation=add && \
grass --tmp-project XY --exec g.extension extension=r.gwr operation=remove -f
grass --tmp-project XY --exec g.extension extension=v.centerpoint operation=add && \
grass --tmp-project XY --exec g.extension extension=v.centerpoint operation=remove -f
grass --tmp-project XY --exec g.extension extension=r.gwr operation=add \
&& grass --tmp-project XY --exec g.extension extension=r.gwr operation=remove -f
grass --tmp-project XY --exec g.extension extension=v.centerpoint operation=add \
&& grass --tmp-project XY --exec g.extension extension=v.centerpoint operation=remove -f

# show GRASS GIS, PROJ, GDAL etc versions
printf "\n############\nPrinting GRASS, PDAL and Python versions:\n############\n"
grass --tmp-project EPSG:4326 --exec g.version -rge && \
pdal --version && \
python3 --version
grass --tmp-project EPSG:4326 --exec g.version -rge \
&& pdal --version \
&& python3 --version

# Test presence of central python packages
printf "\n############\nPrinting versions of central python packages:\n############\n"
python3 -c "import psycopg2;import numpy as np;print(psycopg2.__version__);print(np.__version__)"

# Run testsuite
if [ $TESTSUITE ] ; then
printf "\n############\nRunning the testsuite:\n############\n"
bash /grassdb/.github/workflows/test_thorough.sh
if [ "$TESTSUITE" ]; then
printf "\n############\nRunning the testsuite:\n############\n"
bash /grassdb/.github/workflows/test_thorough.sh
fi
12 changes: 6 additions & 6 deletions general/g.parser/test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -25,27 +25,27 @@
# % required: no
# %end

if [ -z "$GISBASE" ] ; then
if [ -z "$GISBASE" ]; then
echo "You must be in GRASS GIS to run this program." 1>&2
exit 1
fi

if [ "$1" != "@ARGS_PARSED@" ] ; then
if [ "$1" != "@ARGS_PARSED@" ]; then
exec g.parser "$0" "$@"
fi

#### add your code below ####

echo ""

if [ $GIS_FLAG_F -eq 1 ] ; then
g.message message="Flag -f set"
if [ "$GIS_FLAG_F" -eq 1 ]; then
g.message message="Flag -f set"
else
g.message message="Flag -f not set"
g.message message="Flag -f not set"
fi

# test if parameter present:
if [ -n "$GIS_OPT_OPTION1" ] ; then
if [ -n "$GIS_OPT_OPTION1" ]; then
echo "Value of GIS_OPT_OPTION1: '$GIS_OPT_OPTION1'"
fi

Expand Down
5 changes: 3 additions & 2 deletions gui/wxpython/core/testsuite/toolboxes.sh
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
#!/usr/bin/env sh
# Tests generating toolboxes XML

# run make in gui/wxpython before the test
# run test using sh -e

python $GISBASE/gui/wxpython/core/toolboxes.py doctest
python $GISBASE/gui/wxpython/core/toolboxes.py test
python "$GISBASE/gui/wxpython/core/toolboxes.py" doctest
python "$GISBASE/gui/wxpython/core/toolboxes.py" test
Loading
Loading