Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[clang] -Wc++11-narrowing turned into error despite warning suppression (-w) #92630

Open
h-vetinari opened this issue May 18, 2024 · 23 comments
Open
Labels
clang:diagnostics New/improved warning or error message in Clang, but not in clang-tidy or static analyzer

Comments

@h-vetinari
Copy link
Contributor

I'm scratching my head at the following case, where we're trying to build the most recent tensorflow in conda-forge (tensorflow in itself is already a challenge to handle, but we have some distro-specificities on top that make for fun afternoons 😅), and running into the following on osx-64/osx-arm64 with clang 18.1.5 (also 16.0.6):

[8,945 / 11,584] Compiling xla/pjrt/pjrt_c_api_client.cc; 11s local ... (10 actions, 9 running)
ERROR: $BUILD_PREFIX/share/bazel/ba89d728a58893ddb77d57f3b72c9dba/external/local_xla/xla/translate/mhlo_to_hlo/BUILD:70:11: Compiling xla/translate/mhlo_to_hlo/mlir_hlo_to_hlo.cc failed: (Exit 1): cc_wrapper.sh failed: error executing command (from target @local_xla//xla/translate/mhlo_to_hlo:mlir_hlo_to_hlo) 
  (cd $BUILD_PREFIX/share/bazel/ba89d728a58893ddb77d57f3b72c9dba/execroot/org_tensorflow && \
  exec env - \
    PATH=$SRC_DIR:$BUILD_PREFIX/bin:$PREFIX/bin:/Users/ngam/tensorflow-feedstock/miniforge3/condabin:$BUILD_PREFIX/bin:$PREFIX/bin:/Users/ngam/tensorflow-feedstock/miniforge3/bin:/Users/ngam/.micromamba/condabin:/Users/ngam/.local/bin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/Library/Apple/usr/bin:/Users/ngam/.cabal/bin:/Users/ngam/.ghcup/bin \
    PWD=/proc/self/cwd \
    PYTHON_BIN_PATH=$PREFIX/bin/python \
    PYTHON_LIB_PATH=$PREFIX/lib/python3.11/site-packages \
    TF2_BEHAVIOR=1 \
    TF_SYSTEM_LIBS=astor_archive,astunparse_archive,boringssl,com_github_googlecloudplatform_google_cloud_cpp,com_github_grpc_grpc,com_google_absl,com_google_protobuf,curl,cython,dill_archive,flatbuffers,gast_archive,gif,icu,libjpeg_turbo,org_sqlite,png,pybind11,snappy,zlib \
  custom_toolchain/cc_wrapper.sh -MD -MF bazel-out/darwin_arm64-opt/bin/external/local_xla/xla/translate/mhlo_to_hlo/_objs/mlir_hlo_to_hlo/mlir_hlo_to_hlo.pic.d '-frandom-seed=bazel-out/darwin_arm64-opt/bin/external/local_xla/xla/translate/mhlo_to_hlo/_objs/mlir_hlo_to_hlo/mlir_hlo_to_hlo.pic.o' -fPIC '-DEIGEN_MAX_ALIGN_BYTES=64' -DEIGEN_ALLOW_UNALIGNED_SCALARS '-DEIGEN_USE_AVX512_GEMM_KERNELS=0' -DTF_USE_SNAPPY '-DLLVM_ON_UNIX=1' '-DHAVE_BACKTRACE=1' '-DBACKTRACE_HEADER=<execinfo.h>' '-DLTDL_SHLIB_EXT=".so"' '-DLLVM_PLUGIN_EXT=".so"' '-DLLVM_ENABLE_THREADS=1' '-DHAVE_DEREGISTER_FRAME=1' '-DHAVE_LIBPTHREAD=1' '-DHAVE_PTHREAD_GETNAME_NP=1' '-DHAVE_PTHREAD_H=1' '-DHAVE_PTHREAD_SETNAME_NP=1' '-DHAVE_REGISTER_FRAME=1' '-DHAVE_SETENV_R=1' '-DHAVE_STRERROR_R=1' '-DHAVE_SYSEXITS_H=1' '-DHAVE_UNISTD_H=1' '-DHAVE_MACH_MACH_H=1' '-DHAVE_MALLOC_MALLOC_H=1' '-DHAVE_MALLOC_ZONE_STATISTICS=1' '-DHAVE_PROC_PID_RUSAGE=1' '-DHAVE_UNW_ADD_DYNAMIC_FDE=1' '-DLLVM_NATIVE_ARCH="AArch64"' '-DLLVM_NATIVE_ASMPARSER=LLVMInitializeAArch64AsmParser' '-DLLVM_NATIVE_ASMPRINTER=LLVMInitializeAArch64AsmPrinter' '-DLLVM_NATIVE_DISASSEMBLER=LLVMInitializeAArch64Disassembler' '-DLLVM_NATIVE_TARGET=LLVMInitializeAArch64Target' '-DLLVM_NATIVE_TARGETINFO=LLVMInitializeAArch64TargetInfo' '-DLLVM_NATIVE_TARGETMC=LLVMInitializeAArch64TargetMC' '-DLLVM_NATIVE_TARGETMCA=LLVMInitializeAArch64TargetMCA' '-DLLVM_HOST_TRIPLE="arm64-apple-darwin"' '-DLLVM_DEFAULT_TARGET_TRIPLE="arm64-apple-darwin"' '-DLLVM_VERSION_MAJOR=19' '-DLLVM_VERSION_MINOR=0' '-DLLVM_VERSION_PATCH=0' '-DLLVM_VERSION_STRING="19.0.0git"' -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS '-DBLAKE3_USE_NEON=0' -DBLAKE3_NO_AVX2 -DBLAKE3_NO_AVX512 -DBLAKE3_NO_SSE2 -DBLAKE3_NO_SSE41 '-DBAZEL_CURRENT_REPOSITORY="local_xla"' -iquote external/local_xla -iquote bazel-out/darwin_arm64-opt/bin/external/local_xla -iquote external/com_google_absl -iquote bazel-out/darwin_arm64-opt/bin/external/com_google_absl -iquote external/eigen_archive -iquote bazel-out/darwin_arm64-opt/bin/external/eigen_archive -iquote external/ml_dtypes -iquote bazel-out/darwin_arm64-opt/bin/external/ml_dtypes -iquote external/local_tsl -iquote bazel-out/darwin_arm64-opt/bin/external/local_tsl -iquote external/nsync -iquote bazel-out/darwin_arm64-opt/bin/external/nsync -iquote external/double_conversion -iquote bazel-out/darwin_arm64-opt/bin/external/double_conversion -iquote external/com_google_protobuf -iquote bazel-out/darwin_arm64-opt/bin/external/com_google_protobuf -iquote external/snappy -iquote bazel-out/darwin_arm64-opt/bin/external/snappy -iquote external/com_googlesource_code_re2 -iquote bazel-out/darwin_arm64-opt/bin/external/com_googlesource_code_re2 -iquote external/llvm-project -iquote bazel-out/darwin_arm64-opt/bin/external/llvm-project -iquote external/stablehlo -iquote bazel-out/darwin_arm64-opt/bin/external/stablehlo -iquote external/farmhash_archive -iquote bazel-out/darwin_arm64-opt/bin/external/farmhash_archive -Ibazel-out/darwin_arm64-opt/bin/external/ml_dtypes/_virtual_includes/int4 -Ibazel-out/darwin_arm64-opt/bin/external/ml_dtypes/_virtual_includes/float8 -Ibazel-out/darwin_arm64-opt/bin/external/local_xla/xla/mlir_hlo/_virtual_includes/mlir_hlo -Ibazel-out/darwin_arm64-opt/bin/external/local_xla/xla/mlir_hlo/_virtual_includes/canonicalize_inc_gen -Ibazel-out/darwin_arm64-opt/bin/external/local_xla/xla/mlir_hlo/_virtual_includes/convert_op_folder -Ibazel-out/darwin_arm64-opt/bin/external/local_xla/xla/mlir_hlo/_virtual_includes/hlo_ops_attrs_inc_gen -Ibazel-out/darwin_arm64-opt/bin/external/local_xla/xla/mlir_hlo/_virtual_includes/hlo_ops_common -Ibazel-out/darwin_arm64-opt/bin/external/local_xla/xla/mlir_hlo/_virtual_includes/hlo_ops_enums_inc_gen -Ibazel-out/darwin_arm64-opt/bin/external/local_xla/xla/mlir_hlo/_virtual_includes/hlo_ops_inc_gen -Ibazel-out/darwin_arm64-opt/bin/external/local_xla/xla/mlir_hlo/_virtual_includes/hlo_ops_pattern_inc_gen -Ibazel-out/darwin_arm64-opt/bin/external/local_xla/xla/mlir_hlo/_virtual_includes/hlo_ops_typedefs_inc_gen -Ibazel-out/darwin_arm64-opt/bin/external/llvm-project/mlir/_virtual_includes/ArithCanonicalizationIncGen -Ibazel-out/darwin_arm64-opt/bin/external/llvm-project/mlir/_virtual_includes/AsmParserTokenKinds -Ibazel-out/darwin_arm64-opt/bin/external/llvm-project/mlir/_virtual_includes/MLIRShapeCanonicalizationIncGen -Ibazel-out/darwin_arm64-opt/bin/external/stablehlo/_virtual_includes/base -Ibazel-out/darwin_arm64-opt/bin/external/stablehlo/_virtual_includes/base_attr_interfaces_inc_gen -Ibazel-out/darwin_arm64-opt/bin/external/stablehlo/_virtual_includes/broadcast_utils -Ibazel-out/darwin_arm64-opt/bin/external/stablehlo/_virtual_includes/chlo_ops -Ibazel-out/darwin_arm64-opt/bin/external/stablehlo/_virtual_includes/chlo_attrs_inc_gen -Ibazel-out/darwin_arm64-opt/bin/external/stablehlo/_virtual_includes/chlo_enums_inc_gen -Ibazel-out/darwin_arm64-opt/bin/external/stablehlo/_virtual_includes/chlo_ops_inc_gen -Ibazel-out/darwin_arm64-opt/bin/external/stablehlo/_virtual_includes/stablehlo_type_inference -Ibazel-out/darwin_arm64-opt/bin/external/stablehlo/_virtual_includes/stablehlo_assembly_format -Ibazel-out/darwin_arm64-opt/bin/external/local_xla/xla/mlir_hlo/_virtual_includes/lhlo_gpu -Ibazel-out/darwin_arm64-opt/bin/external/local_xla/xla/mlir_hlo/_virtual_includes/lhlo -Ibazel-out/darwin_arm64-opt/bin/external/local_xla/xla/mlir_hlo/_virtual_includes/lhlo_ops_inc_gen -Ibazel-out/darwin_arm64-opt/bin/external/local_xla/xla/mlir_hlo/_virtual_includes/lhlo_ops_structs_inc_gen -Ibazel-out/darwin_arm64-opt/bin/external/local_xla/xla/mlir_hlo/_virtual_includes/lhlo_structured_interface -Ibazel-out/darwin_arm64-opt/bin/external/local_xla/xla/mlir_hlo/_virtual_includes/lhlo_structured_interface_inc_gen -Ibazel-out/darwin_arm64-opt/bin/external/local_xla/xla/mlir_hlo/_virtual_includes/lhlo_gpu_ops_attrdefs_inc_gen -Ibazel-out/darwin_arm64-opt/bin/external/local_xla/xla/mlir_hlo/_virtual_includes/lhlo_gpu_ops_dialect_inc_gen -Ibazel-out/darwin_arm64-opt/bin/external/local_xla/xla/mlir_hlo/_virtual_includes/lhlo_gpu_ops_enums_inc_gen -Ibazel-out/darwin_arm64-opt/bin/external/local_xla/xla/mlir_hlo/_virtual_includes/lhlo_gpu_ops_inc_gen -Ibazel-out/darwin_arm64-opt/bin/external/local_xla/xla/mlir_hlo/_virtual_includes/lhlo_gpu_ops_ops -Ibazel-out/darwin_arm64-opt/bin/external/local_xla/xla/mlir_hlo/_virtual_includes/mhlo_passes -Ibazel-out/darwin_arm64-opt/bin/external/local_xla/xla/mlir_hlo/_virtual_includes/chlo_legalize_to_hlo -Ibazel-out/darwin_arm64-opt/bin/external/local_xla/xla/mlir_hlo/_virtual_includes/chlo_legalize_to_hlo_inc_gen -Ibazel-out/darwin_arm64-opt/bin/external/local_xla/xla/mlir_hlo/_virtual_includes/map_chlo_to_hlo_op -Ibazel-out/darwin_arm64-opt/bin/external/local_xla/xla/mlir_hlo/_virtual_includes/hlo_legalize_to_stablehlo -Ibazel-out/darwin_arm64-opt/bin/external/local_xla/xla/mlir_hlo/_virtual_includes/map_stablehlo_to_hlo_op -Ibazel-out/darwin_arm64-opt/bin/external/stablehlo/_virtual_includes/stablehlo_ops -Ibazel-out/darwin_arm64-opt/bin/external/stablehlo/_virtual_includes/stablehlo_attrs_inc_gen -Ibazel-out/darwin_arm64-opt/bin/external/stablehlo/_virtual_includes/stablehlo_enums_inc_gen -Ibazel-out/darwin_arm64-opt/bin/external/stablehlo/_virtual_includes/stablehlo_ops_inc_gen -Ibazel-out/darwin_arm64-opt/bin/external/stablehlo/_virtual_includes/stablehlo_types_inc_gen -Ibazel-out/darwin_arm64-opt/bin/external/local_xla/xla/mlir_hlo/_virtual_includes/legalize_to_linalg_utils -Ibazel-out/darwin_arm64-opt/bin/external/local_xla/xla/mlir_hlo/_virtual_includes/map_mhlo_to_scalar_op -Ibazel-out/darwin_arm64-opt/bin/external/local_xla/xla/mlir_hlo/_virtual_includes/legalize_to_standard_inc_gen -Ibazel-out/darwin_arm64-opt/bin/external/local_xla/xla/mlir_hlo/_virtual_includes/lower_complex_inc_gen -Ibazel-out/darwin_arm64-opt/bin/external/local_xla/xla/mlir_hlo/_virtual_includes/map_hlo_to_lhlo_op -Ibazel-out/darwin_arm64-opt/bin/external/local_xla/xla/mlir_hlo/_virtual_includes/mhlo_pass_inc_gen -Ibazel-out/darwin_arm64-opt/bin/external/local_xla/xla/mlir_hlo/_virtual_includes/mhlo_rng_utils -Ibazel-out/darwin_arm64-opt/bin/external/local_xla/xla/mlir_hlo/_virtual_includes/mhlo_scatter_gather_utils -Ibazel-out/darwin_arm64-opt/bin/external/local_xla/xla/mlir_hlo/_virtual_includes/shape_component_analysis -Ibazel-out/darwin_arm64-opt/bin/external/local_xla/xla/mlir_hlo/_virtual_includes/stablehlo_legalize_to_hlo -Ibazel-out/darwin_arm64-opt/bin/external/local_xla/xla/mlir_hlo/_virtual_includes/type_conversion -Ibazel-out/darwin_arm64-opt/bin/external/local_xla/xla/mlir_hlo/_virtual_includes/unfuse_batch_norm -isystem external/eigen_archive -isystem bazel-out/darwin_arm64-opt/bin/external/eigen_archive -isystem external/eigen_archive/mkl_include -isystem bazel-out/darwin_arm64-opt/bin/external/eigen_archive/mkl_include -isystem external/ml_dtypes -isystem bazel-out/darwin_arm64-opt/bin/external/ml_dtypes -isystem external/ml_dtypes/ml_dtypes -isystem bazel-out/darwin_arm64-opt/bin/external/ml_dtypes/ml_dtypes -isystem external/nsync/public -isystem bazel-out/darwin_arm64-opt/bin/external/nsync/public -isystem external/llvm-project/llvm/include -isystem bazel-out/darwin_arm64-opt/bin/external/llvm-project/llvm/include -isystem external/llvm-project/mlir/include -isystem bazel-out/darwin_arm64-opt/bin/external/llvm-project/mlir/include -isystem external/farmhash_archive/src -isystem bazel-out/darwin_arm64-opt/bin/external/farmhash_archive/src -isystem $PREFIX/include -ftree-vectorize -fPIC -fstack-protector-strong -O2 -pipe -isystem $PREFIX/include '-fdebug-prefix-map=$SRC_DIR=/usr/local/src/conda/tensorflow-split-2.16.1' '-fdebug-prefix-map=$PREFIX=/usr/local/src/conda-prefix' '-D_FORTIFY_SOURCE=2' -isystem $PREFIX/include '-mmacosx-version-min=11.0' '-mmacosx-version-min=11.0' -ftree-vectorize -fPIC -fstack-protector-strong -O2 -pipe '-stdlib=libc++' -fvisibility-inlines-hidden '-fmessage-length=0' -isystem $PREFIX/include '-fdebug-prefix-map=$SRC_DIR=/usr/local/src/conda/tensorflow-split-2.16.1' '-fdebug-prefix-map=$PREFIX=/usr/local/src/conda-prefix' '-D_FORTIFY_SOURCE=2' -isystem $PREFIX/include '-mmacosx-version-min=11.0' '-mmacosx-version-min=11.0' -DGRPC_BAZEL_BUILD -w '-std=c++17' -c external/local_xla/xla/translate/mhlo_to_hlo/mlir_hlo_to_hlo.cc -o bazel-out/darwin_arm64-opt/bin/external/local_xla/xla/translate/mhlo_to_hlo/_objs/mlir_hlo_to_hlo/mlir_hlo_to_hlo.pic.o)
# Configuration: 26ccd9219187bc442457d10de0c11bfb787234d944869597e4191d621d502366
# Execution platform: @local_execution_config_platform//:platform
external/local_xla/xla/translate/mhlo_to_hlo/mlir_hlo_to_hlo.cc:176:35: error: non-constant-expression cannot be narrowed from type 'uint64_t' (aka 'unsigned long long') to 'unsigned char' in initializer list [-Wc++11-narrowing]
  176 |         array.data()[i] = xla::u4{values[i].getZExtValue()};
      |                                   ^~~~~~~~~~~~~~~~~~~~~~~~
external/local_xla/xla/translate/mhlo_to_hlo/mlir_hlo_to_hlo.cc:176:35: note: insert an explicit cast to silence this issue
  176 |         array.data()[i] = xla::u4{values[i].getZExtValue()};
      |                                   ^~~~~~~~~~~~~~~~~~~~~~~~
      |                                   static_cast<unsigned char>( )
1 error generated.

What's causing me surprise is that part of the invocation is -w (next to the -std=c++17), which is documented to "Suppress all warnings". That should be the absolute antithesis to turning warnings into errors, so I really don't know how -Wc++11-narrowing ends up failing this build (which also runs fine on linux with GCC despite tighter error settings than clang on osx)). I found a couple of old issues related to -Wc++11-narrowing, but nothing that stood out to me - apologies if this ends up being a duplicate.

FWIW, tensorflow specifies is settings mainly here, though how flags get piped through bazel is not super-obvious (especially as the failure is happening in a vendored subfolder that itself has bazel build instructions).

PS. the compiler-wrapper being used does not touch the warning flags.

@github-actions github-actions bot added the clang Clang issues not falling into any other category label May 18, 2024
@EugeneZelenko EugeneZelenko added clang:diagnostics New/improved warning or error message in Clang, but not in clang-tidy or static analyzer and removed clang Clang issues not falling into any other category labels May 18, 2024
@shafik
Copy link
Collaborator

shafik commented May 19, 2024

It looks like -Wno-c++11-narrowing works: https://godbolt.org/z/ze8ajzYrz

We are treating it as ill-formed which is an error, not a warning.

CC @AaronBallman @erichkeane

@erichkeane
Copy link
Collaborator

Perhaps our docs need to be updated?

It SEEMS to me (though I haven't dug into it too much), that our warnings-as-default-errors (which are supposed to be 'things that are really errors, but we cant really make unignorable quite yet) are not suppressed with that -w flag. I think that is a sensible behavior, but probably needs to be documented.

FWIW, I think of warnings-as-default-errors to be more suppressible error instead of warning that we are serious about, which is why it makes sense to me.

@AaronBallman
Copy link
Collaborator

I had discovered this a while back as well -- -w disables all warnings but it does not disable warnings that have been upgraded into errors. This is confusing (IMO) because -w will ignore warnings turned into errors via -Werror. e.g. https://godbolt.org/z/YsacvWqhG

However, it does appear to be by design:

  // Honor -w: this disables all messages which are not Error/Fatal by
  // default (disregarding attempts to upgrade severity from Warning to Error),
  // as well as disabling all messages which are currently mapped to Warning
  // (whether by default or downgraded from Error via e.g. -Wno-error or #pragma
  // diagnostic.)
  if (State->IgnoreAllWarnings) {
    if (Result == diag::Severity::Warning ||
        (Result >= diag::Severity::Error &&
         !isDefaultMappingAsError((diag::kind)DiagID)))
      return diag::Severity::Ignored;
  }

@jyknight, you authored 3f8b916 -- was there a reason why your changes didn't include warnings that default to an error?

@MaskRay
Copy link
Member

MaskRay commented May 29, 2024

I think this is working-as-intended.

Clang's -w matches GCC's semantics: -w suppresses warnings but not DefaultError diagnostics.
For example, the following two errors cannot be suppressed with gcc -w.
(The -Wnarrowing one can be suppressed before GCC 14.1 (https://inbox.sourceware.org/gcc-patches/[email protected]/)).

bool foo(int *x, int y) {
  return x > y; // error: ISO C++ forbids comparison between pointer and integer [-fpermissive]
}

int x = {4.2}; // error: narrowing conversion of ‘4.2000000000000002e+0’ from ‘double’ to ‘int’ [-Wnarrowing]

I believe Clang has much more DefaultError diagnostics than GCC and therefore non-conforming code might feel Clang is "more noisy".

The GCC patch description says:

In the discussion of promoting some pedwarns to be errors by default, rather than move them all into -fpermissive it seems to me to make sense to support DK_PERMERROR with an option flag.

In the future, we might see more GCC DefaultError diagnostics that cannot be suppressed with -w.
They can still be suppressed with -fpermissive, which Clang doesn't support.

@MaskRay MaskRay closed this as not planned Won't fix, can't repro, duplicate, stale May 29, 2024
@h-vetinari
Copy link
Contributor Author

h-vetinari commented May 29, 2024

This is non-sensical. Don't call something a warning if it's an error (by default). If however it is just a warning (e.g. the compiler will produce something highly probable to be useful), then -w must disable it.

-w is an intentionally big hammer. Don't make users run after individual warnings, like "no, this warning is too special to be covered by -w" (even more so if there's no alternative like -fpermissive).

@MaskRay
Copy link
Member

MaskRay commented May 29, 2024

This is non-sensical. Don't call something a warning if it's an error (by default). If however it is just a warning (e.g. the compiler will produce something highly probable to be useful), then -w must disable it.

-w is an intentionally big hammer. Don't make users run after individual warnings, like "no, this warning is too special to be covered by -w" (even more so if there's no alternative like -fpermissive).

The big hammer no longer works with GCC 14's -Wnarrowing:

% cat n.cc
int x = {4.2};
% gcc -c n.cc -w
% ~/Dev/gcc/out/debug/gcc/xgcc -B$HOME/Dev/gcc/out/debug/gcc -c x.cc -w
x.cc: In function ‘void foo()’:
x.cc:3:12: error: narrowing conversion of ‘4.2000000000000002e+0’ from ‘double’ to ‘int’ [-Wnarrowing]
    3 |   int x = {4.2};
      |

I believe over time GCC will have more DefaultError diagnostics that can be suppressed with -Wno-xxx but not with -w.

@EugeneZelenko EugeneZelenko added the invalid Resolved as invalid, i.e. not a bug label May 29, 2024
@AaronBallman
Copy link
Collaborator

In the future, we might see more GCC DefaultError diagnostics that cannot be suppressed with -w.
They can still be suppressed with -fpermissive, which Clang doesn't support.

I disagree with your reasoning that this is by design -- we cannot point to GCC's behavior as rationale here. -fpermissive is a language dialect flag and it would be deeply strange for a warning flag to override a language dialect flag. e.g., that would be like -Wno-some-openmp-thing deciding to override -fopenmp. Clang does not (and will never) support -fpermissive; instead we surface this as individual diagnostics that can be downgraded from an error to a warning. In that case, expecting -w and -Wno- to override behavior of other warning flags is quite natural.

I'm still trying to dig up any information that suggests this behavior is by design in Clang and I've yet to find any. The fact that this issue keeps being raised by users as we add more default-error warnings suggests that we should strongly consider changing our behavior. Reopening this issue.

@AaronBallman AaronBallman reopened this May 29, 2024
@AaronBallman AaronBallman removed the invalid Resolved as invalid, i.e. not a bug label May 29, 2024
@erichkeane
Copy link
Collaborator

In the future, we might see more GCC DefaultError diagnostics that cannot be suppressed with -w.
They can still be suppressed with -fpermissive, which Clang doesn't support.

I disagree with your reasoning that this is by design -- we cannot point to GCC's behavior as rationale here. -fpermissive is a language dialect flag and it would be deeply strange for a warning flag to override a language dialect flag. e.g., that would be like -Wno-some-openmp-thing deciding to override -fopenmp. Clang does not (and will never) support -fpermissive; instead we surface this as individual diagnostics that can be downgraded from an error to a warning. In that case, expecting -w and -Wno- to override behavior of other warning flags is quite natural.

I'm still trying to dig up any information that suggests this behavior is by design in Clang and I've yet to find any. The fact that this issue keeps being raised by users as we add more default-error warnings suggests that we should strongly consider changing our behavior. Reopening this issue.

I can actually see this both ways, but it depends what you think of when you see "warning-as-default-error".

If you consider it: "A very serious warning, we've decided to make stop your build by default", I could see expecting -w to work.

If you consider it: "An error that we need folks to be able to skip sometimes, so we use the warning infrastructure to do this", -w should NOT work.

I lean towards the latter, we typically do warning-as-default-error for things that we want to be a true error, but that there are a handful of usecases in the wild that we want to give time to change. From that perspective, I think it makes sense for -w to not work: We want these to be individually acknowledged, because we one day want them to be a true error. THIS would make -w harmful to being able to mark them as a true error some day.

@AaronBallman
Copy link
Collaborator

I can squint to see it both ways. However, I'm not convinced it matters which way you squint at it. We should not require our users guess our intention as to whether a diagnostic is a serious warning or an unserious error. To me, the important thing is that these are warning flags -- we expose them with a -Wwhatever group and that's how users interact with them -- if it's exposed as a warning flag, the knobs that control warning flags should work. IOW, I think we have ample evidence in our issue tracker that users expect -w and #pragma clang diagnostic(ignore) to both disable these because they're warning diagnostic flags with warning groups.

I think -w and #pragma clang diagnostic(ignore) are morally equivalent to other "catch-all" diagnostic behavior flags like -Weverything and -Werror; if you use them, you get exactly what's written on the tin, and that sometimes means you didn't want to use them after all.

@erichkeane
Copy link
Collaborator

I can squint to see it both ways. However, I'm not convinced it matters which way you squint at it. We should not require our users guess our intention as to whether a diagnostic is a serious warning or an unserious error. To me, the important thing is that these are warning flags -- we expose them with a -Wwhatever group and that's how users interact with them -- if it's exposed as a warning flag, the knobs that control warning flags should work. IOW, I think we have ample evidence in our issue tracker that users expect -w and #pragma clang diagnostic(ignore) to both disable these because they're warning diagnostic flags with warning groups.

I think -w and #pragma clang diagnostic(ignore) are morally equivalent to other "catch-all" diagnostic behavior flags like -Weverything and -Werror; if you use them, you get exactly what's written on the tin, and that sometimes means you didn't want to use them after all.

I just think that is a shame to do in this case. Yes, it surprises users, but the point of warnings-as-errors (because we want to make them a real boy error someday!) IS to surprise the users. If we allow this suppression, they are REALLY going to be surprised, just a few cycles later when they have zero time to fix the issue with their code (because it went from "we ignored this warning" to "hard error").

Effectively: Warnings-as-default-errors are special. They have an intrinsic meaning of "we are going to break you for real someday ". They are thus more serious than a normal warning, and should have special care taken around them.

@AaronBallman
Copy link
Collaborator

Effectively: Warnings-as-default-errors are special. They have an intrinsic meaning of "we are going to break you for real someday".

No, they don't. We have 56 diagnostics that are DefaultError and many of them are unlikely to ever change to be an error only (for a variety of reasons, such as too disruptive compared to the benefit or are an error now but will relax that error once support is added, etc), like:

def ext_dynamic_exception_spec : ExtWarn<
  "ISO C++17 does not allow dynamic exception specifications">,
  InGroup<DynamicExceptionSpec>, DefaultError;

def ext_initializer_union_overrides : ExtWarn<warn_initializer_overrides.Summary>,
  InGroup<InitializerOverrides>, DefaultError, SFINAEFailure;

def ext_implicit_function_decl_c99 : ExtWarn<
  "call to undeclared function %0; ISO C99 and later do not support implicit "
  "function declarations">, InGroup<ImplicitFunctionDeclare>, DefaultError;
(and several other related diagnostics)

and so on.

I took a look through the list and I would say there's not really a "most diagnostics do X" kind of grouping for them. There's a mixture of extensions and standards features, C and C++ features, ones we may want to turn into an error only someday and ones we may want to leave as downgradable, etc.

@erichkeane
Copy link
Collaborator

Effectively: Warnings-as-default-errors are special. They have an intrinsic meaning of "we are going to break you for real someday".

No, they don't. We have 56 diagnostics that are DefaultError and many of them are unlikely to ever change to be an error only (for a variety of reasons, such as too disruptive compared to the benefit or are an error now but will relax that error once support is added, etc), like:

def ext_dynamic_exception_spec : ExtWarn<
  "ISO C++17 does not allow dynamic exception specifications">,
  InGroup<DynamicExceptionSpec>, DefaultError;

def ext_initializer_union_overrides : ExtWarn<warn_initializer_overrides.Summary>,
  InGroup<InitializerOverrides>, DefaultError, SFINAEFailure;

def ext_implicit_function_decl_c99 : ExtWarn<
  "call to undeclared function %0; ISO C99 and later do not support implicit "
  "function declarations">, InGroup<ImplicitFunctionDeclare>, DefaultError;
(and several other related diagnostics)

and so on.

I took a look through the list and I would say there's not really a "most diagnostics do X" kind of grouping for them. There's a mixture of extensions and standards features, C and C++ features, ones we may want to turn into an error only someday and ones we may want to leave as downgradable, etc.

I did a look through the list when I posted that above, and my impression was roughly 1/2 were "we would love to make these full errors someday...". That said, it is definitely a strategy we have used for cases where we wish we could make them errors, ,with the hope we could 'someday'.

@AaronBallman
Copy link
Collaborator

That said, it is definitely a strategy we have used for cases where we wish we could make them errors, ,with the hope we could 'someday'.

Definitely agreed!

But again, our users should not have to make this distinction; that's an internal implementation detail. We expose these as warnings with a warning group and -Wno- support, so I'm still not seeing a compelling reason why -w should not suppress them as with any other warning. The only reason so far is "it might be useful behavior for roughly half the impacted diagnostics for -w to not suppress the diagnostic".

Also worth considering is that we have errors that users upgrade into warnings, and -w does suppress those warnings: https://godbolt.org/z/qjs44sEor DefaultError being equivalent to -Werror= for that particular diagnostic is quite easy for users and maintainers to reason about.

@erichkeane
Copy link
Collaborator

That said, it is definitely a strategy we have used for cases where we wish we could make them errors, ,with the hope we could 'someday'.

Definitely agreed!

But again, our users should not have to make this distinction; that's an internal implementation detail. We expose these as warnings with a warning group and -Wno- support, so I'm still not seeing a compelling reason why -w should not suppress them as with any other warning. The only reason so far is "it might be useful behavior for roughly half the impacted diagnostics for -w to not suppress the diagnostic".

Also worth considering is that we have errors that users upgrade into warnings, and -w does suppress those warnings: https://godbolt.org/z/qjs44sEor DefaultError being equivalent to -Werror= for that particular diagnostic is quite easy for users and maintainers to reason about.

I'm concerned about the following situation:

1- Tons of users use -w because they are comfortable with the state of their codebase regarding warnings, but still like to update their compiler.

2- We have something we REALLY want to be an error, but the transition to doing so requires alerting our users of it so they can have time to fix it (perhaps because the 'fix' for it is non-trivial amounts of work). Historically we use warning-as-default-error here. -w users don't end up getting this warning.

3- After said transition period, we make it a 'real' error. All the non -w users are fine, since they got a significant notice of this, and presumably had a few cycles to fix this. All of the -w users are in big trouble, they now no longer build at all, and have to panic-fix their code if they want it to still compile.

Changing -w to suppress default-errors ends up silently changing its meaning to subtly opt-into the problems in #3. We could perhaps document -w to say, "Suppress all warnings, plus sign you up for giant panic-headaches in the future", but that is perhaps a little silly.

@AaronBallman
Copy link
Collaborator

3- After said transition period, we make it a 'real' error. All the non -w users are fine, since they got a significant notice of this, and presumably had a few cycles to fix this. All of the -w users are in big trouble, they now no longer build at all, and have to panic-fix their code if they want it to still compile.

On the one hand, I agree with your concern. However, IMO, -Werror=whatever -w needs to behave consistently with -w and a warning that defaults to an error. That can be consistent in one of two ways: for warnings which default to an error and for warnings upgraded via -Werror, either -w disables the diagnostic in both cases or it does not disable the diagnostic in either case. Our current behavior where it sometimes does one and sometimes does the other is not reasonable IMO.

If the user specified -w, then the behavior the user asked for is to disable all warnings. Users who disable all warnings are electing to ignore (potential) problems in their code, so when those problems turn out to bite them later... that's what the user asked for. We're not opinionated about what diagnostics we enable with -Weverything when the user says "give me all warnings, even if that will bite me later" though we could be. Similarly, I don't believe we should be opinionated when the user says "give me no warnings, even if that will bite me later" even though we could be.

@erichkeane
Copy link
Collaborator

3- After said transition period, we make it a 'real' error. All the non -w users are fine, since they got a significant notice of this, and presumably had a few cycles to fix this. All of the -w users are in big trouble, they now no longer build at all, and have to panic-fix their code if they want it to still compile.

On the one hand, I agree with your concern. However, IMO, -Werror=whatever -w needs to behave consistently with -w and a warning that defaults to an error. That can be consistent in one of two ways: for warnings which default to an error and for warnings upgraded via -Werror, either -w disables the diagnostic in both cases or it does not disable the diagnostic in either case. Our current behavior where it sometimes does one and sometimes does the other is not reasonable IMO.

I consider them to be somewhat different circumstances, but can see where you are coming from. I wouldn't be opposed to changing the Werror behavior, but I'm not strongly favored one way or the other.

If the user specified -w, then the behavior the user asked for is to disable all warnings. Users who disable all warnings are electing to ignore (potential) problems in their code, so when those problems turn out to bite them later... that's what the user asked for. We're not opinionated about what diagnostics we enable with -Weverything when the user says "give me all warnings, even if that will bite me later" though we could be. Similarly, I don't believe we should be opinionated when the user says "give me no warnings, even if that will bite me later" even though we could be.

If we are willing to take the messaging (and perhaps document it!) that -w is like -Weverything, where "you put fireworks down your pants, you deserve anything you get", I could be ok with having it suppress these diagnostics. It seems user hostile (though in a case where the user is trying to opt into that), but I could be ok with it. That said, even if we diagnose it, users who currently use -w will be harmed.

I don't have a good feel of who uses -w, but I suspect it is folks with an older codebase they don't maintain, so its a "we don't care about warnings", and I feel that the warning-to-error case (3 above) is PARTICULARLY harmful to them, but I also hope/suspect it is rarely enough used that the "fireworks" response above is sufficient.

@jyknight
Copy link
Member

I'm definitely in favor of the keeping the status-quo. I consider default-error diagnostics as "errors that we allow people to opt out of", not "warnings we happen to error on". Yes, we spell the opt-out using the same diagnostic framework as for warnings, because that's useful and convenient, but they are really a different category of issue, in my mind.

Thus, I think it's entirely reasonable and defensible for -w to mean: disable all warnings, but don't disable errors-that-can-be-opted-out-of. Not only for GCC compatibility (although that's a component), but also because it's just a useful behavior.

I'll note that we do provide -Wno-everything if you really really really want NO diagnostics except hard-errors. I don't recommend anyone use that, but it's there, and it does provide this functionality...

@AaronBallman
Copy link
Collaborator

I'm definitely in favor of the keeping the status-quo.

The status quo is inconsistent and I think that needs to change. I'm a strong -1 for keeping the status quo (broadly speaking); I want -w to be consistent regarding diagnostics that have been upgraded to errors and that's not the case with the status quo (it silences warnings the implementation upgrades into an error, it does not silence warnings the command line upgrades into an error).

Thus, I think it's entirely reasonable and defensible for -w to mean: disable all warnings, but don't disable errors-that-can-be-opted-out-of.

I think it's equally as reasonable and defensible for it to mean "disable all warnings" and we don't make users differentiate between warnings upgraded to errors by the implementation and warnings upgraded to errors by the build script.

Yes, we spell the opt-out using the same diagnostic framework as for warnings, because that's useful and convenient, but they are really a different category of issue, in my mind.

I agree that they're different categories to us as compiler authors. However, we are not typical users and that's important to keep in mind. Stepping back:

int main() { // this is \ 
                a bad comment.

}

the diagnostic the user gets for this code is:

error: backslash and newline separated by space [-Werror,-Wbackslash-newline-escape]

and given:

struct S {
  static const float f = 12;
};

the diagnostic the user gets for this code is:

error: in-class initializer for static data member of type 'const float' requires 'constexpr' specifier [-Wstatic-float-init]

In both cases, we give a -W flag at the end of the diagnostic, but we don't add -Werror in the latter case while we do in the former. So there is a very subtle visual distinction for the user to pick up on to help them distinguish between the cases. However, we've trained users that for anything which you have -Wfoo, there is a -Wno-foo sibling to disable the diagnostic. And we document -w as "suppress all warnings". I think the fact that this issue has been reported several times to us (#92630, #93474, I know there's at least one more but I'm not able to find it because searching for -w is not something GitHub wants to do) tells us that users see -Wfoo for the diagnostic to mean that "suppress all warnings" includes those diagnostics. And the fact that -Werror -w does suppress those diagnostics but nobody has reported that as an issue to us (that I've been able to find, anyway) tells us that users don't see the -w behavior of disabling warnings treated as errors as being confusing.

If something has to give (and I think it does because I really don't think the status quo is defensible in its inconsistency), then I think making -w not silence -Werror diagnostics will be more disruptive than -w silencing defaulted errors. If -w stops silencing -Werror diagnostics, some users who upgrade versions of Clang will break. However, the reverse is not true -- if the user already gets an error today, then allowing -w to silence it can only cause us to accept code rather than reject already working code.

@h-vetinari
Copy link
Contributor Author

Tons of users use -w because they are comfortable with the state of their codebase regarding warnings, but still like to update their compiler.

I don't think this hypothetical scenario is giving users' ability to make an informed decision enough credit. That things can break if you ignore all warnings and update your compiler is such an obvious consequence that we should afford users the benefit of the doubt that this is intentional; IOW, the fact that this might happen does not invalidate the use cases for actually ignoring all warnings.

Concrete use case

Following the example from the OP: Tensorflow documents their use of -w (uniformly across all platforms) as "Suppress all C++ compiler warnings, otherwise build logs become 10s of MBs." Obviously they are responsible for ensuring that their project is in a usable state (despite their use of -w) when they release. Now lets say a user (or a distributor, in my case) wants to build their tagged version from source.

In this case, I really want the intent of the upstream developers to be honoured. The alternative is a failed build after several hours (TF is heavy...), find the "special warning", disable it, repeat. Needless to say this is a frustrating waste of time - I do not want to be "protected" from upstream's explicit choice, especially for something as (comparatively) benign as a narrowing conversion, which has a well-defined meaning even if it is lossy.

It's the responsibility of the tensorflow developers to ensure their code does the correct thing (and they have multiple platforms/compilers that provide signals on this), and if indeed a new compiler version errors hard on a newly-diagnosed bug, it will be fixed upstream (or as a user, I stay on an older compiler for the time being which "works"). In either case, it's not appropriate for the compiler to second-guess the intent à la "I don't think you really meant all warnings 😉".

@jyknight
Copy link
Member

The status quo is inconsistent and I think that needs to change. I'm a strong -1 for keeping the status quo (broadly speaking); I want -w to be consistent regarding diagnostics that have been upgraded to errors and that's not the case with the status quo (it silences warnings the implementation upgrades into an error, it does not silence warnings the command line upgrades into an error).

I disagree with the assertion that this option is "inconsistent". I do not believe it is so.

Perhaps it helps to think of the option as having two behaviors:

  1. Counteract all -Werror warning-to-error upgrade requests from the command-line.
  2. Disable non-fatal diagnostic output. (make the build quiet except on failure, in order to avoid build-spam).

This is a useful set of behavior. It is not the only possible useful behavior, and I can see that other behaviors could also make sense, but it's the behavior GCC has long had, and that Clang has also now long had (though much less long than GCC). So I do not think that there is good justification to change that.

In particular, I think it is also useful, and expected, that clang -w foo.c does not accept more programs than clang foo.c.

[example of current diagnostic output being confusing]
In both cases, we give a -W flag at the end of the diagnostic, but we don't add -Werror in the latter case while we do in the former. So there is a very subtle visual distinction for the user to pick up on to help them distinguish between the cases.

Now, this seems a fruitful avenue in which to investigate improvements.

I do agree that it is weird and confusing that default-error diagnostics simply print "-Wfoo" at the end, with no indication as to why they're being emitted as an error, despite "appearing" to be just a warning with no -Werror anywhere to be seen. Potentially we could print something like "[default-error, -Wfoo]" instead, to help address this, or something along those lines.

@carlosgalvezp
Copy link
Contributor

Clang does not (and will never) support -fpermissive

@AaronBallman Could you elaborate on that?

The problem I encounter is that I need to pass different flags to different compilers to suppress a particular warning, for example:

  • GCC: -Wno-incompatible-pointer-types
  • Clang: Wno-incompatible-function-pointer-types

GCC complains when passing an unknown flag (I suppose I could file a bug to them) and the build errors out, which is misleading to the user.

-fpermissive could solve that problem. Otherwise if both compilers used the same name it would of course be great.

@erichkeane
Copy link
Collaborator

Clang does not (and will never) support -fpermissive

@AaronBallman Could you elaborate on that?

The problem I encounter is that I need to pass different flags to different compilers to suppress a particular warning, for example:

* GCC: `-Wno-incompatible-pointer-types`

* Clang: `Wno-incompatible-function-pointer-types`

GCC complains when passing an unknown flag (I suppose I could file a bug to them) and the build errors out, which is misleading to the user.

-fpermissive could solve that problem. Otherwise if both compilers used the same name it would of course be great.

-fpermissive has a TON of language-breaking 'features' to it that is more than just altering warnings, it changes a bunch of errors to implicit conversions/etc. Implementing it in any reasonable fashion would be extensive effort that would result in a worse language dialect that we've been discouraging folks from using for a very long time.

As far as the different warning flags: we attempt to keep our names the same when they do the same thing. In your example, note the clang group is ONLY for function pointer types (not all pointer types!), so are not the same thing. Typically this is something you solve in your cmake script.

That said, we're definitely open to patches that name the same operation/checks the same thing. We also have the ability to put groups into other groups, so if we find 2 of our warning-groups that collectively do the same thing as 1 of GCCs, we can put them in the same group.

That said, this ends up being a bit of a 'patches welcome'. Flags get developed in parallel to the point that we don't necessarily see what is happening on the GCC side when choosing a name (or vice versa).

@carlosgalvezp
Copy link
Contributor

Thanks for the detailed explanation!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
clang:diagnostics New/improved warning or error message in Clang, but not in clang-tidy or static analyzer
Projects
None yet
Development

No branches or pull requests

8 participants