Why are response files expanded in osx_cc_wrapper?

[ParkMyCar]

I'm trying to build a relatively large Rust project with rules_rust and LTO enabled on macOS, but the linking step fails with:

= note: external/llvm_toolchain/bin/cc_wrapper.sh: line 129: external/llvm_toolchain_llvm/bin/clang: Argument list too long

I enabled logging for rustc and I can see that it spills arguments to a response file, i.e.

INFO rustc_codegen_ssa::back::link falling back to passing arguments to linker via an @-file

But then it looks like osx_cc_wrapper (and FWIW the Linux version too) expands response files which ultimately leads to the "Argument list too long error". With how long Bazel files paths can be, and the way Rust parallelizes compilation with codegen units, the argument list can get long very quickly.

Is it possible to stop expanding response files? Or is there some reason that the wrapper needs to? Thanks in advance!

[keith]

it has to to do this manipulation https://github.com/bazelbuild/bazel/blob/7d2bc79fdae52f1420b0e04904f26154ff1f48bf/tools/cpp/osx_cc_wrapper.sh.tpl#L36-L54

i think the easiest solution to this specific issue is to change things to recreate the response file after all of this is done.

[ParkMyCar]

I see, thanks @keith! Yeah I was thinking for each response file we could collect the now manipulated arguments into a list, truncate the response file they came from, and re-spill them back to the file?

If that sounds good to you I'm happy to put up a PR!

[keith]

yea i think that's fine. fwiw i think a lot of folks stopped worrying about this because in a recent-ish version of macOS (maybe 3 years ago?) they 10x'd this limit, so i don't think many folks hit it anymore

[ParkMyCar]

Sweet, from a quick Google search it seems like clang has had support for response files since at least 2016 (phabricator) so we're probably fine from a support perspective?

I'll get something whipped up today or tomorrow

[keith]

yea i think that should be safe at this point