Modifications to ensure compatibility with OpenXL and xlC on AIX

[midronij]

Modify compiler flags, macros, and linked libraries to ensure that OpenJ9 can be built with both OpenXL and xlC on AIX.

Updated version of #20045

[pshipton]

The OMR PR eclipse-omr/omr#7561

[midronij]

@zl-wang here is the new OpenXL PR for OpenJ9. Marked as a WIP since I'm still addressing the remaining review comments from Ishita's PR

[midronij]

I've just finished addressing the review comments from the previous version of this PR. @pshipton @zl-wang when you have a moment could you take a look to ensure that I didn't miss anything/provide any further feedback?

[pshipton]

It's @keithc-ca who should take a look, probably most of the comments were from him.

[pshipton]

I didn't see anything obvious. We'll need to check it doesn't break xlc, both on jenkins and vmfarm, if that hasn't been done yet.

[midronij]

I've confirmed that these changes don't break the xlC build on either jenkins or vmfarm. However, I did run into some issues when trying to do an OpenXL build on jenkins. Namely, when building DDR this error message popped up:

12:17:56  Could not load program /home/jenkins/workspace/Build_JDK23_ppc64_aix_Personal/build/aix-ppc64-server-release/vm/runtime/omr_ddrgen:
12:17:56  	Dependent module /usr/lib/libc++.a(shr2_64.o) could not be loaded.
12:17:56  	Member shr2_64.o is not found in archive 
12:17:56  gmake[10]: *** [CMakeFiles/j9ddr.dir/build.make:94: /home/jenkins/workspace/Build_JDK23_ppc64_aix_Personal/build/aix-ppc64-server-release/vm/runtime/j9ddr.dat] Error 255

(source: https://hyc-runtimes-jenkins.swg-devops.com/job/Build_JDK23_ppc64_aix_Personal/61/console)

I'm able to bypass this temporarily by adding --disable-ddr to the bash configure command, but the build still fails with this error:

13:37:14  ld: 0711-317 ERROR: Undefined symbol: .inflateInit2_
13:37:14  ld: 0711-317 ERROR: Undefined symbol: .inflate
13:37:14  ld: 0711-317 ERROR: Undefined symbol: .inflateEnd
13:37:14  ld: 0711-345 Use the -bloadmap or -bnoquiet option to obtain more information.
13:37:14  .ibm-clang: error: linker command failed with exit code 8 (use -v to see invocation)

(source: https://hyc-runtimes-jenkins.swg-devops.com/job/Build_JDK23_ppc64_aix_Personal/62/console)

I've run into errors like this when building locally, and was able to deal with them by adding additional libraries to the linker commands, but as far as I can tell, the makefile that I would need to modify to deal with this one in particular isn't part of OpenJ9 or OMR:

13:37:14  gmake[3]: *** [test/JtregNativeJdk.gmk:143: /home/jenkins/workspace/Build_JDK23_ppc64_aix_Personal/build/aix-ppc64-server-release/support/test/jdk/jtreg/native/bin/JliLaunchTest] Error 8

[zl-wang]

@pshipton for shr2_64.o issue, a new level of runtime needs to be installed.
@midronij for libz.so problem, there is a build option to bypass it;

[pshipton]

for shr2_64.o issue, a new level of runtime needs to be installed.

I believe we have 17.1.1.2 as per https://github.ibm.com/runtimes/infrastructure/issues/7704, but current issue is https://github.ibm.com/runtimes/infrastructure/issues/9368
The latest is 17.1.1.8 as per https://www.ibm.com/support/pages/fix-list-xl-cc-runtime-aix#171X
Whatever we install will be the minimum level required for customers. What version would you like?

[zl-wang]

i have these on my machine:

  libc++.adt.include        17.1.1.4  COMMITTED  C Set ++ Application
  libc++.rte                17.1.1.4  COMMITTED  IBM XL C++ Runtime for AIX 7.1
  libc++abi.rte             17.1.1.4  COMMITTED  IBM XL C++ Runtime for AIX 7.2

and, shr2_64.o is indeed in libc++.a:

-bash-5.0$ ar -Xany -t libc++.a
libc++.so.1
libc++.so.1
version.cpp.o
cxxabi.o
cxxabi_64.o
shr.o
shr_64.o
shr2.o
shr2_64.o
cxxabi_compat.imp

it might be due to your OpenXL ND (non default) installation, such that libraries were not installed under the usual directory (/usr/lpp/xlC/lib). So that, a separate build-time/link-time -L argument is needed to point to where they were installed (i would assume somewhere under OpenXL compiler installation itself).

[pshipton]

On the build machine

ar -Xany -t /opt/IBM/openxlC/usr/lib/libc++.a
libc++.so.1
libc++.so.1
version.cpp.o
cxxabi.o
cxxabi_64.o
shr.o
shr_64.o
shr2.o
shr2_64.o
cxxabi_compat.imp

[midronij]

@midronij for libz.so problem, there is a build option to bypass it;

I tried building again with --with-zlib=bundled added to the bash configure command, and it seems like that got me past the zlib issue, but the build is still failing with a bunch of errors similar to what I was getting before with DDR enabled:

14:58:03  JVMJ9VM011W Unable to load j9jit29: Could not load module /home/jenkins/workspace/Build_JDK23_ppc64_aix_Personal/build/aix-ppc64-server-release/jdk/lib/default/libj9jit29.so.
14:58:03  	Dependent module /usr/lib/libc++.a(shr2_64.o) could not be loaded.
14:58:03  	Member shr2_64.o is not found in archive 
14:58:03  Could not load module /home/jenkins/workspace/Build_JDK23_ppc64_aix_Personal/build/aix-ppc64-server-release/jdk/lib/default/libj9jit29.so.
14:58:03  	Dependent module /home/jenkins/workspace/Build_JDK23_ppc64_aix_Personal/build/aix-ppc64-server-release/jdk/lib/default/li
14:58:41  Creating java.compiler.jmod
14:58:41  JVMJ9VM011W Unable to load j9jit29: Could not load module /home/jenkins/workspace/Build_JDK23_ppc64_aix_Personal/build/aix-ppc64-server-release/jdk/lib/default/libj9jit29.so.
14:58:41  	Dependent module /usr/lib/libc++.a(shr2_64.o) could not be loaded.
14:58:41  	Member shr2_64.o is not found in archive 
14:58:41  Could not load module /home/jenkins/workspace/Build_JDK23_ppc64_aix_Personal/build/aix-ppc64-server-release/jdk/lib/default/libj9jit29.so.
14:58:41  	Dependent module /home/jenkins/workspace/Build_JDK23_ppc64_aix_Personal/build/aix-ppc64-server-release/jdk/lib/default/li
.
.
.

(source: https://hyc-runtimes-jenkins.swg-devops.com/job/Build_JDK23_ppc64_aix_Personal/63/console)

[zl-wang]

you need to add one config option to indicate to the linker where to look up the runtime libraries (instead of the default location on AIX machine ... i.e. /usr/lib or /usr/lib64). peter already got you that location: /opt/IBM/openxlC/usr/lib
Certain option which eventually turned into -L option of the compiler/linker, that is.

[dsouzai]

I have a few questions since some of these changes impact platforms that are not AIX+OPENXL.

[dsouzai]

LGTM; thanks for making the changes. Could you squash the commits?

[midronij]

Could you squash the commits?

Done! Please feel free to start the CI tests when you have a moment.

[dsouzai]

Jenkins test sanity.functional all jdk8,jdk21 depends eclipse-omr/omr#7561

[jdmpapin]

Windows builds failed due to a lack of machines

[jdmpapin]

Since the PR builds haven't been able to build on Windows or z/OS, have these changes been built successfully on those platforms? I don't particularly expect them to be affected, but we've had a number of build breaks recently

[midronij]

I haven't actually tested those, since these changes are intended for AIX only, but I can do a build just to be sure

[midronij]

@jdmpapin I confirmed that the builds work on Windows and z/OS! I think we should be good to merge this PR and the OMR side one (eclipse-omr/omr#7561)