testXXArgumentTesting target false positive

[sophia-guo]

Number of FAILED tests: 6 out of 9. However testXXArgumentTesting is set as passed. It's false positive
https://ci.adoptium.net/view/Test_functional/job/Test_openjdk11_hs_sanity.functional_x86-64_linux_testList_0/6/

22:18:33  TESTING:
22:18:33  *** Starting test suite: XXArgumentTesting.xml ***
22:18:33  Testing: Ensure -XXnosuballoc32bitmem is recognized on all platforms
22:18:33  Test start time: 2024/07/16 21:18:33 Central Standard Time
22:18:33  Running command: "/home/jenkins/workspace/Test_openjdk11_hs_sanity.functional_x86-64_linux_testList_0/jdkbinary/j2sdk-image/jdk-11.0.24+8/bin/..//bin/java"  -Xcompressedrefs -XXnosuballoc32bitmem -version
22:18:33  Time spent starting: 52 milliseconds
22:18:33  Time spent executing: 77 milliseconds
22:18:33  Test result: FAILED
22:18:33  Output from test:
22:18:33   [ERR] Unrecognized option: -Xcompressedrefs
22:18:33   [ERR] Error: Could not create the Java Virtual Machine.
22:18:33   [ERR] Error: A fatal exception has occurred. Program will exit.
22:18:33  >> Success condition was not found: [Output match: (java|openjdk|semeru) version]
22:18:33  >> Failure condition was not found: [Output match: Command-line option unrecognised]
22:18:33  >> Failure condition was not found: [Output match: Unhandled Exception]
22:18:33  >> Failure condition was not found: [Output match: Exception:]
22:18:33  >> Failure condition was not found: [Output match: Processing dump event]
22:18:33  
22:18:33  Executing command: rm -f testjavacore.txt
22:18:33  
22:18:33  Testing: Generate a javacore without -XXnosuballoc32bitmem
22:18:33  Test start time: 2024/07/16 21:18:33 Central Standard Time
22:18:33  Running command: "/home/jenkins/workspace/Test_openjdk11_hs_sanity.functional_x86-64_linux_testList_0/jdkbinary/j2sdk-image/jdk-11.0.24+8/bin/..//bin/java"  -Xcompressedrefs -Xdump:java:events=vmstop,file=testjavacore.txt -version
22:18:33  Time spent starting: 3 milliseconds
22:18:33  Time spent executing: 23 milliseconds
22:18:33  Test result: FAILED
22:18:33  Output from test:
22:18:33   [ERR] Unrecognized option: -Xcompressedrefs
22:18:33   [ERR] Error: Could not create the Java Virtual Machine.
22:18:33   [ERR] Error: A fatal exception has occurred. Program will exit.
22:18:33  >> Success condition was not found: [Output match: (java|openjdk|semeru) version]
22:18:33  >> Failure condition was not found: [Output match: Command-line option unrecognised]
22:18:33  >> Failure condition was not found: [Output match: Unhandled Exception]
22:18:33  >> Failure condition was not found: [Output match: Exception:]
22:18:33  
22:18:33  Testing: Verify that the sub allocator memory is listed in the javacore
22:18:33  Test start time: 2024/07/16 21:18:33 Central Standard Time
22:18:33  Running command: cat testjavacore.txt
22:18:33  Time spent starting: 2 milliseconds
22:18:33  Time spent executing: 7 milliseconds
22:18:33  Test result: FAILED
22:18:33  Output from test:
22:18:33   [ERR] cat: testjavacore.txt: No such file or directory
22:18:33  >> Success condition was not found: [Output match: 32bit allocation regions]
22:18:33  
22:18:33  Executing command: rm -f testjavacore.txt
22:18:33  
22:18:33  Testing: Generate a javacore with -XXnosuballoc32bitmem
22:18:33  Test start time: 2024/07/16 21:18:33 Central Standard Time
22:18:33  Running command: "/home/jenkins/workspace/Test_openjdk11_hs_sanity.functional_x86-64_linux_testList_0/jdkbinary/j2sdk-image/jdk-11.0.24+8/bin/..//bin/java"  -Xcompressedrefs -XXnosuballoc32bitmem -Xdump:java:events=vmstop,file=testjavacore.txt -version
22:18:33  Time spent starting: 3 milliseconds
22:18:33  Time spent executing: 22 milliseconds
22:18:33  Test result: FAILED
22:18:33  Output from test:
22:18:33   [ERR] Unrecognized option: -Xcompressedrefs
22:18:33   [ERR] Error: Could not create the Java Virtual Machine.
22:18:33   [ERR] Error: A fatal exception has occurred. Program will exit.
22:18:33  >> Success condition was not found: [Output match: (java|openjdk|semeru) version]
22:18:33  >> Failure condition was not found: [Output match: Command-line option unrecognised]
22:18:33  >> Failure condition was not found: [Output match: Unhandled Exception]
22:18:33  >> Failure condition was not found: [Output match: Exception:]
22:18:33  
22:18:33  Testing: Verify that the sub allocator memory is not listed in the javacore on z/OS. All other platforms should use the sub-allocator when -XXnosuballoc32bitmem is used.
22:18:33  Test start time: 2024/07/16 21:18:33 Central Standard Time
22:18:33  Running command: cat testjavacore.txt
22:18:33  Time spent starting: 5 milliseconds
22:18:33  Time spent executing: 10 milliseconds
22:18:33  Test result: FAILED
22:18:33  Output from test:
22:18:33   [ERR] cat: testjavacore.txt: No such file or directory
22:18:33  >> Success condition was not found: [Output match: Javacore filename]
22:18:33  
22:18:33  Testing: Verify -XX:MaxHeapSize -XX:InitialHeapSize -XX:ThreadStackSize
22:18:33  Test start time: 2024/07/16 21:18:33 Central Standard Time
22:18:33  Running command: "/home/jenkins/workspace/Test_openjdk11_hs_sanity.functional_x86-64_linux_testList_0/jdkbinary/j2sdk-image/jdk-11.0.24+8/bin/..//bin/java"  -XX:MaxHeapSize=234m -XX:InitialHeapSize=11m -verbose:sizes -version
22:18:33  Time spent starting: 4 milliseconds
22:18:33  Time spent executing: 139 milliseconds
22:18:33  Test result: FAILED
22:18:33  Output from test:
22:18:33   [ERR] openjdk version "11.0.24" 2024-07-16
22:18:33   [ERR] OpenJDK Runtime Environment Temurin-11.0.24+8 (build 11.0.24+8)
22:18:33   [ERR] OpenJDK 64-Bit Server VM Temurin-11.0.24+8 (build 11.0.24+8, mixed mode)
22:18:33  >> Success condition was not found: [Output match: OpenJ9]
22:18:33  >> Required condition was not found: [Output match: -Xmx234M]
22:18:33  >> Required condition was not found: [Output match: -Xms11M]
22:18:33  
22:18:33  Testing: Verify -XX:+PrintFlagsFinal
22:18:33  Test start time: 2024/07/16 21:18:33 Central Standard Time
22:18:33  Running command: "/home/jenkins/workspace/Test_openjdk11_hs_sanity.functional_x86-64_linux_testList_0/jdkbinary/j2sdk-image/jdk-11.0.24+8/bin/..//bin/java"  -XX:+PrintFlagsFinal -version
22:18:33  Time spent starting: 6 milliseconds
22:18:34  Time spent executing: 200 milliseconds
22:18:34  Test result: PASSED
22:18:34  
22:18:34  Testing: Verify rightmost -XX:+PrintFlagsFinal overrides earlier -XX:-PrintFlagsFinal
22:18:34  Test start time: 2024/07/16 21:18:33 Central Standard Time
22:18:34  Running command: "/home/jenkins/workspace/Test_openjdk11_hs_sanity.functional_x86-64_linux_testList_0/jdkbinary/j2sdk-image/jdk-11.0.24+8/bin/..//bin/java"  -XX:-PrintFlagsFinal -XX:+PrintFlagsFinal -version
22:18:34  Time spent starting: 5 milliseconds
22:18:34  Time spent executing: 187 milliseconds
22:18:34  Test result: PASSED
22:18:34  
22:18:34  Testing: Verify rightmost -XX:-PrintFlagsFinal overrides earlier -XX:+PrintFlagsFinal
22:18:34  Test start time: 2024/07/16 21:18:34 Central Standard Time
22:18:34  Running command: "/home/jenkins/workspace/Test_openjdk11_hs_sanity.functional_x86-64_linux_testList_0/jdkbinary/j2sdk-image/jdk-11.0.24+8/bin/..//bin/java"  -XX:+PrintFlagsFinal -XX:-PrintFlagsFinal -version
22:18:34  Time spent starting: 3 milliseconds
22:18:34  Time spent executing: 161 milliseconds
22:18:34  Test result: PASSED
22:18:34  
22:18:34  
22:18:34  ---TEST RESULTS---
22:18:34  Number of PASSED tests: 3 out of 9
22:18:34  Number of FAILED tests: 6 out of 9
22:18:34  
22:18:34  ---SUMMARY OF FAILED TESTS---
22:18:34  Ensure -XXnosuballoc32bitmem is recognized on all platforms
22:18:34  Generate a javacore without -XXnosuballoc32bitmem
22:18:34  Verify that the sub allocator memory is listed in the javacore
22:18:34  Generate a javacore with -XXnosuballoc32bitmem
22:18:34  Verify that the sub allocator memory is not listed in the javacore on z/OS. All other platforms should use the sub-allocator when -XXnosuballoc32bitmem is used.
22:18:34  Verify -XX:MaxHeapSize -XX:InitialHeapSize -XX:ThreadStackSize
22:18:34  -----------------------------
22:18:34  
22:18:34  -----------------------------------
22:18:34  testXXArgumentTesting_0_PASSED

Looks like those failing tests should be limited to impl=openj9 or IBM and move from XXArgumentTesting.xml to XXArgumentTesting_j9.xml

False positive appears to be caused by missing option -nonZeroExitWhenError.

[pshipton]

@llxia fyi

[llxia]

@LongyuZhang could you take a look? Could not create the Java Virtual Machine should be a failure condition.
Update, failed subtests apply to openj9 only. And -nonZeroExitWhenError should be added.