revisit disabled tests #10917

llxia · 2020-10-16T21:37:18Z

We should revisit disabled tests to see if the issues are resolved. If they are, we need to re-enable the tests. For these ones that do not have any comments, we should check them again.

For example:
https://github.com/eclipse/openj9/blob/ad7c3f09279a54f8d4d253af03da2a2a31f86f8e/test/functional/cmdLineTests/modularityddrtests/playlist.xml#L26
https://github.com/eclipse/openj9/blob/e2c38649abb06128ee30c06691b6dcc9e19db55b/test/functional/VM_Test/playlist.xml#L148
https://github.com/eclipse/openj9/blob/ab7f03ab6faa069307a3ddedf09ac26f79b9702c/test/functional/cmdLineTests/pltest/playlist.xml#L175

llxia · 2023-06-12T13:38:58Z

@KapilPowar could you take a look at this issue? Thanks

KapilPowar · 2023-07-06T12:56:11Z

@pshipton

While testing VM test suite for JDK8 we are seeing test case failures.
https://github.com/eclipse/openj9/blob/e2c38649abb06128ee30c06691b6dcc9e19db55b/test/functional/VM_Test/playlist.xml#L148

Here is the grinder link

Test cases enabled --
recreateClassTest (1/20 failed )
CFdumptest (1/8 failed)

There was 1 failure:
17:13:10  1) testInnerClasses(com.ibm.j9.recreateclass.tests.RecreateClassCompareTest)junit.framework.AssertionFailedError: Incorrect number of mismatches found expected:<1> but was:<5>
17:13:10  	at com.ibm.j9.recreateclass.tests.RecreateClassCompareTest.compareClassFiles(RecreateClassCompareTest.java:63)
17:13:10  	at com.ibm.j9.recreateclass.tests.RecreateClassCompareTest.runCfdumpAndCompareTools(RecreateClassCompareTest.java:74)
17:13:10  	at com.ibm.j9.recreateclass.tests.RecreateClassCompareTest.runTest(RecreateClassCompareTest.java:86)
17:13:10  	at com.ibm.j9.recreateclass.tests.RecreateClassCompareTest.runTest(RecreateClassCompareTest.java:79)
17:13:10  	at com.ibm.j9.recreateclass.tests.RecreateClassCompareTest.testInnerClasses(RecreateClassCompareTest.java:154)
17:13:10  	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
17:13:10  	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
17:13:10  	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
17:13:10  
17:13:10  FAILURES!!!
17:13:10  Tests run: 20,  Failures: 1,  Errors: 0
--------------------------------------------------------------------

CFdumptest (1/8 failed)

There was 1 failure:
17:13:11  1) testInvalidClassFile(com.ibm.j9.cfdump.tests.lineardump.LinearDumpTests)junit.framework.AssertionFailedError: Expected: <null> but was: bad magic number
17:13:11  	at com.ibm.j9.cfdump.tests.lineardump.LinearDumpTests.testInvalidClassFile(LinearDumpTests.java:123)
17:13:11  	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
17:13:11  	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
17:13:11  	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
17:13:11  
17:13:11  FAILURES!!!
17:13:11  Tests run: 8,  Failures: 1,  Errors: 0

Please confirm if it is product issue.

pshipton · 2023-08-04T16:51:34Z

How do I re-run these tests? The grinder link is expired. When I try to run them every test fails with Error: cfdump not found on java.library.path!. I see that cfdump is part of the test image.

https://openj9-jenkins.osuosl.org/view/Test/job/Grinder/2649

KapilPowar · 2023-08-07T09:32:50Z

In playlist.xml file please try to add cfdump in the lib path - using $(ADD_JVM_LIB_DIR_TO_LIBPATH) \ in <command>
for the test cases.

e.g.
<command>$(ADD_JVM_LIB_DIR_TO_LIBPATH) \ $(JAVA_COMMAND) $(JVM_OPTIONS) \

KapilPowar · 2023-08-09T05:32:19Z

Here are latest Grinder Links
aarch64_linux --Java 8
x86-64_linux -- Java 8
x86-64_windows -- Java 8

pshipton · 2023-08-11T17:30:30Z

Failure details from recreateClassTest

13:15:46  Start testInnerClasses
13:15:46  Running command: /home/jenkins/workspace/Grinder/openjdkbinary/openjdk-test-image/openj9/cfdump -t:/home/jenkins/workspace/Grinder/aqa-tests/TKG/output_16917741452730/recreateClassTest_0/InnerClassesTest /home/jenkins/workspace/Grinder/aqa-tests/TKG/output_16917741452730/recreateClassTest_0/InnerClassesTest.class
13:15:46  cfdump: Wrote 1212 bytes to output file /home/jenkins/workspace/Grinder/aqa-tests/TKG/output_16917741452730/recreateClassTest_0/InnerClassesTest.j9class
13:15:46  Inner class access flags differ: 0X8, 0X0
13:15:46  Aug 11, 2023 5:15:46 PM com.ibm.j9.test.tools.classfilecompare.ClassFileCompare reportDifference
13:15:46  WARNING: Inner class access flags differ: 0X8, 0X0
13:15:46  Inner class inner name differ: LocalClass, null
13:15:46  Aug 11, 2023 5:15:46 PM com.ibm.j9.test.tools.classfilecompare.ClassFileCompare reportDifference
13:15:46  WARNING: Inner class inner name differ: LocalClass, null
13:15:46  Inner class access flags differ: 0X8, 0X0
13:15:46  Aug 11, 2023 5:15:46 PM com.ibm.j9.test.tools.classfilecompare.ClassFileCompare reportDifference
13:15:46  WARNING: Inner class access flags differ: 0X8, 0X0
13:15:46  Inner class access flags differ: 0X9, 0X0
13:15:46  Aug 11, 2023 5:15:46 PM com.ibm.j9.test.tools.classfilecompare.ClassFileCompare reportDifference
13:15:46  WARNING: Inner class access flags differ: 0X9, 0X0
13:15:46  Inner class inner name differ: InnerClass, null
13:15:46  Aug 11, 2023 5:15:46 PM com.ibm.j9.test.tools.classfilecompare.ClassFileCompare reportDifference
13:15:46  WARNING: Inner class inner name differ: InnerClass, null
13:15:46  Mismatches spotted: 5

pshipton · 2023-08-11T17:49:58Z

@TobiAjila can someone pls take a look at this recreateClassTest failure in the previous comment, it appears we may need a fix.

My changes to run the tests and add debug are in https://github.com/pshipton/openj9/tree/tests

tajila · 2023-08-11T18:22:34Z

@hangshao0 Can you please take a look at this

Issue eclipse-openj9#10917 Signed-off-by: Peter Shipton <[email protected]>

pshipton · 2023-08-11T20:56:26Z

Re-enabling CFdumptest via #17939

hangshao0 · 2023-08-14T14:50:00Z

I will look at this.

hangshao0 · 2023-08-15T20:20:30Z

I've manually run cfdump on the inner class. Looking at the generated j9class and original class file:

cfdump -d 'InnerClassesTest$InnerClass.class'
Magic: CAFEBABE
Version: 52.0
Constant Pool Size: 17
Access Flags: 0x21 (
  access: public super )
ThisClass: 2 -> InnerClassesTest$InnerClass
SuperClass: 3 -> java/lang/Object
Interfaces (0):

Fields (0):
Methods (1):
  Name: 4 -> <init>
  Signature: 5 -> ()V
  Access Flags: 0x1 (
  access: public )
  Attributes (1):
    Code:
      Max stack: 1
      Max Locals: 1
      Code Length: 5
      Caught Exceptions (0):
      Attributes (1):
        LineNumberTable:
          pc: 0 line number: 2

Attributes (2):
  SourceFile:
    Source File: 9 -> InnerClassesTest.java
  InnerClasses:
    Inner Class: 2 -> InnerClassesTest$InnerClass
      Outer Class: 11 -> InnerClassesTest
      Inner Name: 13 -> InnerClass
      Inner Class Access Flags: 0x9 (
  access: public )

cfdump -d 'InnerClassesTest$InnerClass.j9class'
Magic: CAFEBABE
Version: 52.0
Constant Pool Size: 17
Access Flags: 0x21 (
  access: public super )
ThisClass: 5 -> InnerClassesTest$InnerClass
SuperClass: 2 -> java/lang/Object
Interfaces (0):

Fields (0):
Methods (1):
  Name: 15 -> <init>
  Signature: 3 -> ()V
  Access Flags: 0x1 (
  access: public )
  Attributes (1):
    Code:
      Max stack: 1
      Max Locals: 1
      Code Length: 5
      Caught Exceptions (0):
      Attributes (1):
        LineNumberTable:
          pc: 0 line number: 2

Attributes (2):
  InnerClasses:
    Inner Class: 5 -> InnerClassesTest$InnerClass
      Outer Class: 12 -> InnerClassesTest
      Inner Name: 8 -> InnerClass
      Inner Class Access Flags: 0x9 (
  access: public )
  SourceFile:
    Source File: 13 -> InnerClassesTest.java

I see the access flags and inner class names are the same. The differences might be from the outer class that cause the test to fail.

hangshao0 · 2023-08-16T20:33:02Z

cfdump -d InnerClassesTest.class
Magic: CAFEBABE
Version: 52.0
Constant Pool Size: 24
Access Flags: 0x21 (
  access: public super )
ThisClass: 3 -> InnerClassesTest
SuperClass: 4 -> java/lang/Object
Interfaces (0):

Fields (0):
Methods (2):
  Name: 9 -> <init>
  Signature: 10 -> ()V
  Access Flags: 0x1 (
  access: public )
  Attributes (1):
    Code:
      Max stack: 1
      Max Locals: 1
      Code Length: 5
      Caught Exceptions (0):
      Attributes (1):
        LineNumberTable:
          pc: 0 line number: 1

  Name: 13 -> getLocalClass
  Signature: 14 -> ()Ljava/lang/Class;
  Access Flags: 0x9 (
  access: public static )
  Attributes (2):
    Code:
      Max stack: 1
      Max Locals: 0
      Code Length: 3
      Caught Exceptions (0):
      Attributes (1):
        LineNumberTable:
          pc: 0 line number: 8
    Signature:
      Signature: 16 -> ()Ljava/lang/Class<*>;

Attributes (2):
  SourceFile:
    Source File: 18 -> InnerClassesTest.java
  InnerClasses:
    Inner Class: 5 -> InnerClassesTest$InnerClass
      Outer Class: 3 -> InnerClassesTest
      Inner Name: 6 -> InnerClass
      Inner Class Access Flags: 0x9 (
  access: public )
    Inner Class: 2 -> InnerClassesTest$1LocalClass
      Outer Class: (not a member)
      Inner Name: 8 -> LocalClass
      Inner Class Access Flags: 0x0 (
  access: default )

cfdump -d InnerClassesTest.j9class
Magic: CAFEBABE
Version: 52.0
Constant Pool Size: 22
Access Flags: 0x21 (
  access: public super )
ThisClass: 7 -> InnerClassesTest
SuperClass: 3 -> java/lang/Object
Interfaces (0):

Fields (0):
Methods (2):
  Name: 21 -> <init>
  Signature: 14 -> ()V
  Access Flags: 0x1 (
  access: public )
  Attributes (1):
    Code:
      Max stack: 1
      Max Locals: 1
      Code Length: 5
      Caught Exceptions (0):
      Attributes (1):
        LineNumberTable:
          pc: 0 line number: 1

  Name: 12 -> getLocalClass
  Signature: 13 -> ()Ljava/lang/Class;
  Access Flags: 0x9 (
  access: public static )
  Attributes (2):
    Code:
      Max stack: 1
      Max Locals: 0
      Code Length: 3
      Caught Exceptions (0):
      Attributes (1):
        LineNumberTable:
          pc: 0 line number: 8
    Signature:
      Signature: 4 -> ()Ljava/lang/Class<*>;

Attributes (2):
  InnerClasses:
    Inner Class: 20 -> InnerClassesTest$InnerClass
      Outer Class: 7 -> InnerClassesTest
      Inner Name: (anonymous)                 <------------------------------------- this is different from the original class
      Inner Class Access Flags: 0x0 (         <------------------------------------- this is different from the original class
  access: default )
    Inner Class: 1 -> InnerClassesTest$1LocalClass
      Outer Class: (not a member)
      Inner Name: (anonymous)
      Inner Class Access Flags: 0x0 (
  access: default )
  SourceFile:
    Source File: 5 -> InnerClassesTest.java

The difference is from the inner class attribute of the outer class.
The inner class simple name and access flags are not preserved in the romClass of the outer class:

openj9/runtime/bcutil/ClassFileWriter.cpp

Lines 1054 to 1056 in 949a4df

    
           /* NOTE: innerClassAccessFlags and innerNameIndex are not preserved in the ROM class - technically incorrect, but this should only matter to compilers */ 
        
           writeU16(0); /* innerNameIndex */ 
        
           writeU16(0); /* innerClassAccessFlags */

If we decided to not record these information in the romClass in #13199, the test needs to be updated to accept such differences in .class and .j9class files.

hangshao0 · 2023-08-16T20:33:17Z

@ChengJin01 ^^

llxia · 2023-08-23T20:19:27Z

CFdumptest is only enabled for JDK8. I think it is able to run against all versions.
https://github.com/pshipton/openj9/blob/master/test/functional/VM_Test/playlist.xml#L167
@KapilPowar could you run CFdumptest on JDK11 and JDK17 when you have time? Thanks

pshipton · 2023-08-24T01:14:11Z

I tried other versions but there are problems that would need to be resolved.

KapilPowar · 2023-08-29T11:20:29Z

Grinder test runs for CFdumptest
JDK11 x86-64_linux
JDK11 x86-64_mac
JDK11 x86-64_windows
JDK17 x86-64_linux
JDK17 x86-64_mac
JDK17 x86-64_windows

llxia · 2023-08-29T14:23:14Z

Thanks @KapilPowar . Could also run on other missing platforms in Grinder? Thanks

KapilPowar · 2023-08-30T15:40:12Z

Grinder test runs for CFdumptest
JDK17
s390x_linux --UNSTABLE
aarch64_mac --Success
ppc64_aix --UNSTABLE

JDK11
aarch64_mac -- Success
s390x_linux--UNSTABLE
ppc64_aix --UNSTABLE

KapilPowar · 2023-09-19T11:39:27Z

@llxia @pshipton FYI
Grinder test run for recreateClassTest still results in same error -- JDK8 x86-64_linux

There was 1 failure:
17:01:36  1) testInnerClasses(com.ibm.j9.recreateclass.tests.RecreateClassCompareTest)junit.framework.AssertionFailedError: Incorrect number of mismatches found expected:<1> but was:<5>
17:01:36  	at com.ibm.j9.recreateclass.tests.RecreateClassCompareTest.compareClassFiles(RecreateClassCompareTest.java:63)
17:01:36  	at com.ibm.j9.recreateclass.tests.RecreateClassCompareTest.runCfdumpAndCompareTools(RecreateClassCompareTest.java:74)
17:01:36  	at com.ibm.j9.recreateclass.tests.RecreateClassCompareTest.runTest(RecreateClassCompareTest.java:86)
17:01:36  	at com.ibm.j9.recreateclass.tests.RecreateClassCompareTest.runTest(RecreateClassCompareTest.java:79)
17:01:36  	at com.ibm.j9.recreateclass.tests.RecreateClassCompareTest.testInnerClasses(RecreateClassCompareTest.java:154)
17:01:36  	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
17:01:36  	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
17:01:36  	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
17:01:36  
17:01:36  FAILURES!!!
17:01:36  Tests run: 20,  Failures: 1,  Errors: 0
17:01:36  
17:01:36  -----------------------------------
17:01:36  recreateClassTest_0_FAILED
17:01:36  -----------------------------------

pshipton · 2023-09-19T14:39:18Z

I assume this can be moved forward.

pshipton · 2023-09-19T14:40:46Z

Re #10917 (comment), we're waiting for @ChengJin01 to have time to take a look, based on #10917 (comment)

ChengJin01 · 2023-09-19T14:45:47Z

I agree with @hangshao0 's investigation & conclusions given we don' record the access flags in the romClass by default.

pshipton · 2023-09-19T14:48:08Z

So the next step is to update the test itself to ignore those differences. @TobiAjila

KapilPowar · 2023-09-21T11:56:19Z

@llxia
Grinder test run for cmdLineTester_forceLazySymbolResolution
JDK17 x86-64_linux

Grinder test run for cmdLineTester_jvmtitests_hcr_OSRG_nongold
JDK17 x86-64_linux
JDK17 x86-64_mac
JDK17 x86-64_windows

llxia · 2023-09-21T13:30:53Z

@KapilPowar if the Grinders pass, please create PR to reenable the tests. Thanks

KapilPowar · 2023-09-22T11:26:25Z

@KapilPowar if the Grinders pass, please create PR to reenable the tests. Thanks

PR created #18181

KapilPowar · 2023-09-27T13:21:59Z

@llxia
Grinder test run for TestFlushReflectionCache & jit_jitt_openj9_none_SCC
x86-64_linux
x86-64_mac
x86-64_windows

PR created #18206

KapilPowar · 2023-09-29T05:53:05Z

@llxia
Grinder test run for cmdLineTester_dumpromclasstests
aix_ppc-64 JDK8
aix_ppc-64 JDK11
aix_ppc-64 JDK17

PR created #18215

tajila · 2024-05-14T13:21:11Z

@llxia Do we still need this issue?

llxia · 2024-08-13T13:20:39Z

I believe this issue can be closed.

llxia added the comp:test label Oct 16, 2020

pshipton added this to the Release 0.41 (Java 8, 11, 17) Oct refresh milestone Aug 11, 2023

pshipton added the comp:vm label Aug 11, 2023

pshipton added a commit to pshipton/openj9 that referenced this issue Aug 11, 2023

Update LinearDumpTests.testInvalidClassFile() and enable CFdumptest

55f338f

Issue eclipse-openj9#10917 Signed-off-by: Peter Shipton <[email protected]>

pshipton added a commit to pshipton/openj9 that referenced this issue Aug 11, 2023

Update LinearDumpTests.testInvalidClassFile() and enable CFdumptest

e0a67c7

Issue eclipse-openj9#10917 Signed-off-by: Peter Shipton <[email protected]>

pshipton mentioned this issue Aug 11, 2023

Update LinearDumpTests.testInvalidClassFile() and enable CFdumptest #17939

Merged

pshipton modified the milestones: Release 0.41 (Java 8, 11, 17) Oct refresh, Release 0.43 (Java 8, 11, 17, 21) Jan refresh Sep 19, 2023

KapilPowar mentioned this issue Sep 22, 2023

re-enable cmdLineTester_forceLazySymbolResolution & cmdLineTester_jvmtitests_hcr_OSRG_nongold #18181

Merged

KapilPowar mentioned this issue Sep 27, 2023

re-enable TestFlushReflectionCache & jit_jitt_openj9_none_SCC #18206

Merged

KapilPowar mentioned this issue Sep 29, 2023

re-enable cmdLineTester_dumpromclasstests #18215

Merged

tajila modified the milestones: Release 0.43 (Java 8, 11, 17, 21) Jan refresh, Release 0.44 (Java 8, 11, 17, 21) Apr refresh Nov 13, 2023

tajila modified the milestones: Release 0.44 (Java 8, 11, 17, 21) Apr refresh, Release 0.46 (Java 8, 11, 17, 21, 22) July refresh Mar 15, 2024

pshipton removed the comp:vm label Jun 18, 2024

pshipton modified the milestones: Release 0.46 (Java 8, 11, 17, 21, 22) July refresh, Java 23 (0.47) Aug 12, 2024

llxia closed this as completed Aug 13, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

revisit disabled tests #10917

revisit disabled tests #10917

llxia commented Oct 16, 2020

llxia commented Jun 12, 2023

KapilPowar commented Jul 6, 2023

pshipton commented Aug 4, 2023

KapilPowar commented Aug 7, 2023

KapilPowar commented Aug 9, 2023

pshipton commented Aug 11, 2023

pshipton commented Aug 11, 2023

tajila commented Aug 11, 2023

pshipton commented Aug 11, 2023

hangshao0 commented Aug 14, 2023

hangshao0 commented Aug 15, 2023

hangshao0 commented Aug 16, 2023

hangshao0 commented Aug 16, 2023

llxia commented Aug 23, 2023

pshipton commented Aug 24, 2023

KapilPowar commented Aug 29, 2023

llxia commented Aug 29, 2023

KapilPowar commented Aug 30, 2023

KapilPowar commented Sep 19, 2023

pshipton commented Sep 19, 2023

pshipton commented Sep 19, 2023

ChengJin01 commented Sep 19, 2023

pshipton commented Sep 19, 2023

KapilPowar commented Sep 21, 2023 •

edited

Loading

llxia commented Sep 21, 2023

KapilPowar commented Sep 22, 2023

KapilPowar commented Sep 27, 2023

KapilPowar commented Sep 29, 2023

tajila commented May 14, 2024

llxia commented Aug 13, 2024

revisit disabled tests #10917

revisit disabled tests #10917

Comments

llxia commented Oct 16, 2020

llxia commented Jun 12, 2023

KapilPowar commented Jul 6, 2023

pshipton commented Aug 4, 2023

KapilPowar commented Aug 7, 2023

KapilPowar commented Aug 9, 2023

pshipton commented Aug 11, 2023

pshipton commented Aug 11, 2023

tajila commented Aug 11, 2023

pshipton commented Aug 11, 2023

hangshao0 commented Aug 14, 2023

hangshao0 commented Aug 15, 2023

hangshao0 commented Aug 16, 2023

hangshao0 commented Aug 16, 2023

llxia commented Aug 23, 2023

pshipton commented Aug 24, 2023

KapilPowar commented Aug 29, 2023

llxia commented Aug 29, 2023

KapilPowar commented Aug 30, 2023

KapilPowar commented Sep 19, 2023

pshipton commented Sep 19, 2023

pshipton commented Sep 19, 2023

ChengJin01 commented Sep 19, 2023

pshipton commented Sep 19, 2023

KapilPowar commented Sep 21, 2023 • edited Loading

llxia commented Sep 21, 2023

KapilPowar commented Sep 22, 2023

KapilPowar commented Sep 27, 2023

KapilPowar commented Sep 29, 2023

tajila commented May 14, 2024

llxia commented Aug 13, 2024

KapilPowar commented Sep 21, 2023 •

edited

Loading