Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix java 1.8 detection #28

Open
severach opened this issue Apr 12, 2023 · 7 comments
Open

Fix java 1.8 detection #28

severach opened this issue Apr 12, 2023 · 7 comments

Comments

@severach
Copy link

Java 1.8 jre8 jdk8 and apps bundled with jre8 constantly report as needing an update due to missing libjvm.so and others. They are missing library dependencies by design which are resolved by the running jvm. Java 11 and 17 are missing many of the same files but do not show in the listing.

Example apps are: cyberduck, mirth-connect-administrator-launcher, jre8, jdk8.

@maximbaz
Copy link
Owner

Hello, I tried on one of the example apps, but it doesn't seem like I am able to reproduce the issue:

image

@severach
Copy link
Author

I can't find any java exclusions in your code so in my opinion we should see it for all javas.

$ pacman -Qs jre
local/jre17-openjdk 17.0.6.u10-2
    OpenJDK Java 17 full runtime environment
local/jre17-openjdk-headless 17.0.6.u10-2
    OpenJDK Java 17 headless runtime environment
local/jre8 8u351-1
    Oracle Java 8 Runtime Environment LTS

$ checkrebuild -v
foreign	jre8

ldd /usr/lib/jvm/java-8-jre/jre/lib/amd64/libverify.so

	libjvm.so => not found

ldd /usr/lib/jvm/java-8-jre/jre/lib/amd64/libzip.so

	libjvm.so => not found
	libjvm.so => not found
	libjvm.so => not found

ldd /usr/lib/jvm/java-8-jre/jre/lib/amd64/libunpack.so

	libjvm.so => not found
	libjvm.so => not found
	libjvm.so => not found

ldd /usr/lib/jvm/java-8-jre/jre/lib/amd64/libt2k.so

	libjvm.so => not found
	libjvm.so => not found
	libjvm.so => not found
	libjvm.so => not found
	libjvm.so => not found

ldd /usr/lib/jvm/java-8-jre/jre/lib/amd64/libresource.so

	libjvm.so => not found
	libjvm.so => not found
	libjvm.so => not found

ldd /usr/lib/jvm/java-8-jre/jre/lib/amd64/libsctp.so

	libjvm.so => not found
	libjvm.so => not found
	libjvm.so => not found
	libjvm.so => not found

ldd /usr/lib/jvm/java-8-jre/jre/lib/amd64/libnio.so

	libjvm.so => not found
	libjvm.so => not found
	libjvm.so => not found

ldd /usr/lib/jvm/java-8-jre/jre/lib/amd64/libnet.so

	libjvm.so => not found
	libjvm.so => not found
	libjvm.so => not found

ldd /usr/lib/jvm/java-8-jre/jre/lib/amd64/libmanagement.so

	libjvm.so => not found
	libjvm.so => not found
	libjvm.so => not found

ldd /usr/lib/jvm/java-8-jre/jre/lib/amd64/libmlib_image.so

	libjvm.so => not found
	libjvm.so => not found
	libjvm.so => not found

ldd /usr/lib/jvm/java-8-jre/jre/lib/amd64/liblcms.so

	libjvm.so => not found
	libjvm.so => not found
	libjvm.so => not found
	libjvm.so => not found

ldd /usr/lib/jvm/java-8-jre/jre/lib/amd64/libjsound.so

	libjvm.so => not found
	libjvm.so => not found
	libjvm.so => not found

ldd /usr/lib/jvm/java-8-jre/jre/lib/amd64/libjsoundalsa.so

	libjvm.so => not found
	libjvm.so => not found
	libjvm.so => not found

ldd /usr/lib/jvm/java-8-jre/jre/lib/amd64/libjfxwebkit.so

	libjvm.so => not found

ldd /usr/lib/jvm/java-8-jre/jre/lib/amd64/libjpeg.so

	libjvm.so => not found
	libjvm.so => not found
	libjvm.so => not found

ldd /usr/lib/jvm/java-8-jre/jre/lib/amd64/libjawt.so

	libjvm.so => not found
	libjvm.so => not found
	libjvm.so => not found
	libjvm.so => not found
	libjvm.so => not found

ldd /usr/lib/jvm/java-8-jre/jre/lib/amd64/libjava.so

	libjvm.so => not found
	libjvm.so => not found

ldd /usr/lib/jvm/java-8-jre/jre/lib/amd64/libfontmanager.so

	libjvm.so => not found
	libjvm.so => not found
	libjvm.so => not found
	libjvm.so => not found

ldd /usr/lib/jvm/java-8-jre/jre/lib/amd64/libdcpr.so

	libjvm.so => not found
	libjvm.so => not found
	libjvm.so => not found

ldd /usr/lib/jvm/java-8-jre/jre/lib/amd64/libawt.so

	libjvm.so => not found
	libjvm.so => not found
	libjvm.so => not found

ldd /usr/lib/jvm/java-8-jre/jre/lib/amd64/libawt_xawt.so

	libjvm.so => not found
	libjvm.so => not found
	libjvm.so => not found
	libjvm.so => not found

ldd /usr/lib/jvm/java-8-jre/jre/lib/amd64/libawt_headless.so

	libjvm.so => not found
	libjvm.so => not found
	libjvm.so => not found
	libjvm.so => not found

ldd /usr/lib/jvm/java-8-jre/jre/lib/amd64/libavplugin-ffmpeg-56.so

	libavcodec-ffmpeg.so.56 => not found
	libavformat-ffmpeg.so.56 => not found

ldd /usr/lib/jvm/java-8-jre/jre/lib/amd64/libavplugin-ffmpeg-57.so

	libavcodec.so.57 => not found
	libavformat.so.57 => not found

ldd /usr/lib/jvm/java-8-jre/jre/lib/amd64/libavplugin-57.so

	libavcodec.so.57 => not found
	libavformat.so.57 => not found

ldd /usr/lib/jvm/java-8-jre/jre/lib/amd64/libavplugin-55.so

	libavcodec.so.55 => not found
	libavformat.so.55 => not found

ldd /usr/lib/jvm/java-8-jre/jre/lib/amd64/libavplugin-56.so

	libavcodec.so.56 => not found
	libavformat.so.56 => not found

ldd /usr/lib/jvm/java-8-jre/jre/lib/amd64/libavplugin-54.so

	libavcodec.so.54 => not found
	libavformat.so.54 => not found

ldd /usr/lib/jvm/java-8-jre/jre/lib/amd64/libavplugin-53.so

	libavcodec.so.53 => not found
	libavformat.so.53 => not found

$ ldd /usr/lib/jvm/java-17-openjdk/lib/libawt.so 
	linux-vdso.so.1 (0x00007ffd0c98e000)
	libjava.so => /usr/lib/jvm/java-17-openjdk/lib/libjava.so (0x00007f587a77a000)
	libm.so.6 => /usr/lib/libm.so.6 (0x00007f587a664000)
	libc.so.6 => /usr/lib/libc.so.6 (0x00007f587a47d000)
	/usr/lib64/ld-linux-x86-64.so.2 (0x00007f587a897000)
	libjvm.so => not found

$ ldd /usr/lib/jvm/java-8-jre/jre/lib/amd64/libavplugin-57.so 
	linux-vdso.so.1 (0x00007ffee636e000)
	libgstreamer-lite.so => not found
	libglib-lite.so => not found
	libavcodec.so.57 => not found
	libavformat.so.57 => not found
	libc.so.6 => /usr/lib/libc.so.6 (0x00007f2d60016000)
	/usr/lib64/ld-linux-x86-64.so.2 (0x00007f2d60238000)

@maximbaz
Copy link
Owner

Hello, is this still an issue, or did you manage to find a workaround? I haven't been able to replicate it, and if the issue was with any java app, I would expect more people to report...

@severach
Copy link
Author

severach commented Aug 26, 2023

$ pacman -Qs jre8
local/jre8 8u371-1
    Oracle Java 8 Runtime Environment LTS
$ pacman -Qs jdk8
local/jdk8 8u371-1
    Oracle Java 8 Development Kit LTS
$ pacman -Qs jre11
local/jre11 11.0.19-1
    Oracle Java 11 Runtime Environment LTS
$ pacman -Qs jre17
local/jre17-openjdk 17.0.8.1.u1-1
    OpenJDK Java 17 full runtime environment
local/jre17-openjdk-headless 17.0.8.1.u1-1
    OpenJDK Java 17 headless runtime environment

$ ldd /usr/lib/jvm/java-8-jre/jre/lib/amd64/libverify.so 
	linux-vdso.so.1 (0x00007ffec1f6c000)
	libjvm.so => not found
	libc.so.6 => /usr/lib/libc.so.6 (0x00007f9668800000)
	/usr/lib64/ld-linux-x86-64.so.2 (0x00007f9668f2d000)
$ ldd /usr/lib/jvm/java-11-jdk/lib/libverify.so 
ldd: warning: you do not have execution permission for `/usr/lib/jvm/java-11-jdk/lib/libverify.so'
	linux-vdso.so.1 (0x00007ffdc8f3f000)
	libjvm.so => not found
	libc.so.6 => /usr/lib/libc.so.6 (0x00007f39e1600000)
	/usr/lib64/ld-linux-x86-64.so.2 (0x00007f39e1db3000)
$ ldd /usr/lib/jvm/java-17-openjdk/lib/libverify.so 
	linux-vdso.so.1 (0x00007fffbd1c0000)
	libjvm.so => not found
	libc.so.6 => /usr/lib/libc.so.6 (0x00007f3066000000)
	/usr/lib64/ld-linux-x86-64.so.2 (0x00007f306642f000)

$ checkrebuild
foreign	jdk8
foreign	jre8

Yes it's still a problem. All Java versions have missing libs. There are a number of bugs depending on expected behavior.

jre11 is not reported. The executable bit is clear on *.so. checkrebuild seems to skip these. Is not reporting *.so without executable bit intentional? Not showing a needed rebuild because it's 0644 doesn't seem safe to me. I have 75 .so files without executable that wouldn't be checked.

$ find /usr/lib -type f -name '*.so' ! -executable | grep -Fv $'python\njvm' | wc -l
find: ‘/usr/lib/mysql/plugin/auth_pam_tool_dir’: Permission denied
75

jre8 jdk8 is reported. If not scanning non executable *.so is expected behavior, this is a bug I can fix by clearing the executable bit on *.so. Your package should have executable *.so. Not reporting on your system is a bug.

jre17 is not reported. The executable bit is set. Unless specifically excluded, not reporting is a bug.

@maximbaz
Copy link
Owner

Looks like checking only executable files was intentional indeed, we need a good heuristic to call "ldd" only on relevant files, I'm not sure if we should include all files with so extension 🤔

filter_executable |

@severach
Copy link
Author

jre17-openjdk is not reported because by default checkrebuild only checks foreign packages. It is reported with checkrebuild -i extra.

Clearing the execute bit is the solution.

Here's a patch for the executable filter. It's faster than forking stat grep file grep for every file and language independent. grep is removed getting one step closer to using all \0 terminated file lists.

diff -ur a/checkrebuild b/checkrebuild
--- a/checkrebuild	2023-08-29 20:17:06.108897659 -0400
+++ b/checkrebuild	2023-08-29 20:22:42.462668375 -0400
@@ -54,10 +54,70 @@
 }
 
 filter_executable() {
-    LANG=C xargs -r0 stat --printf "%F %a\t%n\0" |
-    grep -ozP "^regular file \d?[1357]\d\d\t\K.*" |
-    xargs -r0 file -N |
-    grep -oP ".*(?=: ELF )"
+    local pl='#!/usr/bin/perl
+
+use strict;
+use warnings;
+use integer;
+use bytes;
+no utf8;
+no overloading;
+no locale;
+
+use Data::Dumper;
+use File::stat;
+use Fcntl ":mode";
+
+$/="\0";
+my $test=0;
+my $ELF=chr(127)."ELF";
+
+#print Dumper(\$ELF); exit(1);
+
+sub _iself {
+  my $fname=shift();
+  my $rv=0;
+  my $fh;
+  if (open($fh,"<",$fname)) {
+    binmode($fh);
+    my $text;
+    if (read($fh,$text,4)) {
+      #print Dumper(\$text);
+      if ($text eq $ELF) {
+        $rv=1;
+      }
+    }
+    close($fh);
+  }
+  return($rv);
+}
+
+my $line;
+my $st;
+foreach $line (<STDIN>) {
+  chomp($line);
+  $st=stat($line);
+  if ($st) {
+    if ($test) {
+      if (S_ISDIR($st->mode)) {
+        printf("DIR 0%03o %s\n",$st->mode&0777,$line);
+      } elsif (not ($st->mode & 0111)) {
+        printf("NEX 0%03o %s\n",$st->mode&0777,$line);
+      } elsif (not _iself($line)) {
+        printf("NEL 0%03o %s\n",$st->mode&0777,$line);
+      } else {
+        printf("EXE 0%03o %s\n",$st->mode&0777,$line);
+      }
+    } else {
+      if (not S_ISDIR($st->mode) and ($st->mode & 0111) && _iself($line)) {
+        print($line,"\n");
+      }
+    }
+  }
+}
+exit(0);
+'
+    perl -e "${pl}"
 }
 
 check_broken_ldd() {

@maximbaz
Copy link
Owner

Wow thanks, how much of a perf difference do you get with it? The downside is of course the increased complexity of this, as I'm not quite experienced with perl.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants