Compatible with (but not mandatory) Java 21

[denis-yuen]

Description
Make the CLI build compatible with Java 21.
Does not switch to only Java 21, yet

Some notes:

• A new version of Ubuntu so we get Java 21 with a new machine executor
• New Ubuntu means new Docker daemon version which finally terminates support for older Docker images https://docs.docker.com/engine/deprecated/#pushing-and-pulling-with-image-manifest-v2-schema-1
  • this means I needed to touch and update a bunch of old tutorial images
  • in some cases this means forking into dockstore-testing, in others just download and uploading with a new daemon
  • this does mean that our tutorials may die over time with new Docker versions
• fixing failed tests is a pain after the death of System Stubs and its replacement with System Lambda i.e. we cannot catch System.exit as an exception anymore across the board, instead the whole VM exits with an unexpected exit with little output, interrupting test result upload (something linked to Switch to Junit5, remove powermock dockstore#5284 )
• some images forked and uploaded into quay.io/dockstore-testing, some workflows modified in testing repos such as https://github.com/dockstore-testing/dockstore-tool-md5sum https://github.com/dockstore-testing/mutect-tool https://github.com/dockstore-testing/dockstore-tool-linux-sort but in the end, I mostly downloaded and uploaded Docker images using a compatible Docker daemon rather than fixing old Dockerfiles
• there's a complex set of four mocked tests using powermock (which is dead) and old versions of junit and mockito. While useful, on balance they probably are not critical or worth blocking testing our compatibility with Java 21, so splitting up a follow-up ticket https://ucsc-cgl.atlassian.net/browse/SEAB-6851

Review Instructions
Could do some spot checking, listing and running workflows as in our getting started tutorials

Issue
https://ucsc-cgl.atlassian.net/browse/SEAB-6611
Follow-up with https://ucsc-cgl.atlassian.net/browse/SEAB-6851

Security
If there are any concerns that require extra attention from the security team, highlight them here.

Please make sure that you've checked the following before submitting your pull request. Thanks!

• Check that you pass the basic style checks and unit tests by running ./mvnw clean install
• If this PR is for a user-facing feature, create and link a documentation ticket for this feature (usually in the same milestone as the linked issue). Style points if you create a documentation PR directly and link that instead.

[github-advanced-security]

This pull request sets up GitHub code scanning for this repository. Once the scans have completed and the checks have passed, the analysis results for this pull request branch will appear on this overview. Once you merge this pull request, the 'Security' tab will show more code scanning analysis results (for example, for the default branch). Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results. For more information about GitHub code scanning, check out the documentation.

[sonarcloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 70.27%. Comparing base (5d706b6) to head (e628a28).
Report is 1 commits behind head on develop.

Additional details and impacted files

@@              Coverage Diff              @@
##             develop     #293      +/-   ##
=============================================
- Coverage      70.29%   70.27%   -0.02%     
- Complexity      1060     1061       +1     
=============================================
  Files             48       48              
  Lines           6073     6073              
  Branches         801      801              
=============================================
- Hits            4269     4268       -1     
- Misses          1457     1458       +1     
  Partials         347      347              

Flag	Coverage Δ
bitbuckettests	9.83% <ø> (ø)
confidentialtooltests	55.03% <ø> (-0.65%)	⬇️
confidentialworkflowtests	30.23% <ø> (+0.01%)	⬆️
nonconfidentialtests	32.30% <ø> (-0.09%)	⬇️
singularitytests	16.58% <ø> (ø)
toilintegrationtests	15.21% <ø> (ø)
unittests	8.34% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[denis-yuen]

To write up my findings before the weekend

1. New Ubuntu to get a new Java version
2. New Ubuntu means new Docker (which means many of our oldest Docker images are no longer compatible due to Track and warn on old Docker versions dockstore#5878 )
3. New pipx install method seemed to mean that I needed to install schema salad, cwlref-runner and their paths separately

[denis-yuen]

Some Docker repos are forked, some can be downloaded and reuploaded in place (by simply pulling and pushing with an older client, which I had handy on my laptop)

This is an example of forking, the pull/push updates don't leave a trace.

[denis-yuen]

This matches 1.16 quickstart with pipx, but had to do a bit of extra work to get schema_salad and cwlref-runner (may be using pipx runpip wrong)

[sonarcloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[kathy-t]

Why is the tag different for this and the toil_wes_test_executor below?

[denis-yuen]

A sad reason
While experimenting, I bumped to a newer version to try to get a newer Java version on the machine executor. It turned out to be unnecessary, but since we ran the tests anyway I kept the newer versions that worked (toil_wes_test_executor used for wes-toil-test).

Something about ubuntu-2204:2024.11.1 breaks the toil install silently (but not all of toil, the part used for regular integration tests including integration-tests-toil-integration-tests, not a typo)