CI: re-organize GitHub Actions

[liushuyu]

This pull request re-organizes GitHub Actions, where Linux and macOS tasks are now split.
This is needed because the Linux build environment uses Docker containers to minimize the impact of GitHub Actions adding/removing PPAs from their host VM systems.

In addition, Linux build containers switch to LLVM's upstream APT archive instead of downloading the binary files from GitHub Release.

A LLVM 18.1 build job is also added to the build matrix to support #4599.

[kinke]

This is needed because the Linux build environment uses Docker containers to minimize the impact of GitHub Actions adding/removing PPAs from their host VM systems.

Which is a problem because of the LLVM apt repo now? - We've used that apt repo in the past, years ago, and it wasn't reliable IIRC - downtimes, as well as conflicts with the CI runner image from time to time [with whatever CI system we had back then, way before GitHub Actions came around].

The availability of binary LLVM packages from GitHub directly has always been a bit problematic, especially since we cannot use Ubuntu 22 due to debuginfo/gdb regressions (#4389).

[liushuyu]

Which is a problem because of the LLVM apt repo now? - We've used that apt repo in the past, years ago, and it wasn't reliable IIRC - downtimes, as well as conflicts with the CI runner image from time to time [with whatever CI system we had back then, way before GitHub Actions came around].

We can instead make a prebuilt Docker container image so that apt.llvm.org availability problem will be avoided. Since it's inside a container, it won't be influenced by the host system and can be easily migrated if another CI system is used in the future.

The availability of binary LLVM packages from GitHub directly has always been a bit problematic, especially since we cannot use Ubuntu 22 due to debuginfo/gdb regressions (#4389).

Has the issue been forwarded to the GDB upstream? If yes, was this issue fixed in a newer GDB release?

[kinke]

We can instead make a prebuilt Docker container image so that apt.llvm.org availability problem will be avoided.

I think that'd be a bit too much effort. :) - What we have usually done so far is wait a bit for the new LLVM major to mature; then almost every time around there was at least one usable GitHub artifact among the various patch releases. For the recent LLVM versions without Ubuntu 20 packages, we've used macOS arm64 ones as you've seen. I expect one to pop up soon for v18.1.x too.

On rare occasions, we've been temporarily resorting to our LLVM fork and its artifacts. Those are normally only used for the main.yml GHA workflow and the LDC CI/release artifacts. The first prebuilt packages based on v18.1.* are available here (optionally with enabled assertions): https://github.com/ldc-developers/llvm-project/releases/tag/CI

The LLVM apt repo dependency aside, I'm not against a container-based approach in principle, but I definitely don't like the current duplication by splitting up the existing job into 2. There's probably a way to set something like container: ${{ null }} for the Mac jobs, to keep a single job definition. Possibly of interest: https://stackoverflow.com/questions/76925314/conditional-use-of-container-depending-on-the-group-it-runs-on-for-github-action

Has the issue been forwarded to the GDB upstream? If yes, was this issue fixed in a newer GDB release?

Not that I know of, mainly because it's not clear whether it isn't our fault. The same tests seem to pass with the DMD compiler on Ubuntu 22, using its default gdb version.

[liushuyu]

I think that'd be a bit too much effort. :) - What we have usually done so far is wait a bit for the new LLVM major to mature; then almost every time around there was at least one usable GitHub artifact among the various patch releases. For the recent LLVM versions without Ubuntu 20 packages, we've used macOS arm64 ones as you've seen. I expect one to pop up soon for v18.1.x too.

I have already prepared a possible solution for making container images: https://github.com/liushuyu/ldc-build-environments.

The LLVM apt repo dependency aside, I'm not against a container-based approach in principle, but I definitely don't like the current duplication by splitting up the existing job into 2. There's probably a way to set something like container: ${{ null }} for the Mac jobs, to keep a single job definition. Possibly of interest: https://stackoverflow.com/questions/76925314/conditional-use-of-container-depending-on-the-group-it-runs-on-for-github-action

If using prebuilt container images, we can merge the tasks back and skip all the setup steps for Linux.