Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Skipping the GHC 9.8 LTS / GHC 9.10 Nightly #7462

Closed
ysangkok opened this issue Jun 24, 2024 · 35 comments
Closed

Skipping the GHC 9.8 LTS / GHC 9.10 Nightly #7462

ysangkok opened this issue Jun 24, 2024 · 35 comments

Comments

@ysangkok
Copy link
Contributor

LTS major version Y in X.Y.Z GHC version for .0 LTS GHC version in .0 LTS LTS release date for .0 Lag after first release of .1 release of next major GHC
15 .2 8.8.2 2020-02-16 37 days after 8.10.1
17 .3 8.10.3 2021-01-24 11 days before 9.0.1
19 .2 9.0.2 2022-03-17 139 days after 9.2.1
20 .5 9.2.5 2022-11-16 101 days after 9.4.1
21 .5 9.4.5 2023-06-20 102 days after 9.6.1
22 .3 9.6.3 2023-12-16 68 days after 9.8.1

GHC 9.10.1 was released on 2024-05-10, which is already 45 days ago. Cabal-install-3.12, which is compatible, just got a release candidate.

Given that GHC devs decided to deprioritize GHC 9.8, I wonder if it makes sense to just skip the GHC 9.8 LTS, and move nightly to GHC 9.10? Then the GHC 9.6 LTS could be kept alive for longer.

@juhp
Copy link
Contributor

juhp commented Jun 24, 2024

Hmm, this has crossed my mind too, though it does feel quite unconventional...

I do wish this would be formalized more by ghc versioning.

@alaendle
Copy link
Member

Personally I tend to keep the current schema; so maybe just waiting for 9.6.6 (eta 2024-06-28 https://gitlab.haskell.org/ghc/ghc/-/milestones/401#tab-issues), create last lts-22, create lts-23 with ghc 9.8 and move nightly to ghc-9.10 feels more natural to me - and also means smaller steps (for us and potential stackage users that move from lts to lts).

@DanBurton
Copy link
Contributor

I'd like to defer the decision until after 9.6.6 is released. We certainly want to keep making lts-22 releases until that happens in order to pick up that ghc release.

I'm still inclined to do lts-23 with ghc-9.8 and to delay the nightly switchover to 9.10 until after that begins. The de-prioritization of ghc-9.8 was not declared to be permanent, they are just temporarily prioritizing continued support of 9.6. They stated:

The next release in this series [9.8] will likely be scheduled after the 9.6.6 release.

@endgame
Copy link

endgame commented Jun 30, 2024

@ysangkok has flagged that Amazonka doesn't have a GHC 9.8-compatible release on Hackage. What's your timeframe on cutting 9.8 LTS? I don't really use Stack, but if we can get a 9.8-compatible release onto Hackage before LTS-23 snapshots start rolling out, that seems good for those that do.

@ysangkok
Copy link
Contributor Author

ysangkok commented Jul 1, 2024

@DanBurton
they are just temporarily prioritizing continued support of 9.6

That's not the way I am reading it at all. The post outlines how 9.8 is only supported for the near future, not the long term. If 9.8 was supported for the long term, it wouldn't make sense to point out that it's supported for the "near future", this is a quote:

9.8: We plan to continue supporting this release series for the near future, but updates to this series might proceed at a slower rate than usual as we prioritise the new release (9.10) and supporting earlier releases with high uptake (9.6).

While they say that they'll focus on 9.8 because 9.6.6 is out, I am still getting the impression that after the upcoming 9.8 release, they'll go back to focusing on 9.6 again. This is also supported by the first sentence:

Given limited time and resources, we plan to prioritise work on the 9.6, 9.10 and master branches of GHC for the next few months.

Note how they're not mentioning 9.8 there at all. I am getting the impression that the upcoming 9.8.3 release might be the last.

@juhp
Copy link
Contributor

juhp commented Jul 3, 2024

I would say it is not certain that there will be a further release of 9.6 - well TBD I expect on an as-needed basis.
Whereas there most likely will be a 9.8.3 within a few months and I expect a 9.10 release before that. 🤷‍♂️
To me the main message was that they wanted to put out a 9.6 release first now because more users are still there currently.

(Though I admit I have heard it said that 9.8 is a weaker release than 9.10...
So it is possible LTS might stay on 9.8 for a shorter period before moving to 9.10 perhaps, once 9.12 is out.
This does seem a more incremental safer approach though overall.)

Of course nobody is forced to adopt ghc-9.8 now: users can choose to stay on lts-22 longer if they so wish.
For example Fedora is only just about to start moving to lts-22 from now.

@endgame bit hard to say right now: but I would hope ideally by this month or at least August. 🤞
I am planning to update lts-22 to ghc-9.6.6 this week...

@andreasabel
Copy link
Contributor

It would be good to publish a TODO list for the GHC bump (9.8 LTS, 9.10 nightly) with a proposed deadline so that maintainers and stackage curators and volunteers would know what to prioritize.
E.g. packages that do not accept base-4.20 would be high prio, and it would be good to collect these in an issue.
(In contrast, I would deem hashable-1.5 low prio (#7476).)

@juhp
Copy link
Contributor

juhp commented Jul 8, 2024

I think you are right @andreasabel - we don't yet have a good feeling for the state of the ecosystem wrt to ghc-9.10,
though we can afford to be a bit more radical/aggressive with Nightly in terms of pruning.
Still it would be good to give some early heads-up about breakage.
(Personally I haven't built ghc-9.10 yet, due to hadrian kind of requiring ghc-9.6 (well Cabal 3.10),
but I will soon - in the meantime people with it installed can start to open issues to track core library breakage via curator check output.)

I do feel it would be nice(r) to have the ghc-9.8.3 release first, but we don't know yet when that will happen.
I suppose we could ask GHC HQ for an ETA, but my current understanding is they plan it after ghc-9.10.2.

@andreasabel
Copy link
Contributor

So, almost two months later, what's the take on a timeline for nightly 9.10 and/or LTS 9.8 now?
There haven't been any new GHC releases in the meantime, and the milestone horoscope https://gitlab.haskell.org/ghc/ghc/-/milestones does not indicate specific upcoming ones, except maybe GHC 9.12:
Screenshot 2024-08-31 at 09 16 32
Especially for 9.6 there isn't much planned anymore.

@ysangkok
Copy link
Contributor Author

ysangkok commented Sep 4, 2024

@andreasabel I think we're waiting for 9.8.3, at least that's how I interpret Jens' last message. But it seems 9.8/9.10 currently are not being worked on, since the 9.12 fork is coming up on Sept 18th (scroll to the top of the page, note the chart shows the creation of issues for this milestone). After the 9.12 fork, and the release of 9.10.2, 9.8.3 is scheduled next, according to Zubin. Not sure why 9.12 wasn't covered in the blog post about release priorities.

You can view the ghc 9.8 branch and the ghc 9.10 branch. They haven't seen commits for three months.

@phadej
Copy link
Contributor

phadej commented Sep 4, 2024

Not sure why 9.12 wasn't covered

maybe because that post is from May this year, about two weeks after 9.10.1 release?

@ysangkok
Copy link
Contributor Author

ysangkok commented Sep 4, 2024

that post is from May

Ok, I thought they might have had a rough schedule for 9.12 at that time, but since it was so quickly after 9.10, it does make sense that they wouldn't have thought it'd influence the releases they mention.

They haven't seen commits for three months.

Actually, I suppose this doesn't mean much, since apparently merge requests (MRs) are only merged when the release is being prepared. I suppose that means there is not really a 'current branch' for 9.8/9.10, you have to synthesize it from the MRs you think will go into the next release.

@andreasabel
Copy link
Contributor

It would be good to publish a TODO list for the GHC bump (9.8 LTS, 9.10 nightly) with a proposed deadline so that maintainers and stackage curators and volunteers would know what to prioritize.

I propose to go ahead with this now as no minor GHC releases are imminent.

@alaendle
Copy link
Member

I have since changed my mind a little and now advocate waiting for a another new GHC release.

I hope for GHC 9.8.3 (or 9.10.2 if necessary).

Mainly because there is currently no GHC 9.8/9.10 release in which all packages are the same or newer than 9.6.6. So an upgrade would be a downgrade for some dependencies - which I would find strange. In addition, the 9.8 series has a number of open problems (https://gitlab.haskell.org/ghc/ghc/-/milestones/398#tab-issues), some of which are serious.

All in all, not an ideal situation, and we will certainly never meet all of our users requirements, but it seems to me that the primary requirement for Stackage to provide a stable environment is currently best met in the 9.6 series.

And believe me, I'm sorry to have to say this, because I'm really a fan of new features and generally latest bits 😿

@ysangkok
Copy link
Contributor Author

no GHC 9.8/9.10 release in which all packages are the same or newer than 9.6.6.

Ah! I finally realize you're probably referring to boot packages (e.g. Cabal) here. It took me a while, because I was thinking of regular Hackage packages. But I agree, it does make sense to avoid a downgrade of boot libraries, especially since e.g. the Cabal bug relating to GHC plugins has been fixed since November 2023, so many have probably forgotten about the issue by now, if they ever knew about it.

@Bodigrim
Copy link
Contributor

Judging from https://gitlab.haskell.org/ghc/ghc/-/merge_requests/13319 and https://gitlab.haskell.org/ghc/ghc/-/merge_requests/13320, I'd speculate that GHC 9.8.3 might be released in upcoming weeks.

@juhp
Copy link
Contributor

juhp commented Oct 25, 2024

I don't think we are going to skip 9.8 LTS, so I guess this can be closed.

9.8.3 is now in Nightly so we should be able to cut/release lts-23 soon based on it.
We just need to push @bergmark 's tool (#6359) to generate the build-constraints.yaml first (lts22 was kind of bootstrapped out of tree). Also we need to prepare a PR for ghc-9.10.

@juhp juhp closed this as completed Oct 25, 2024
@andreasabel
Copy link
Contributor

9.8.3 is now in Nightly

Wow, that was quick, 9.8.3 isn't even on ghcup yet.
Maybe too quick? At least quite unusual considering the usual (intended) delays...

This means that CIs using the --system-ghc installed by ghcup need to wait bumping their nightly. Not a big issue.

No action needed, we can sit this out.

@juhp
Copy link
Contributor

juhp commented Oct 26, 2024

(I see - currently also facing trouble with curator and ghc-9.10: particularly it's not finding Cabal)

@andreasabel
Copy link
Contributor

andreasabel commented Oct 27, 2024

P.S.:

Moving nightly to 9.10 is maybe blocked by this issue:

@andreasabel
Copy link
Contributor

Since @hasufell decided to not add 9.8.3 to the ghcup default channel, I wonder whether nightly shouldn't be downgraded to 9.8.2 again.
Creating a schism between ghcup and stack seems not advisable as it brings confusion and extra costs to the end-users.
E.g., for maintaining my stack CIs that use ghcup to install ghc, I have now to choose to either add the vanilla channel to ghcup everywhere or to freeze the CIs to nightly-2024-10-21 which was the last using 9.8.2.
ATTN: @juhp

@mihaimaruseac
Copy link
Contributor

Oh, this is going to be nasty. I think downgrading ghc in stackage will also cause issues with bounds and packages that have already been upgraded.

@ysangkok
Copy link
Contributor Author

ysangkok commented Nov 6, 2024

Given that Stackage is used with Stack a lot, not necessarily with GHCup, I don't think it makes sense to limit Stackage users to releases in the default GHCup channel. At work, we're using the vanilla channel with no issues. As far as I know, one problem with vanilla releases are e.g. man pages. One could argue that they are broken because most people look up documentation online. So should it really hold Stackage back? The GHC team is aware of these issues and it seems they are not prioritizing them. I think it's fine for Stackage to use vanilla GHC releases. After all, that's what everyone did before GHCup was released.

@Bodigrim
Copy link
Contributor

Bodigrim commented Nov 6, 2024

The problem is that moving LTS from GHC 9.6.6 to 9.8.3 would downgrade filepath from 1.4.300.1 to 1.4.200.1. This is not cool: we are forcing stability-focused clients to throw away bug fixes.

@hasufell
Copy link
Contributor

hasufell commented Nov 6, 2024

E.g., for maintaining my stack CIs that use ghcup to install ghc, I have now to choose to either add the vanilla channel to ghcup everywhere or to freeze the CIs to nightly-2024-10-21 which was the last using 9.8.2

GHCup can also use stack metadata to pick GHC: https://www.haskell.org/ghcup/guide/#using-stacks-setup-info-metadata-to-install-ghc

@juhp
Copy link
Contributor

juhp commented Nov 7, 2024

I think the best action for filepath, would be to encourage GHC HQ to make a 9.8.4 release, but that will take some time.
I don't see Stackage downgrading 9.8 that just does not make sense, nightly has been on 9.8 for months.
Are people saying that 9.8.2 is better than 9.8.3?

It is getting a bit ridiculous: so I opened haskell/ghcup-metadata#257

@andreasabel
Copy link
Contributor

@Bodigrim wrote:

The problem is that moving LTS from GHC 9.6.6 to 9.8.3 would downgrade filepath from 1.4.300.1 to 1.4.200.1. This is not cool: we are forcing stability-focused clients to throw away bug fixes.

That would suggest to keep LTS on GHC 9.6 until GHC 9.8.4 emerges.
Maybe no further action is needed then until GHC 9.8.4. (Personally, though, I might stay pinned to nightly-2024-10-21 until then, to save myself from modifying my CI scripts.)

@AndreasPK
Copy link

In case this helps with decision making I just want to state clearly that there is no concrete plan for a 9.8.4 release at this point. And that unless new serious issues arise it might not happen at all.


The problem is that moving LTS from GHC 9.6.6 to 9.8.3 would downgrade filepath from 1.4.300.1 to 1.4.200.1. This is not cool: we are forcing stability-focused clients to throw away bug fixes.

While I don't know which notion of stability you had in mind with this statement there are important correctness fixes in 9.8.3 (see release notes) that I believe users are more likely to encounter than the only documented bugfix in filepath-1.4.300.1 (haskell/filepath#219).

Are people saying that 9.8.2 is better than 9.8.3?

If someone is aware of noteworthy regressions in 9.8.3 over 9.8.2 it would be great if they could be reported upstream. I'm not currently aware of any.

@hasufell
Copy link
Contributor

hasufell commented Nov 9, 2024

While I don't know which notion of stability you had in mind with this statement there are important correctness fixes in 9.8.3 (see release notes) that I believe users are more likely to encounter than the only documented bugfix in filepath-1.4.300.1

This assessment is true. However, I did not suggest that 9.8.2 is "better" than 9.8.3. But 9.8.3 has been a blunder and in hope of a quick follow up release (9.8.4) I think it's fair to consider skipping it.

In case this helps with decision making I just want to state clearly that there is no concrete plan for a 9.8.4 release at this point.

This means I can't rely on GHC HQ to ship boot libraries correctly, so I decided I will take it into my own hands.

Now the question is whether stack and ghcup want to coordinate this or whether stack wants to stick to vanilla upstream bindists.

For the release at hand, I'm inclined to just fix it in-place without a version bump, since the exposure is rather low.

I also want everyone to understand that I'm not paid for this and it will greatly delay the availability of new GHC releases in GHCup.

@juhp
Copy link
Contributor

juhp commented Nov 9, 2024

@bgamari : is it possible to have a quick ghc-9.8.4 release to get over this?


Discussion already continuing in haskell/ghcup-metadata#255 but I feel obliged to respond:

While I don't know which notion of stability you had in mind with this statement there are important correctness fixes in 9.8.3 (see release notes) that I believe users are more likely to encounter than the only documented bugfix in filepath-1.4.300.1

Indeed, which is why it is surprising that ghcup still does not list 9.8.3 yet, when it basically only improves on 9.8.2.

This assessment is true. However, I did not suggest that 9.8.2 is "better" than 9.8.3. But 9.8.3 has been a blunder and in hope of a quick follow up release (9.8.4) I think it's fair to consider skipping it.

It doesn't seem fair when many people want and are asking for 9.8.3 to be available in ghcup by default.

Mike Pilgrem, who I believe is a Windows user, even revised stack-3.1.1 to allow it to build easily with ghc-9.8.3, so I have to wonder how serious the filepath version issue is relatively overall.

Default listing in GHCup also affects haskell-actions/setup for example, and in general is reducing the exposure and testing of 9.8.3 in the ecosystem.

In case this helps with decision making I just want to state clearly that there is no concrete plan for a 9.8.4 release at this point.

This means I can't rely on GHC HQ to ship boot libraries correctly, so I decided I will take it into my own hands.

Now the question is whether stack and ghcup want to coordinate this or whether stack wants to stick to vanilla upstream bindists.

That should be discussed also with @mpilgrem, perhaps in https://github.com/commercialhaskell/stackage-content/, but Stack defaults to using the official vanilla ghc upstream binaries and I am not sure that we or Mike want to change that. Considering also it would delay the availability of new ghc versions considerably too it seems, I am not sure it is a good idea.

For the release at hand, I'm inclined to just fix it in-place without a version bump, since the exposure is rather low.

Even if that might be okay in this case, it could also set a worrying precedent leading to another schism in the Haskell space?
A one-off for 9.8.3 filepath might be innocent enough, but setting this as the ghcup stance going forward without wider consensus may lead to more user confusion.

Though even for 9.8.3 it would create two differing "official" binaries amongst users.

I also want everyone to understand that I'm not paid for this and it will greatly delay the availability of new GHC releases in GHCup.

(The discussion is getting more off topic already, but I want to say) I don't think anyone is asking or expecting you to do more - just maintaining ghcup itself is already quite enough of a service to the community, thank you. 🙏

To me, ghcup's primary role should be just to allow users to easily download and install the latest versions of ghc easily and seamlessly. It is okay that ghcup chooses to label certain versions as recommended or stable, etc (though it should really be done with broader consensus IMO), but as the official Haskell installer project it is not really ghcup's role to curate and decide unilaterally if official released versions should be listed or not. Inexperienced users can follow its default or given recommendations - but experienced users should be able to easily pull down the latest vanilla binary version regardless in my opinion without any extra effort or friction. We all want to see high quality ghc releases, but if ghcup can't list the latest releases by default in a timely fashion then it doesn't seem to fully meet the requirements of experienced Haskellers - I would add that if official ghc binaries are "botched" as you call it, it is upstream's problem not ghcup's.

On the other hand I have not seen any complaints about the state of vanilla ghc-9.8.3 from Stackage Nightly (stack) users (and Stackage Nightly has been on 9.8 seen the beginning of this year).

@hasufell
Copy link
Contributor

hasufell commented Nov 9, 2024

To me, ghcup's primary role should be just to allow users to easily download and install the latest versions of ghc easily and seamlessly. It is okay that ghcup chooses to label certain versions as recommended or stable, etc (though it should really be done with broader consensus IMO), but as the official Haskell installer project it is not really ghcup's role to curate and decide unilaterally if official released versions should be listed or not.

This is not how GHCup sees itself: https://hasufell.github.io/posts/2023-11-14-ghcup-is-not-an-installer.html

If you have a different opinion, you are free to:

  1. create your own GHCup metadata
  2. provide a boostrap script that configures GHCup to use that by default

You can do exactly that already and use the vanilla channel.

There is a reason the vanilla channel is not the default.

@mpilgrem
Copy link
Member

mpilgrem commented Nov 9, 2024

Echoing @juhp, @bgamari, if the 'cost' to the GHC project of releasing a GHC 9.8.4 that is like GHC 9.8.3 but incorporates the filepath-1.4.300.1 bug fix is low, it would be good to see such a GHC 9.8.4 sooner rather than later:

As for stack-3.1.1 easing its lower bound on Hackage, no weight should be placed on that. stack is not a 'proper' library in the sense that (as it states) it exists on Hackage only to make its Haddock documentation available and for (rare) bootstrap purposes. Also, I did not know @hasufell's view of the seriousness of the filepath bug when I eased the lower bound (I considered only 'could build' not 'should build').

@bgamari
Copy link
Contributor

bgamari commented Nov 9, 2024

@juhp , we are not necessarily opposed to doing such a release but we do need to balance this need against other priorities. At the moment, our funding for GHC work is quite limited and, as such, doing work on another release necessarily comes at the expense of other work, some of which takes precedence. If we are going to do another release, I want to be certain that we have identified all of the factors driving the need, have backported all of the relevant work addressing those factors, and have updated our processes to avoid this happening again in the future.

@andreasabel
Copy link
Contributor

GHC 9.8.4 is on GHCup so I guess the nightly can be moved to 9.8.4 now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests