Stronger emphasis on reaching Recommendation #443
Comments
plehegar
added
Type: Enhancement
AC-review
There are several perma-threads tangled up here and it's hard to resolve them separately:
So if you want to insist on an HR chokepoint, you should also insist that the process emphasize perpetual maintenance. Maybe Recommendations auto-expire after x number of years? Maybe adopt the WHATWG innovation of putting "support tables" inline in specs that show the reader the current results for interoperability (+ A11Y, I18N, etc.?) tests in various implementations, and let the reader assess how well the spec is implemented and maintained. But more fundamentally, the W3C Process can't force the external world to value the Recommendation label enough to use it as a way to pressure a group to do the extra work needed to achieve it. The browser engine monoculture exacerbates this problem because the way things work in all the Chromium browsers + iOS Safari is the de facto standard whatever W3C says. My personal takeaway: People in the W3C community should invest less of their brainpower in tweaking the formal Process Document, and more energy into improving the informal culture. If a WG is not addressing issues raised in horizontal reviews, that's a COMMUNICATIONS problem more than a Process problem.
|
In my view this is the wrong way around. Instead, I made a proposal at #406 (comment) to make CRs expire on inactivity, with ways to make it fairly easy for WGs to decide the timeout period and to do refreshes to reset the clock. |
|
Can we agree that unmaintained Recommendations and CRs should lose whatever claims to authoritativeness they possess? "Maintained" in some rare cases might mean that a qualified group has re-assessed the spec and determined that there are no issues to fix and that it still meets the need for which it was crafted. But in general, if a spec isn't getting continually updated it's becoming irrelevant. But fundamentally, what real problem is "stronger emphasis on reaching Recommendation" trying to fix? I don't think it's "broader interoperability on the web", because web interoperability is getting better at the same time "emphasis on reaching Recommendation" is declining, and it's generally accepted that the current WD or even latest GitHub commit is the version developers should reference. |
|
@michaelchampion wrote
That depends... The only claim to authorative that a CR has is that the Working Group believes it has described an appropriate solution to a problem, in a way that could be used as a basis for interoperable development. Given the many structural barriers (most particularly language and timing) that limit the participation in a Working Group, the broader review and the checking that the WG solution works at least for those who are interested in implementing at the bleeding edge is a useful reality check. If the CR is neither updated (perhaps just to say "status - same as before"), nor becomes a Recommendation, it seems reasonable to ask whether something is going wrong - has the Working Group changed its mind about something, is there just not enough information to know if their proposal is actually any good, or is it just that having laid out a solution for the benefit of the Working Group they don't have enough motivation to clarify this for the rest of the ecosystem by moving the document along the process. The latter problem is something we could work out how to address. Given that W3C has a competent techincal staff, perhaps we should be asking them to do more of the final cleanup that results in a spec that is ready actually becoming a Recommendation. For a Recommendation, we can probably agree that where "unmaintained" means that there are known errors that have been brought to W3C's attention, it is important that we have a mechanism to update the published Recommendation that actually works, whether the changes are small ones or really a new version with significant improvements, new or re-designed features, or the like. My reluctance to specify a particular rule for this is based on the fact that there are different kinds of recommendation, written differently and operating in different environments. The XML Recommendations are very widely used, and while it might be helpful to have some minor corrections, it seems unlikely for the most part that we need a way to write a complete new version of XML - it seems that the market has divided into those who are basically happy with what there is, and those who want something new and shiny (like curly brackets and the opportunity to invent new approaches to internationalisation each time you make a vocabulary, instead of having a standard way of making it work). On the other hand, it seems plausible that the specification of e.g. like Verifiable Claims will evolve, for example to include details of a protocol allowing interoperability. None of this seems to militate against reaching Recommendation, but instead suggests that we need to be clear about the ways we do maintenance. |
Issues, including those raised by Horizontal review, are expected to be addressed in order to enter the CR phase. To go from CR to REC, you need:
The rest, including issues from HR groups, is expected to be addressed earlier. A few chosen bits from the Process supporting that (found both in the 2019 and 2020 versions of the Process):
So, so far, at least on a first pass, CR, not REC, is where horizontal review issues (and any other issue) are expected to be addressed, and Process 2020 changes nothing about that. However we also have this bit (which is also in both P2019 and P2020):
The goal is that you're not stuck and unable to republish anything if you've only addressed some of the issues that have been raised since the last CR, so that you can keep an up to date document on TR. The intent is that you're expected to address all the issues over time, but may only address a few at a time. However, I can see how this effectively removes the obligation to deal with issues at all. The transition to REC would force you to deal with all issues, but if a working group doesn't push a spec to REC, and stays in CR indefinitely, there's never a forcing function to address all issues. I think this is a problem worth addressing, but we should stick to the intent of CR, and address it at that phase. Encouraging people to go to REC may also be a good thing, but not for that reason once we plug this hole. Note that the same problem exists about the lack of requirement to address all issues filed against a REC before the next publication of the same REC. Mostly, I agree with @michaelchampion that if a WG is not addressing issues raised by HR groups, we have more than a Process problem, but a simple tweak to the process could provide a bit of a nudge towards good behavior. Here what I think we could do: for each issue opened against a CR (or REC) which has not yet been solved by the time the Working Group wants to publish the next CR Snapshot (or REC), the working group has to (a) make an explicit Group Decision to defer the issue the a subsequent publication, and (b) mark the issue explicitly in the spec. (a) makes it so that a group cannot accidentally or negligently ignore issues. They need to make an explicit decision, which creates an opportunity to discuss the issue within the group, and needs consensus to move forward. Things can be deferred, but they cannot be forgotten. That decision also becomes an opportunity to file a Group Decision Appeal, should people find that the decision to defer is made in bad faith and is a "wontfix" is disguise. (b) Gives visibility into the problem to readers of the spec who aren't following the day to day tribulations of the WG. |
|
[Responding as a Chair, and with my own observations, not necessarily BBC's]
@michaelchampion Where to start?! The comparison here is between shipping a Rec or remaining at CR.
I have not seen any evidence to support this statement and suspect that it is false. As more and more fine-grained feature-based specifications are incubated and implemented by potentially only one UA, if they do not advance through the Process's maturity levels, web interoperability declines. In particular, web interoperability measurability declines for specs not at Rec because there is no stable comparison point and no requirement for the existence of tests.
This is an independent and orthogonal problem that deserves separate consideration: if we are concerned about which version developers should reference then we have the same problem whatever the maturity level is, when there may be multiple proposals for features or wording, either in pull requests, editor's drafts, working drafts or CRs. As demonstrated by P2020 it is possible in principle to iterate Recs, adding "non-Rec" features as notifications to developers. In my view the version referencing issue is not an argument against emphasising reaching Recommendation. Rather, it is possibly an argument in favour, depending on the publishing WG's approach. |
@frivoal probably a minor nit, that could be covered by the word "solved" (!), but noting that this could be abused, there would also need to be the opportunity for the WG to reject the issue as "no change needed" having given it due consideration. |
|
@nigelmegitt wrote
Hmm, that's an interesting hypothesis. It's not consistent with my experience when I worked on the Edge team; the folk wisdom in the browser implementer community was "CR is good enough to get real interoperability" I freely admit that my subjective experience and the implementer folklore could be wrong, what evidence could we use to shed light here? https://caniuse.com/#home and https://caniuse.com/#index is the best-known resource to look at features by interoperability status and standards maturity. I don't see an easy way to break out the data across time to see if standards support / interoperability is increasing or decreasing, but it looks pretty good at the moment. But to the question of whether "CR is good enough" or " we need stronger emphasis on reaching Recommendation", skimming the the CanIUse support tables doesn't show any obvious correlation between broad interoperability and Recommendation status. I don't get paid to argue about this stuff anymore :-) so I'm not inclined to do a more rigorous analysis of the data, but feel free to make a data-based case if I'm missing something. Some of @nigelmegitt 's bullet points for why it's important to move from CR to Rec are formally true but empirically irrelevant if we assume that actual web developers look at CanIUse rather than w3.org/TR to determine whether a spec is mature enough to use.: The support tables tell the reader whether a spec is actually implemented and thus whether it can be implemented, and whether the implementations are good enough to get interoperability. The other points seem to focus on the value of Recommendations for W3C rather than the value to the community. So, I continue to be unconvinced that "broad interoperability" is a compelling rationale for having the process document more strongly emphasize the need to reach Recommendation. There might be other reasons, especially if there are real examples of specs remaining in CR for a long time with unresolved A11Y, I18N, security, etc. issues that were raised by reviewers. I would welcome data with which to assess that hypothesis. In #443 (comment) @frivoal suggests some tweaks to the CR entrance and re-publication criteria that seem useful to prevent or mitigate such problems. |
|
@nigelmegitt fair point. Replace "solved" by "addressed" |
@michaelchampion I'm wary of assumptions (my own included); do you know of any data about how developers make these determinations? |
|
Again, it was folklore in the browser developer community when I worked on Edge that website developers used tools such as caniuse.com and MDN web docs rather than the W3C. I'm not expert on website analytics, but the first source of data I'd check is the https://en.wikipedia.org/wiki/Alexa_Internet#Alexa_Traffic_Rank for the various sites. Installing a Chromium extension https://chrome.google.com/webstore/detail/webrank-seo/mkhilblbmkdnapffblmecglknalglfji into Edge and checking the sites I see (low rank means higher traffic):
So, at first glance, there's less traffic to caniuse.com than w3.org/TR (although this doesn't indicate what developers are doing). But there's MUCH more traffic to MDN Web Docs than W3.org/TR, and presumably it's mostly developers who look at MDN Web Docs. So I may have mis-spoken referring to CanIUse.com, but am probably right that developers use MDN Web Docs much more than w3.org/TR. And MDN uses caniuse.com data , see https://hacks.mozilla.org/2019/09/caniuse-and-mdn-compat-data-collaboration/, so maybe I was indirectly right :-) |
|
We have trouble working out what is actionable here, and what specific change to the process might result? |
|
See also w3c/Guide#151 |
plehegar
added
the
Agenda+
|
The Revising W3C Process CG just discussed
The full IRC log of that discussion<fantasai> Subtopic: Stronger emphasis on reaching Recommendation<fantasai> github: https://github.com//issues/443 <fantasai> plh: Any objections to close this issue? We have an open PR on the Guide, and no proposals for altering the Process <fantasai> florian: Close it <fantasai> RESOLVED: Close #443 <plh> https://github.com//issues/462 |
css-meeting-bot
removed
the
Agenda+
|
@plehegar, for the purposes of the Disposition of Comments, it would be good to know if the original commenter (which you did not identify) is OK with the way this issue was closed. |
|
@chrisn, @nigelmegitt, please confirm if you're satisfied with the outcome. |
|
From #443 (comment), one thing that is actionable is:
Was any change made to address this? |
Not specifically, but the Team is empowered to deny a CR Snapshot request if there are any Formal Objections filed against the document. If a WG is repeatedly and intentionaly failing to address certain issues, it would certainly be possible to raise a Formal Objection against the next decision to publish (and this could be done by a member of the Team itself, or the WG, a Horizontal Group, or the public in general). Open to other ideas, but that's what we have in place at the moment. Note that we're intentionally not requiring a WG to address all issues in order to publish an updated CR, because that could prevent any publication (and thus leave the spec perpetually out of date) if issues are being filed faster than the WG can resolve them; but the WG should be making a good faith effort at addressing all the issues being reported to them. @cpn Lmk if that works for you, or if we should file a separate issue to discuss further; and also whether this is at least acceptable for P2023. |
[raised by the 2020 AC Review]
In the interest of broader interoperability on the web we'd like to see more emphasis placed on groups taking specs to Rec status rather than being maintained perpetually in CR. We suggest that this is prioritized in the next process iteration.
When a specification remains perpetually in Candidate Recommendation there seems to be no requirement that issues raised during horizontal review are addressed (as would be required before transitioning to Recommendation).
Related issues: #402 and #406
The text was updated successfully, but these errors were encountered: