Json license list populated by git hash and not version number

[alandtse]

The licenseListVersion at https://spdx.org/licenses/licenses.json and https://spdx.org/licenses/exceptions.json is now populated by the git hash and not the version number.

EDIT: Current value is:

  "licenseListVersion": "81e2f82",

Moved from #1220

[swinslow]

@goneall For what it's worth, I also ran into this with the version number in the website index file. I manually updated it from the hash to 3.12 in the copy of index.html before uploading it to the license list website, but looks like we'll want to adjust it especially if it's affecting the JSON files as @alandtse noted.

[goneall]

I think this may have been caused by the move from Travis to Github Actions.

Version is passed in as a parameter to LicenseListPublisher:

license-list-XML/Makefile

Line 13 in 57b3b0d

VERSION = $(subst V,,$(subst v,,$(GITVERSION)))

I'll move this back to the LicenseListXML since it is either an issue with the Makefile or the Github Action (not sure which at this point).

[goneall]

@iamwillbar Any ideas why this may no longer work?

[goneall]

I'll manually fix the github data with the correct version.

@jlovejoy @swinslow @alandtse Hold off any merges to master in license-list-XML until I finish

[goneall]

@jlovejoy @swinslow @alandtse I manually updated the license-list-data repo and reset the version to the corrected tag. Hopefully rewriting the history doesn't trip up too many tools.

I'm not quite sure why this stopped working. The tag was associated with the has that showed up in the log. Perhaps someone more experienced in shell programming and Git can figure this out.

We definitely want to fix this before the next release.

[alandtse]

Don't we have a check to ensure the version number conforms if not, it should be added? It should fail the CI if the version is wrong.

[goneall]

Don't we have a check to ensure the version number conforms if not, it should be added? It should fail the CI if the version is wrong.

The version number is taken from the tag. If the License-List-XML commit is tagged, the tag will be used for the version otherwise it is the short hash of the tag.

The bug is in the CI no longer picking up the tag for some reason.

Take a look at the Makefile referenced above and let me know if you see any issues. I don't think the code that checks the version changed, but we did change how the code is checked out when we moved from Travis to Github Actions.

[goneall]

@swinslow Could you update the license list website with the license data from license-list-data repo tagged v3.12?

[swinslow]

@goneall So, I've pulled the updated version and synced it to the website... except, weirdly, the licenses.json file doesn't seem to be coming through and is still showing the old version with the incorrect version number: https://spdx.org/licenses/licenses.json

Weirdly, after several re-checks I can tell that I have the correct version, with the correct v3.12 version number; and it has gotten pushed to S3 correctly: https://spdx-org-s3-website.s3.amazonaws.com/licenses/licenses.json

I'm going to wave my arms and guess that it's some sort of caching issue? But I'll keep an eye on it and if it doesn't resolve shortly, I'll ask the LF IT team who manages the AWS account to see if they have any thoughts.

[goneall]

@swinslow I just looked from my local machine and https://spdx.org/licenses/licenses.json is showing the correct version number.

However, when I run the SPDX tools that read the same file, they are somehow fetching the old one - definitely sounds like caching issues.

[alandtse]

It updated for my machines so I think it's a AWS propagation issue.

Did we change the url keys to be relative? It just broke spdx-license-diff. I didn't see anything in the notes about a change there. Perhaps we should adopt semantic versioning so any libraries relying on the list are aware of breaking changes?

[goneall]

Did we change the url keys to be relative?

@alandtse Can you describe a bit more on this issue?

I'm not aware of any change that would be incompatible, but there was a major upgrade in the LicenseListPublisher which may have caused an issue.

[alandtse]

spdx-license-diff broke in it's update function because it's now trying to download things relative to the extension. It relies on parsing the license/exception.json for full information.

"licenses": [
    {
      "reference": "./AFL-2.0.json",
      "isDeprecatedLicenseId": false,
      "detailsUrl": "./AFL-2.0.html",
      "referenceNumber": 0,
      "name": "Academic Free License v2.0",
      "licenseId": "AFL-2.0",
      "seeAlso": [
        "http://wayback.archive.org/web/20060924134533/http://www.opensource.org/licenses/afl-2.0.txt"
      ],
      "isOsiApproved": true
    },

I can't know for sure without looking at an old json, but I believe the reference and detailsUrl used to be absolute urls.

[alandtse]

Also, I believe the detailsUrl used to reference the json and the reference was the html. I have a potential fix for for spdx-license-diff, but if your'e going to revert the keys back then I'll wait.

EDIT: Actually, I have a fix for this version or a reversion. So I'll be good for now.

[goneall]

@alandtse This was likely caused by an unintentional side effect of changes in the LicenseListPublisher.

Looking back at version 3.11, the detailsUrl was absolute, but the reference was relative.

I'm unsure whether they should be absolute or relative. If they are relative, it would work for the archive URLs for past versions. If we make them absolute, the archive website would end up pointing to the current version, not the correct version.

I'm tempted to leave it as is (even though it was unintentional). Feedback on this is welcome.

Even if I made this compatible in the LicenseListPublisher, we would need to regenerate all the data and re-publish which would take a few days.

If you could handle either case (relative or absolution), that would future proof the implementation.

[alandtse]

I'm ok with either case. I just think we should probably test to make sure it doesn't change in the future and consider whether we adopt a versioning system that would be useful for breaking changes like this.

[goneall]

I just think we should probably test to make sure it doesn't change in the future and consider whether we adopt a versioning system that would be useful for breaking changes like this.

I agree - do feel free to make a pull request to enhance the CI for additional tests.

In addition, we should specify the format in the formal spec under change control. I added an issue to spdx spec which is waiting for a volunteer.

[Bo98]

I manually updated the license-list-data repo and reset the version to the corrected tag.

Can you mark 3.12 as the "latest release"? https://github.com/spdx/license-list-data/releases

[swinslow]

Done -- thank you @Bo98! Ooof, I hadn't realized I'd left the release listed as "draft" and hadn't hit the "Publish" button. Thanks for flagging.

[jlovejoy]

@goneall @swinslow @alandtse - where are we at with this? (above my head... ;)

[alandtse]

I think it's fixed and can be closed.