chapters/appendix-I: Bump license list to v2.6 (via new script) #10
Conversation
|
I don't know of any nginx configs available, but I'll went ahead and filed a ticket with the Linux foundation helpdesk - request ID #43220. |
The newline removal in exception names works around [1]. The UTF-8 fallback works around the source not being served with a charset: $ curl -I https://spdx.org/licenses/licenses.json HTTP/1.1 200 OK Server: nginx Date: Tue, 18 Jul 2017 20:12:58 GMT Content-Type: application/json Content-Length: 108139 Connection: keep-alive Vary: Accept-Encoding Last-Modified: Fri, 06 Jan 2017 21:34:52 GMT Expires: Tue, 18 Jul 2017 22:12:58 GMT Cache-Control: max-age=7200 Strict-Transport-Security: max-age=16070400 Accept-Ranges: bytes $ curl -I https://spdx.org/licenses/exceptions.json HTTP/1.1 200 OK Server: nginx Date: Tue, 18 Jul 2017 20:13:02 GMT Content-Type: application/json Content-Length: 7985 Connection: keep-alive Vary: Accept-Encoding Last-Modified: Fri, 06 Jan 2017 21:34:52 GMT Expires: Tue, 18 Jul 2017 22:13:02 GMT Cache-Control: max-age=7200 Strict-Transport-Security: max-age=16070400 Accept-Ranges: bytes [1]: https://github.com/spdx/license-list-data/issues/10 Subject: Newline in 389-exception name
Generated with: $ ./bin/pull-license-list.py Then I removed the trailing link labels now that the tables are using inline links. The inline links make the table a bit wider, but they are easier to automatically manage because each table is self-contained (vs. accumulating a set of link lables between all tables). The date formatting for "6 Jan 2016" is unfortunate. I'd prefer "2016-01-06" or "January 6th, 2016", but the script is just pulling the raw releaseDate from the JSON, so we get whatever the upstream data is using.
wking
force-pushed
the
automatic-license-list-update
branch
from
aea3ce1
to
1e38bd9
Compare
|
Unless I'm misunderstanding something, this SO article makes the case that adding a charset to application/json is pointless because all RFC-compliant clients must ignore the charset on this particular type. Is it only /licenses/licenses.json that you are requesting this be added for? Doing a quick check on the server shows many ASCII .json files under licenses, and I'm not aware of an nginx or httpd option to dynamically pick the correct charset. |
All .json files in the /licenses directory. All of the JSON files should be encoded using UTF-8. I believe the ASCII text found was actually the word "ASCII" in the data portion of the files. @wking Based on the article SO article referenced above, perhaps we do not need the charset declaration and can just assume it is UTF-8. BTW - if you find any encoding issues, it is likely due to the source text in the license-list or license-list-XML repositories being incorrect. Everything should be UTF-8, but I have found several cases where the source license text was not in fact UTF-8. |
Which adds support for parsing JSON directly from bytes without first decoding to a string [1,2]. This allows interacting with RFC-compliant servers, because RFC 7159 does not define a charset parameter in its application/json registration [3]. Reported by Eric Searcy [4]. [1]: https://docs.python.org/3/whatsnew/3.6.html#json [2]: https://bugs.python.org/issue17909 [3]: https://tools.ietf.org/html/rfc7159#page-11 [4]: spdx#10 (comment)
|
On Thu, Nov 09, 2017 at 06:10:47PM +0000, Eric Searcy wrote:
Unless I'm misunderstanding something, [this SO
article](https://stackoverflow.com/a/26206930) makes the case that
adding a charset to application/json is pointless because all
RFC-compliant clients must ignore the charset on this particular
type.
Fair. It looks like Python has added direct-byte-handling for this in
3.6 [1,2], so I'll change my mind and say it's appropriate to continue
to not set charset for application/json.
[1]: https://docs.python.org/3/whatsnew/3.6.html#json
[2]: https://bugs.python.org/issue17909
|
|
This PR is superseded by #151. |
Which adds support for parsing JSON directly from bytes without first decoding to a string [1,2]. This allows interacting with RFC-compliant servers, because RFC 7159 does not define a charset parameter in its application/json registration [3]. Reported by Eric Searcy [4]. [1]: https://docs.python.org/3/whatsnew/3.6.html#json [2]: https://bugs.python.org/issue17909 [3]: https://tools.ietf.org/html/rfc7159#page-11 [4]: #10 (comment)
Based off #10 this resolves a minor bug in the license list update script and updates the license list to v3.7.
Which adds support for parsing JSON directly from bytes without first decoding to a string [1,2]. This allows interacting with RFC-compliant servers, because RFC 7159 does not define a charset parameter in its application/json registration [3]. Reported by Eric Searcy [4]. [1]: https://docs.python.org/3/whatsnew/3.6.html#json [2]: https://bugs.python.org/issue17909 [3]: https://tools.ietf.org/html/rfc7159#page-11 [4]: spdx#10 (comment)
Based off spdx#10 this resolves a minor bug in the license list update script and updates the license list to v3.7.
Add
pull-license-list.pyto automatically pull the current license and exception JSON into Markdown tables. Using JavaScript might have been more idiomatic than the Python script, but it's faster for me to write the Python; someone who is more familiar with JavaScript can translate the script and incorperate it into the build tooling later if we decide against carrying the auto-generated tables in this repository.Two things made the auto-generation more difficult than it needed to be:
Newlines in exception names (spdx/license-list-data#10).
[edit: addressed below] No charset declaration in JSON served from spdx.org:
it should have been
Content-Type: application/json; charset=utf-8. I don't know if there is a repository with the Nginx config used for spdx.org, so I haven't filed an issue for this last one.Fixes #6.