fix: enable url_standard compatbility flag
#2540
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We were running into an issue parsing the w3up UCAN. It turns out that we rely on the platform's URL class to parse DIDs in @iplad/dag-ucan: https://github.com/ipld/js-dag-ucan/blob/21bae4e0ad7ff76e0c7b73938dcc228726854a43/src/schema.js#L136 that's a little quirky since DIDs are really URIs not URLs but it works! except.... Cloudflare's URL implementation is not spec compliant! https://developers.cloudflare.com/workers/runtime-apis/web-standards/#url-api Critically, it only supports "URLs conforming to HTTP and HTTPS schemes" which, of course, DIDs do not. So what was happening was this attempt to parse the UCAN was trying to parse as CBOR, failing and swallowing the error, and then falling back to JWT, which also failed, but it swallowed the original error: https://github.com/ipld/js-dag-ucan/blob/21bae4e0ad7ff76e0c7b73938dcc228726854a43/src/lib.js#L50 I did a little extra work to parse as CBOR in a context where the error wouldn't be thrown: https://github.com/nftstorage/nft.storage/pull/2536/files#diff-81e52ca70244fb599510375cb3757b01c7efda6d0c16a9b570fc49451eb08d17R71 and that finally gave me the confusing-but-ultimately-illuminating error "ParseError: Capability has invalid 'with: "did:key:z6M..."', value must be a valid URI string" THE GOOD NEWS is that this is trivial to fix - Cloudflare has a url_standard compatibility flag that I've tested in staging and confirmed fixes this error: #2537
Deploying nft-storage with
|
| Latest commit: |
9276446
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://a8abf751.nft-storage-1at.pages.dev |
| Branch Preview URL: | https://fix-staging-ucan-parsing.nft-storage-1at.pages.dev |
travis
pushed a commit
that referenced
this pull request
Apr 3, 2024
🤖 I have created a release *beep* *boop* --- ## [4.5.0](api-v4.4.3...api-v4.5.0) (2024-04-02) ### Features * POST /uploads can write uploads to w3up for web3 serving and dagcargoless filecoin deal-making ([#2522](#2522)) ([0c77cb3](0c77cb3)) * uploadCarWithStat avoids copying stat.carBytes via Blob construction ([#2543](#2543)) ([bcb67a3](bcb67a3)) ### Bug Fixes * add a fake piece hasher to try to fix staging upload issues ([#2542](#2542)) ([542983c](542983c)) * avoid car repack ([#2551](#2551)) ([b0d622e](b0d622e)) * avoid car repack ([#2552](#2552)) ([b4ff571](b4ff571)) * change to post method ([#2547](#2547)) ([65e20fb](65e20fb)) * enable `url_standard` compatbility flag ([#2540](#2540)) ([bdde10a](bdde10a)) * enable the `url_standard` compatibility flag ([#2537](#2537)) ([e822bab](e822bab)) * enabled nodejs compat ([#2545](#2545)) ([654cde7](654cde7)) * log out proof ([225cf28](225cf28)) * log proof for debugging ([#2535](#2535)) ([0ecace3](0ecace3)) * more logging ([#2536](#2536)) ([11ae646](11ae646)) * remove debugging console.logs and nodejs_compat flag ([#2553](#2553)) ([6ceb299](6ceb299)) * Revert "fix: try older w3up client" ([#2549](#2549)) ([b31847b](b31847b)) * revert debugging prs ([#2539](#2539)) ([ba22d01](ba22d01)) * try older w3up client ([#2548](#2548)) ([5d94ec5](5d94ec5)) * upgrade @ipld/dag-cbor ([#2534](#2534)) ([964a12c](964a12c)) ### Other Changes * add a bunch of console.logs ([#2544](#2544)) ([2fc34f3](2fc34f3)) * expose simple route to debug ([#2546](#2546)) ([8a9aef5](8a9aef5)) * simple test case to check limits ([#2550](#2550)) ([6a50a70](6a50a70)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We were running into an issue parsing the w3up UCAN.
It turns out that we rely on the platform's URL class to parse DIDs in @iplad/dag-ucan: https://github.com/ipld/js-dag-ucan/blob/21bae4e0ad7ff76e0c7b73938dcc228726854a43/src/schema.js#L136 that's a little quirky since DIDs are really URIs not URLs but it works! except....
Cloudflare's URL implementation is not spec compliant! https://developers.cloudflare.com/workers/runtime-apis/web-standards/#url-api
Critically, it only supports "URLs conforming to HTTP and HTTPS schemes" which, of course, DIDs do not.
So what was happening was this attempt to parse the UCAN was trying to parse as CBOR, failing and swallowing the error, and then falling back to JWT, which also failed, but it swallowed the original error: https://github.com/ipld/js-dag-ucan/blob/21bae4e0ad7ff76e0c7b73938dcc228726854a43/src/lib.js#L50
I did a little extra work to parse as CBOR in a context where the error wouldn't be thrown: https://github.com/nftstorage/nft.storage/pull/2536/files#diff-81e52ca70244fb599510375cb3757b01c7efda6d0c16a9b570fc49451eb08d17R71 and that finally gave me the confusing-but-ultimately-illuminating error "ParseError: Capability has invalid 'with: "did:key:z6M..."', value must be a valid URI string"
THE GOOD NEWS is that this is trivial to fix - Cloudflare has a url_standard compatibility flag that I've tested in staging and confirmed fixes this error:
#2537