test: fix "it_removes_lassie_temp_on_start" #631
Conversation
Also add more diagnostics to simplify troubleshooting in the future. Here is how to run this single test and get the additional info: ``` cargo test --test daemon_tests -- --nocapture ``` Signed-off-by: Miroslav Bajtoš <[email protected]>
|
An example of a failed CI run: |
| @@ -1,6 +1,6 @@ | |||
| // Signal that we are going to start the retrieval | |||
| Zinnia.activity.info("fetch:start"); | |||
| const response = await fetch("ipfs://QmdmQXB2mzChmMeKY47C43LxUdg1NDJ5MWcKMKxDu7RgQm"); | |||
| const response = await fetch("ipfs://bafybeiazvkej6ou3w6xmva5ed6suonxjv3jkhq4ke73q5hgmcjmf76uos4"); | |||
There was a problem hiding this comment.
Why does changing the CID fix the tests?
There was a problem hiding this comment.
It's a Lassie bug. I thought I linked to it somewhere in the changes, looks like I did not.
There was a problem hiding this comment.
Thanks! I subscribed to this issue as well. Could this be impacting Spark production data?
There was a problem hiding this comment.
I'm getting a slightly different result with spark's manual check:
[...]
Querying IPNI to find retrieval providers for QmdmQXB2mzChmMeKY47C43LxUdg1NDJ5MWcKMKxDu7RgQm
IPNI returned 56 provider results
Failed to fetch QmdmQXB2mzChmMeKY47C43LxUdg1NDJ5MWcKMKxDu7RgQm from /dns/http.f02620.devtty.eu/https/http-path/%2Fipni-provider%2F12D3KooWNXvbyvLUUd1qQEqhzjTpVoT5fdYUZEv4RJSxZ3rDF2c7 using http
Error: Cannot parse "/dns/http.f02620.devtty.eu/https/http-path/%2Fipni-provider%2F12D3KooWNXvbyvLUUd1qQEqhzjTpVoT5fdYUZEv4RJSxZ3rDF2c7": unsupported protocol "https"
at multiaddrToHttpUrl (file:///Users/julian/dev/filecoin-station/spark/lib/multiaddr.js:10:7)
at getRetrievalUrl (file:///Users/julian/dev/filecoin-station/spark/lib/spark.js:272:21)
at Spark.fetchCAR (file:///Users/julian/dev/filecoin-station/spark/lib/spark.js:106:19)
at Spark.executeRetrievalCheck (file:///Users/julian/dev/filecoin-station/spark/lib/spark.js:80:16)
at eventLoopTick (ext:core/01_core.js:178:11)
at async file:///Users/julian/dev/filecoin-station/spark/manual-check.js:23:1 {
code: "UNSUPPORTED_MULTIADDR_PROTO"
}
Measurement: {
cid: "QmdmQXB2mzChmMeKY47C43LxUdg1NDJ5MWcKMKxDu7RgQm",
minerId: "f0frisbii",
indexerResult: "OK",
statusCode: 702,
byteLength: 0,
providerId: "12D3KooWNXvbyvLUUd1qQEqhzjTpVoT5fdYUZEv4RJSxZ3rDF2c7",
protocol: "http",
providerAddress: "/dns/http.f02620.devtty.eu/https/http-path/%2Fipni-provider%2F12D3KooWNXvbyvLUUd1qQEqhzjTpVoT5fdYUZE"... 14 more characters,
startAt: 2024-12-07T07:22:18.898Z,
endAt: 2024-12-07T07:22:18.899Z
}
The retrieval failed.
The provider address %j cannot be converted to a URL: /dns/http.f02620.devtty.eu/https/http-path/%2Fipni-provider%2F12D3KooWNXvbyvLUUd1qQEqhzjTpVoT5fdYUZEv4RJSxZ3rDF2c7 Cannot parse "/dns/http.f02620.devtty.eu/https/http-path/%2Fipni-provider%2F12D3KooWNXvbyvLUUd1qQEqhzjTpVoT5fdYUZEv4RJSxZ3rDF2c7": unsupported protocol "https"
There was a problem hiding this comment.
Therefore I believe the lassie issue doesn't affect Spark data, as the check fails before talking to lassi
There was a problem hiding this comment.
I need to take a closer look.
There was a problem hiding this comment.
I ran manual check while substituting Frisbii PeerID with various Provider.ID values returned by https://cid.contact/cid/QmdmQXB2mzChmMeKY47C43LxUdg1NDJ5MWcKMKxDu7RgQm
I tested both Graphsync and HTTP retrievals.
I verified that both cases called Lassie in a way that does not trigger the issue with parsing http-path multiaddrs.
To explain: we tell Lassie the protocol & provider address to use for the retrieval, therefore Lassie skips the provider lookup (where the http-path parsing issue happens) and goes directly to the retrieval step.
Of course, if an SP advertises to IPNI retrievals via http-path multiaddr, then Spark checks will fail, but only for this SP. The measurements will indicate status code 702 (error code UNSUPPORTED_MULTIADDR_PROTO), which we will see in our data.
A bunch of Dependabot PRs are blocked from landing because of this test failing.
In this PR, I am fixing the test by changing the CID we are downloading, to avoid the Lassie bug described in filecoin-project/lassie#492.
I am also adding more diagnostics to simplify troubleshooting in the future.
Here is how to run this single test and get the additional info: