chore!: use URL-safe alphabet for base64 encoding (ENG-913) #1651
Conversation
Fraser999
requested review from
a team,
joroshiba,
SuperFluffy and
noot
as code owners
github-actions
bot
added
conductor
Cargo.toml
Outdated
There was a problem hiding this comment.
pbjson-types is listed as a workspace dependency here. Instead of making our fork of pbjson a direct dependency, should we make it a patch pointing to our branch?
There was a problem hiding this comment.
I don't think a [patch] section is ideal, since (at least to me) it implies a temporary workaround, whereas this change is likely to be fairly long-standing. More importantly, patches are ignored if they're in deps' manifests, so if we use a patch here, it would be ignored by a crate outside the monorepo depending on e.g. astria-core.
If you were also meaning we should specify our fork of the pbjson repo for all the pbjson crates, I'd have less issue with that, assuming we don't do it via a patch :)
| @@ -9,7 +9,7 @@ repository = "https://github.com/astriaorg/astria" | |||
| homepage = "https://astria.org" | |||
|
|
|||
| [dependencies] | |||
| base64 = { workspace = true } | |||
| base64 = "0.22.1" | |||
There was a problem hiding this comment.
I guess the base64 encoded blob data we feed into the blob parser command is not controlled by us?
There was a problem hiding this comment.
Yes, correct - it's from a block explorer.
|
Closing pending discussion around the scope of changes for this update. |
Summary
This changes all base64-encoding to use the URL-safe alphabet.
Background
We want our base64-encoded data to be URL-safe, and this was not previously the case.
Changes
astria-core-utilswith a single public modulebase64. This module provides a Display/Debug helper for formatting byte strings while avoiding allocations, and simpleencodeanddecodefunctions. Further functions can be provided in the future if required (e.g. encoding to a buffer).base64crate and oftelemetry::display::base64have been replaced by using the new functions/type...astria-sequencer-utils, where the input to the blob-parser will be standard base64. Hence, in that crate only, we still read the input data as standard base64, but all outputs are made using the URL-safe encoding fromastria-core-utilspbjsondependency was changed to use a fork, which has the effect of making the generated serde impls of the protobuf types also use the URL-safe alphabet when encoding. (pbjsonalready supports decoding from URL-safe-encoded data).Testing
Unit tests were added.
Breaking Changelist
Related Issues
Closes #1649.