Add sv2-serde-json #1323
Draft
Add sv2-serde-json #1323
+597
−0
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1323 +/- ##
==========================================
- Coverage 19.29% 19.07% -0.23%
==========================================
Files 164 166 +2
Lines 10852 10979 +127
==========================================
Hits 2094 2094
- Misses 8758 8885 +127
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
Bencher Report
|
| Branch | 2024-12-27-sv2-serde-json |
| Testbed | sv2 |
Click to view all benchmark results
| Benchmark | Estimated Cycles | Benchmark Result 1e3 x estimated cycles (Result Δ%) | Upper Boundary 1e3 x estimated cycles (Limit %) | Instructions | Benchmark Result instructions (Result Δ%) | Upper Boundary instructions (Limit %) | L1 Accesses | Benchmark Result accesses (Result Δ%) | Upper Boundary accesses (Limit %) | L2 Accesses | Benchmark Result accesses (Result Δ%) | Upper Boundary accesses (Limit %) | RAM Accesses | Benchmark Result accesses (Result Δ%) | Upper Boundary accesses (Limit %) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| client_sv2_handle_message_common | 📈 view plot 🚷 view threshold | 2.17 (+2.71%) | 2.23 (97.23%) | 📈 view plot 🚷 view threshold | 473.00 (-0.05%) | 488.92 (96.74%) | 📈 view plot 🚷 view threshold | 732.00 (-0.49%) | 757.93 (96.58%) | 📈 view plot 🚷 view threshold | 7.00 (+40.88%) | 11.89 (58.87%) | 📈 view plot 🚷 view threshold | 40.00 (+3.75%) | 41.61 (96.14%) |
| client_sv2_handle_message_mining | 📈 view plot 🚷 view threshold | 8.29 (+0.81%) | 8.39 (98.83%) | 📈 view plot 🚷 view threshold | 2,137.00 (-0.01%) | 2,140.01 (99.86%) | 📈 view plot 🚷 view threshold | 3,153.00 (-0.19%) | 3,169.00 (99.50%) | 📈 view plot 🚷 view threshold | 40.00 (+12.43%) | 41.95 (95.36%) | 📈 view plot 🚷 view threshold | 141.00 (+1.04%) | 144.11 (97.84%) |
| client_sv2_mining_message_submit_standard | 📈 view plot 🚷 view threshold | 6.35 (+0.87%) | 6.44 (98.60%) | 📈 view plot 🚷 view threshold | 1,750.00 (-0.02%) | 1,766.07 (99.09%) | 📈 view plot 🚷 view threshold | 2,548.00 (-0.15%) | 2,574.44 (98.97%) | 📈 view plot 🚷 view threshold | 19.00 (+11.30%) | 24.23 (78.42%) | 📈 view plot 🚷 view threshold | 106.00 (+1.34%) | 108.63 (97.58%) |
| client_sv2_mining_message_submit_standard_serialize | 📈 view plot 🚷 view threshold | 14.82 (+0.63%) | 14.93 (99.22%) | 📈 view plot 🚷 view threshold | 4,694.00 (-0.01%) | 4,710.07 (99.66%) | 📈 view plot 🚷 view threshold | 6,746.00 (-0.15%) | 6,785.70 (99.41%) | 📈 view plot 🚷 view threshold | 53.00 (+17.96%) | 61.00 (86.89%) | 📈 view plot 🚷 view threshold | 223.00 (+0.82%) | 226.32 (98.53%) |
| client_sv2_mining_message_submit_standard_serialize_deserialize | 📈 view plot 🚷 view threshold | 27.83 (+0.88%) | 28.01 (99.34%) | 📈 view plot 🚷 view threshold | 10,645.00 (+0.40%) | 10,688.15 (99.60%) | 📈 view plot 🚷 view threshold | 15,501.00 (+0.44%) | 15,576.61 (99.51%) | 📈 view plot 🚷 view threshold | 92.00 (+10.56%) | 98.06 (93.82%) | 📈 view plot 🚷 view threshold | 339.00 (+1.10%) | 342.52 (98.97%) |
| client_sv2_open_channel | 📈 view plot 🚷 view threshold | 4.49 (+2.26%) | 4.56 (98.60%) | 📈 view plot 🚷 view threshold | 1,461.00 (-0.02%) | 1,476.92 (98.92%) | 📈 view plot 🚷 view threshold | 2,153.00 (-0.34%) | 2,183.69 (98.59%) | 📈 view plot 🚷 view threshold | 13.00 (+60.77%) | 13.58 (95.73%) | 📈 view plot 🚷 view threshold | 65.00 (+3.74%) | 67.16 (96.78%) |
| client_sv2_open_channel_serialize | 📈 view plot 🚷 view threshold | 14.09 (+0.55%) | 14.19 (99.31%) | 📈 view plot 🚷 view threshold | 5,064.00 (-0.00%) | 5,079.92 (99.69%) | 📈 view plot 🚷 view threshold | 7,314.00 (-0.16%) | 7,351.48 (99.49%) | 📈 view plot 🚷 view threshold | 47.00 (+30.30%) | 47.44 (99.07%) | 📈 view plot 🚷 view threshold | 187.00 (+0.52%) | 190.94 (97.94%) |
| client_sv2_open_channel_serialize_deserialize | 📈 view plot 🚷 view threshold | 22.87 (+0.78%) | 23.01 (99.39%) | 📈 view plot 🚷 view threshold | 8,040.00 (+0.11%) | 8,054.11 (99.82%) | 📈 view plot 🚷 view threshold | 11,686.00 (+0.02%) | 11,711.31 (99.78%) | 📈 view plot 🚷 view threshold | 87.00 (+15.46%) | 88.84 (97.93%) | 📈 view plot 🚷 view threshold | 307.00 (+1.10%) | 311.19 (98.65%) |
| client_sv2_setup_connection | 📈 view plot 🚷 view threshold | 4.75 (+1.21%) | 4.79 (99.13%) | 📈 view plot 🚷 view threshold | 1,502.00 (-0.02%) | 1,517.92 (98.95%) | 📈 view plot 🚷 view threshold | 2,275.00 (-0.14%) | 2,299.53 (98.93%) | 📈 view plot 🚷 view threshold | 11.00 (+16.85%) | 15.42 (71.35%) | 📈 view plot 🚷 view threshold | 69.00 (+2.21%) | 70.06 (98.48%) |
| client_sv2_setup_connection_serialize | 📈 view plot 🚷 view threshold | 16.24 (+0.58%) | 16.31 (99.59%) | 📈 view plot 🚷 view threshold | 5,963.00 (-0.00%) | 5,978.92 (99.73%) | 📈 view plot 🚷 view threshold | 8,654.00 (-0.12%) | 8,691.59 (99.57%) | 📈 view plot 🚷 view threshold | 48.00 (+20.07%) | 54.05 (88.80%) | 📈 view plot 🚷 view threshold | 210.00 (+0.88%) | 211.96 (99.08%) |
| client_sv2_setup_connection_serialize_deserialize | 📈 view plot 🚷 view threshold | 35.77 (+0.52%) | 35.90 (99.63%) | 📈 view plot 🚷 view threshold | 14,888.00 (+0.16%) | 14,913.03 (99.83%) | 📈 view plot 🚷 view threshold | 21,870.00 (+0.14%) | 21,911.82 (99.81%) | 📈 view plot 🚷 view threshold | 106.00 (+13.65%) | 116.62 (90.90%) | 📈 view plot 🚷 view threshold | 382.00 (+0.68%) | 385.12 (99.19%) |
|
| Branch | 2024-12-27-sv2-serde-json |
| Testbed | sv1 |
Click to view all benchmark results
| Benchmark | Latency | Benchmark Result nanoseconds (ns) (Result Δ%) | Upper Boundary nanoseconds (ns) (Limit %) |
|---|---|---|---|
| client-submit-serialize | 📈 view plot 🚷 view threshold | 6,612.80 (+0.40%) | 6,911.94 (95.67%) |
| client-submit-serialize-deserialize | 📈 view plot 🚷 view threshold | 7,366.70 (-0.86%) | 7,829.15 (94.09%) |
| client-submit-serialize-deserialize-handle/client-submit-serialize-deserialize-handle | 📈 view plot 🚷 view threshold | 8,011.50 (-1.10%) | 9,252.32 (86.59%) |
| client-sv1-authorize-serialize-deserialize-handle/client-sv1-authorize-serialize-deserialize-handle | 📈 view plot 🚷 view threshold | 860.77 (-0.60%) | 937.29 (91.84%) |
| client-sv1-authorize-serialize-deserialize/client-sv1-authorize-serialize-deserialize | 📈 view plot 🚷 view threshold | 679.89 (+0.88%) | 715.57 (95.01%) |
| client-sv1-authorize-serialize/client-sv1-authorize-serialize | 📈 view plot 🚷 view threshold | 244.97 (-1.62%) | 269.63 (90.85%) |
| client-sv1-get-authorize/client-sv1-get-authorize | 📈 view plot 🚷 view threshold | 155.68 (-1.09%) | 166.11 (93.72%) |
| client-sv1-get-submit | 📈 view plot 🚷 view threshold | 6,255.80 (-1.60%) | 6,765.55 (92.47%) |
| client-sv1-get-subscribe/client-sv1-get-subscribe | 📈 view plot 🚷 view threshold | 309.18 (+9.16%) | 329.13 (93.94%) |
| client-sv1-subscribe-serialize-deserialize-handle/client-sv1-subscribe-serialize-deserialize-handle | 📈 view plot 🚷 view threshold | 710.76 (-2.18%) | 775.01 (91.71%) |
| client-sv1-subscribe-serialize-deserialize/client-sv1-subscribe-serialize-deserialize | 📈 view plot 🚷 view threshold | 588.26 (-0.20%) | 624.61 (94.18%) |
| client-sv1-subscribe-serialize/client-sv1-subscribe-serialize | 📈 view plot 🚷 view threshold | 205.31 (-0.47%) | 223.42 (91.90%) |
|
| Branch | 2024-12-27-sv2-serde-json |
| Testbed | sv1 |
Click to view all benchmark results
| Benchmark | Estimated Cycles | Benchmark Result 1e3 x estimated cycles (Result Δ%) | Upper Boundary 1e3 x estimated cycles (Limit %) | Instructions | Benchmark Result 1e3 x instructions (Result Δ%) | Upper Boundary 1e3 x instructions (Limit %) | L1 Accesses | Benchmark Result 1e3 x accesses (Result Δ%) | Upper Boundary 1e3 x accesses (Limit %) | L2 Accesses | Benchmark Result accesses (Result Δ%) | Upper Boundary accesses (Limit %) | RAM Accesses | Benchmark Result accesses (Result Δ%) | Upper Boundary accesses (Limit %) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| get_authorize | 📈 view plot 🚷 view threshold | 8.39 (-0.74%) | 8.67 (96.73%) | 📈 view plot 🚷 view threshold | 3.69 (-1.12%) | 3.86 (95.54%) | 📈 view plot 🚷 view threshold | 5.16 (-1.31%) | 5.45 (94.69%) | 📈 view plot 🚷 view threshold | 10.00 (+22.61%) | 15.63 (63.96%) | 📈 view plot 🚷 view threshold | 91.00 (-0.12%) | 96.13 (94.66%) |
| get_submit | 📈 view plot 🚷 view threshold | 95.19 (-0.18%) | 95.67 (99.51%) | 📈 view plot 🚷 view threshold | 59.35 (-0.12%) | 59.70 (99.40%) | 📈 view plot 🚷 view threshold | 85.22 (-0.13%) | 85.81 (99.31%) | 📈 view plot 🚷 view threshold | 48.00 (+8.26%) | 59.78 (80.29%) | 📈 view plot 🚷 view threshold | 278.00 (-0.83%) | 290.32 (95.75%) |
| get_subscribe | 📈 view plot 🚷 view threshold | 7.87 (-1.61%) | 8.24 (95.58%) | 📈 view plot 🚷 view threshold | 2.77 (-1.89%) | 2.94 (93.94%) | 📈 view plot 🚷 view threshold | 3.85 (-2.18%) | 4.14 (92.85%) | 📈 view plot 🚷 view threshold | 14.00 (+14.43%) | 20.29 (68.99%) | 📈 view plot 🚷 view threshold | 113.00 (-1.30%) | 117.83 (95.90%) |
| serialize_authorize | 📈 view plot 🚷 view threshold | 12.12 (-1.18%) | 12.51 (96.85%) | 📈 view plot 🚷 view threshold | 5.27 (-0.71%) | 5.43 (97.05%) | 📈 view plot 🚷 view threshold | 7.33 (-0.85%) | 7.60 (96.39%) | 📈 view plot 🚷 view threshold | 13.00 (+30.71%) | 18.49 (70.31%) | 📈 view plot 🚷 view threshold | 135.00 (-2.01%) | 142.47 (94.75%) |
| serialize_deserialize_authorize | 📈 view plot 🚷 view threshold | 24.67 (-0.17%) | 25.18 (97.97%) | 📈 view plot 🚷 view threshold | 9.84 (-0.27%) | 10.01 (98.24%) | 📈 view plot 🚷 view threshold | 13.88 (-0.33%) | 14.18 (97.88%) | 📈 view plot 🚷 view threshold | 38.00 (+6.48%) | 45.55 (83.43%) | 📈 view plot 🚷 view threshold | 303.00 (-0.07%) | 313.06 (96.79%) |
| serialize_deserialize_handle_authorize | 📈 view plot 🚷 view threshold | 30.19 (-0.46%) | 30.72 (98.25%) | 📈 view plot 🚷 view threshold | 12.02 (-0.35%) | 12.19 (98.57%) | 📈 view plot 🚷 view threshold | 17.00 (-0.43%) | 17.30 (98.26%) | 📈 view plot 🚷 view threshold | 62.00 (+11.35%) | 67.19 (92.28%) | 📈 view plot 🚷 view threshold | 368.00 (-0.75%) | 379.17 (97.05%) |
| serialize_deserialize_handle_submit | 📈 view plot 🚷 view threshold | 126.30 (-0.12%) | 126.78 (99.62%) | 📈 view plot 🚷 view threshold | 73.20 (-0.08%) | 73.53 (99.56%) | 📈 view plot 🚷 view threshold | 104.91 (-0.10%) | 105.49 (99.45%) | 📈 view plot 🚷 view threshold | 114.00 (+7.43%) | 125.20 (91.05%) | 📈 view plot 🚷 view threshold | 595.00 (-0.39%) | 608.20 (97.83%) |
| serialize_deserialize_handle_subscribe | 📈 view plot 🚷 view threshold | 27.78 (-0.45%) | 28.41 (97.80%) | 📈 view plot 🚷 view threshold | 9.58 (-0.54%) | 9.76 (98.21%) | 📈 view plot 🚷 view threshold | 13.53 (-0.67%) | 13.84 (97.81%) | 📈 view plot 🚷 view threshold | 71.00 (+10.47%) | 77.44 (91.68%) | 📈 view plot 🚷 view threshold | 397.00 (-0.49%) | 410.31 (96.76%) |
| serialize_deserialize_submit | 📈 view plot 🚷 view threshold | 115.18 (-0.05%) | 115.71 (99.54%) | 📈 view plot 🚷 view threshold | 68.06 (-0.01%) | 68.41 (99.49%) | 📈 view plot 🚷 view threshold | 97.65 (-0.02%) | 98.27 (99.37%) | 📈 view plot 🚷 view threshold | 69.00 (+7.02%) | 85.69 (80.52%) | 📈 view plot 🚷 view threshold | 491.00 (-0.36%) | 501.92 (97.82%) |
| serialize_deserialize_subscribe | 📈 view plot 🚷 view threshold | 23.21 (-0.48%) | 23.83 (97.40%) | 📈 view plot 🚷 view threshold | 8.14 (-0.59%) | 8.31 (97.93%) | 📈 view plot 🚷 view threshold | 11.45 (-0.70%) | 11.74 (97.50%) | 📈 view plot 🚷 view threshold | 42.00 (+8.83%) | 50.38 (83.36%) | 📈 view plot 🚷 view threshold | 330.00 (-0.41%) | 342.64 (96.31%) |
| serialize_submit | 📈 view plot 🚷 view threshold | 99.58 (-0.22%) | 100.13 (99.45%) | 📈 view plot 🚷 view threshold | 61.41 (-0.10%) | 61.73 (99.48%) | 📈 view plot 🚷 view threshold | 88.08 (-0.11%) | 88.64 (99.37%) | 📈 view plot 🚷 view threshold | 52.00 (+9.46%) | 65.88 (78.93%) | 📈 view plot 🚷 view threshold | 321.00 (-1.24%) | 335.69 (95.62%) |
| serialize_subscribe | 📈 view plot 🚷 view threshold | 11.32 (-0.73%) | 11.60 (97.54%) | 📈 view plot 🚷 view threshold | 4.12 (-1.18%) | 4.28 (96.13%) | 📈 view plot 🚷 view threshold | 5.71 (-1.45%) | 6.00 (95.17%) | 📈 view plot 🚷 view threshold | 16.00 (+18.24%) | 23.27 (68.75%) | 📈 view plot 🚷 view threshold | 158.00 (-0.21%) | 163.64 (96.56%) |
|
What crates will depend on this new |
This change would only impact |
For reference only, I did a Stratum v1 no_std client implementation (https://github.com/Foundation-Devices/foundation-rs/tree/main/stratum-v1) and the json parsing was a nightmare... |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces a sv2-serde-json as a utility module. The goal is to eliminate the dependency on
serdefrom the message generator, ultimately phasing outbinary_sv2_serde. Currently,sv2_serde_jsonhandles serialization and deserialization by converting data into aValueconstruct, which must then be mapped onto Rust's structs and enums.By adopting this approach, we aim to remove all dependencies on
serde, paving the way for the eventual deprecation ofbinary_sv2_serdeentirely. Alternatively, if we decide not to proceed with this new utility, we will extend the existingbinary_sv2to support JSON serialization and deserialization directly.This PR is marked as a draft while we finalize the chosen approach. In the meantime, this branch will be maintained for developing JSON features using the external utility, rather than extending
binary_sv2.