From a9e920dbd4b2f80ac32d7b100bfdc7152d6c78a8 Mon Sep 17 00:00:00 2001 From: Kolby Moroz Liebl <31669092+KolbyML@users.noreply.github.com> Date: Mon, 21 Oct 2024 16:38:37 -0600 Subject: [PATCH] Define ping/pong payload extensions --- beacon-chain/beacon-network.md | 7 +- .../canonical-transaction-index-network.md | 7 +- history/history-network.md | 6 +- ping-payload-extensions/extensions.md | 14 ++++ .../extensions/template.md | 25 +++++++ ping-payload-extensions/extensions/type-1.md | 24 +++++++ ping-payload-extensions/extensions/type-10.md | 34 ++++++++++ ping-payload-extensions/extensions/type-2.md | 14 ++++ ping-payload-extensions/extensions/type-3.md | 14 ++++ ping-payload-extensions/extensions/type-4.md | 14 ++++ ping-payload-extensions/extensions/type-5.md | 14 ++++ .../ping-custom-payload-extensions.md | 67 +++++++++++++++++++ state/state-network.md | 7 +- transaction-gossip/transaction-gossip.md | 14 +--- 14 files changed, 225 insertions(+), 36 deletions(-) create mode 100644 ping-payload-extensions/extensions.md create mode 100644 ping-payload-extensions/extensions/template.md create mode 100644 ping-payload-extensions/extensions/type-1.md create mode 100644 ping-payload-extensions/extensions/type-10.md create mode 100644 ping-payload-extensions/extensions/type-2.md create mode 100644 ping-payload-extensions/extensions/type-3.md create mode 100644 ping-payload-extensions/extensions/type-4.md create mode 100644 ping-payload-extensions/extensions/type-5.md create mode 100644 ping-payload-extensions/ping-custom-payload-extensions.md diff --git a/beacon-chain/beacon-network.md b/beacon-chain/beacon-network.md index a2b17f4..a4a4c80 100644 --- a/beacon-chain/beacon-network.md +++ b/beacon-chain/beacon-network.md @@ -76,12 +76,7 @@ The beacon chain network supports the following protocol messages: #### `Ping.custom_data` & `Pong.custom_data` -In the beacon chain network the `custom_payload` field of the `Ping` and `Pong` messages is the serialization of an SSZ Container specified as `custom_data`: - -```python -custom_data = Container(data_radius: uint256) -custom_payload = serialize(custom_data) -``` +In the beacon chain network the `custom_payload` field of the `Ping` and `Pong` messages is the serialization of an SSZ Container specified as [Type 3 Ping Custom Payload Extension](../ping-payload-extensions/extensions/type-3.md) ### Routing Table diff --git a/canonical-transaction-index/canonical-transaction-index-network.md b/canonical-transaction-index/canonical-transaction-index-network.md index 3eea4b3..6f23510 100644 --- a/canonical-transaction-index/canonical-transaction-index-network.md +++ b/canonical-transaction-index/canonical-transaction-index-network.md @@ -56,12 +56,7 @@ The canonical transaction index network supports the following protocol messages #### `Ping.custom_data` & `Pong.custom_data` -In the canonical transaction index network the `custom_payload` field of the `Ping` and `Pong` messages is the serialization of an SSZ Container specified as `custom_data`: - -``` -custom_data = Container(data_radius: uint256) -custom_payload = SSZ.serialize(custom_data) -``` +In the canonical transaction index network the `custom_payload` field of the `Ping` and `Pong` messages is the serialization of an SSZ Container specified as [Type 4 Ping Custom Payload Extension](../ping-payload-extensions/extensions/type-4.md) ### Routing Table diff --git a/history/history-network.md b/history/history-network.md index 375b4d2..98841fb 100644 --- a/history/history-network.md +++ b/history/history-network.md @@ -61,12 +61,8 @@ The history network supports the following protocol messages: #### `Ping.custom_data` & `Pong.custom_data` -In the history network the `custom_payload` field of the `Ping` and `Pong` messages is the serialization of an SSZ Container specified as `custom_data`: +In the history network the `custom_payload` field of the `Ping` and `Pong` messages is the serialization of an SSZ Container specified as [Type 1 Ping Custom Payload Extension](../ping-payload-extensions/extensions/type-1.md) -```python -custom_data = Container(data_radius: uint256) -custom_payload = SSZ.serialize(custom_data) -``` ### Routing Table diff --git a/ping-payload-extensions/extensions.md b/ping-payload-extensions/extensions.md new file mode 100644 index 0000000..69ab4bf --- /dev/null +++ b/ping-payload-extensions/extensions.md @@ -0,0 +1,14 @@ +# This document lists defined extensions +Some extensions are network specific some are generic + +This is a list and short description of all the extensions + + +| Type number | Name | Supported sub-networks | Short Description | Is this call Required to Implement | +|---|---|---|---|---| +| [1](extensions/type-1.md) | History Distance | History | Allows you to get the radius and other nodes metadata | Yes | +| [2](extensions/type-2.md) | State Distance | State | Allows you to get the state nodes radius | Yes | +| [3](extensions/type-3.md) | Beacon Distance | Beacon | Allows you to get the beacon nodes radius | Yes | +| [4](extensions/type-4.md) | Portal Canonical Transaction Distance | Portal Canonical Transaction | Allows you to get the Portal Canonical Transaction nodes radius | Yes | +| [5](extensions/type-5.md) | Transaction Gossip Distance | Transaction Gossip | Allows you to get the Transaction Gossip nodes radius | Yes | +| [10](extensions/type-10.md) | Client Info | All | It will return you something like `trin/0.1.1-2b00d730/linux-x86_64/rustc1.81.0` | Yes | diff --git a/ping-payload-extensions/extensions/template.md b/ping-payload-extensions/extensions/template.md new file mode 100644 index 0000000..e69f07b --- /dev/null +++ b/ping-payload-extensions/extensions/template.md @@ -0,0 +1,25 @@ +# [Title] + +[Description] + +## version [Version Number] + +[Version Description] + +Ping payload +```python +[Container Name] = Container( + type: [Type Number], + version: [Type Version], + payload: Container([Version Description]) +) +``` + +Pong payload +```python +[Container Name] = Container( + type: [Type Number], + version: [Type Version], + payload: Container([Version Description]) +) +``` diff --git a/ping-payload-extensions/extensions/type-1.md b/ping-payload-extensions/extensions/type-1.md new file mode 100644 index 0000000..9a2f09a --- /dev/null +++ b/ping-payload-extensions/extensions/type-1.md @@ -0,0 +1,24 @@ +# Portal History Network Distance + +This payload is only supposed to be used for the history network + +## version 1 + +Ping and Pong Payload +```python +HistoryNetworkDistanceV1 = Container( + type: 1, + version: 1, + payload: Container(data_radius: uint256) +) +``` + +## version 2 +Ping and Pong Payload +```python +HistoryNetworkDistanceV2 = Container( + type: 1, + version: 2, + payload: Container(data_radius: uint256, ephemeral_header_count: uint16) +) +``` diff --git a/ping-payload-extensions/extensions/type-10.md b/ping-payload-extensions/extensions/type-10.md new file mode 100644 index 0000000..8839b11 --- /dev/null +++ b/ping-payload-extensions/extensions/type-10.md @@ -0,0 +1,34 @@ +# Get Client info + +This payload is only supposed to be usable from all Networks. This payload allows you to figure out peer's client info. Which will be useful for censuses. + +## version 1 + +### Type Specifications + +Client info are ASCII hex encoded strings. + +Client info strings consist of 4 parts +- client name (e.x. `trin`,`fluffy`) +- client version + short commit (e.x. `0.1.1-2b00d730`) +- operating system + cpu archtecture (e.x. `linux-x86_64`) +- programming language + language version (e.x. `rustc1.81.0`) + +Example +- String: `trin/0.1.1-2b00d730/linux-x86_64/rustc1.81.0` +- Hex encoding: `0x7472696E2F302E312E312D32623030643733302F6C696E75782D7838365F36342F7275737463312E38312E30` + + +Max length of a client info we should accept +- MAX_CLIENT_INFO_BYTE_LENGTH = 200 + + +Ping and Pong payload +```python + +HistoryNetworkDistanceV1 = Container( + type: 10, + version: 1, + payload: Container(client_info: ByteList[MAX_CLIENT_INFO_BYTE_LENGTH]) +) +``` diff --git a/ping-payload-extensions/extensions/type-2.md b/ping-payload-extensions/extensions/type-2.md new file mode 100644 index 0000000..4b88039 --- /dev/null +++ b/ping-payload-extensions/extensions/type-2.md @@ -0,0 +1,14 @@ +# Portal State Network Distance + +This payload is only supposed to be used for the state network + +## version 1 + +Ping and Pong payload +```python +StateNetworkDistanceV1 = Container( + type: 2, + version: 1, + payload: Container(data_radius: uint256) +) +``` diff --git a/ping-payload-extensions/extensions/type-3.md b/ping-payload-extensions/extensions/type-3.md new file mode 100644 index 0000000..f21ddc7 --- /dev/null +++ b/ping-payload-extensions/extensions/type-3.md @@ -0,0 +1,14 @@ +# Portal Beacon Network Distance + +This payload is only supposed to be used for the beacon network + +## version 1 + +Ping and Pong payload +```python +BeaconNetworkDistanceV1 = Container( + type: 3, + version: 1, + payload: Container(data_radius: uint256) +) +``` diff --git a/ping-payload-extensions/extensions/type-4.md b/ping-payload-extensions/extensions/type-4.md new file mode 100644 index 0000000..ece0124 --- /dev/null +++ b/ping-payload-extensions/extensions/type-4.md @@ -0,0 +1,14 @@ +# Portal Canonical Transaction Index Network Distance + +This payload is only supposed to be used for the Canonical Transaction Index Network + +## version 1 + +Ping and Pong payload +```python +CanonicalTransactionIndexNetworkDistanceV1 = Container( + type: 4, + version: 1, + payload: Container(data_radius: uint256) +) +``` diff --git a/ping-payload-extensions/extensions/type-5.md b/ping-payload-extensions/extensions/type-5.md new file mode 100644 index 0000000..b74e39d --- /dev/null +++ b/ping-payload-extensions/extensions/type-5.md @@ -0,0 +1,14 @@ +# Portal Transaction Gossip Network Distance + +This payload is only supposed to be used for the Transaction Gossip network + +## version 1 + +Ping and Pong payload +```python +Transaction GossipNetworkDistanceV1 = Container( + type: 5, + version: 1, + payload: Container(data_radius: uint256) +) +``` diff --git a/ping-payload-extensions/ping-custom-payload-extensions.md b/ping-payload-extensions/ping-custom-payload-extensions.md new file mode 100644 index 0000000..8017197 --- /dev/null +++ b/ping-payload-extensions/ping-custom-payload-extensions.md @@ -0,0 +1,67 @@ +# Ping Custom Payload Extensions + +## The problem + +Over time we will need to change what is sent over ping/pong `custom_payload`'s to adjust for what is needed in the future. +Currently the only way to update a `custom_payload` is through breaking changes which break protocol compatible between Portal implementations. +As we get users, deploying breaking changes to mainnet will no longer be an option as we are aiming for a 100% uptime. + +## The Solution + +Ping Custom Payload Extensions. A versioned type prefixed format where we can upgrade endpoints on a new version well giving a window for all Portal implementations to update before deprecating the older version. This will allow clients to implement new functionality without breaking compatibility with the standard specification. + +# Type's + +There are 4_294_967_296 unique type ids. + +Types 0-10_000 and 4_294_957_295-4_294_967_296 are reserved for for future upgrades. + +The rest are first come first serve, but they should still be defined in this repo to avoid overlaps. + + +## Requirements + +All payloads used in the Ping `custom_payload` MUST follow the `Ping Custom Payload Extensions` format. + +## Custom Payload Extensions Format + +- **type**: what payload type is it +- **verison**: what version of the type it is +- **payload**: a ssz container which contents are specified by the type and version field + + +```python +CustomPayloadExtensionsFormat = Container( + type: Bytes4, + version: Bytes4, + payload: Container(inner payload is defined by type and version) +) +``` + +## Ping vs Pong +The relationship between Ping and Pong message will be determined by the requirements of the type/version. + +Currently type 1,2,3 are mirrored formats, but there is room for a Ping `custom_payload` to specify details about what it wants to request, then pong handles it. + + +### Error responses +If the ping receiver can't handle the ping for any reason the pong should return an error payload + +Pong payload +```python + +# Max ASCII hex encoded strings length +MAX_ERROR_BYTE_LENGTH = 300 + +CustomPayloadExtensionsFormat = Container( + type: 4_294_967_295, + version: 1, + payload: Container(error_code: Bytes4, message: ByteList[MAX_ERROR_BYTE_LENGTH]) +) +``` + +### Error Code's + +- 0: Extension not supported +- 1: Requested data not found +- 2: System error diff --git a/state/state-network.md b/state/state-network.md index 410ca85..85ebee7 100644 --- a/state/state-network.md +++ b/state/state-network.md @@ -56,12 +56,7 @@ The execution state network supports the following protocol messages: #### `Ping.custom_data` & `Pong.custom_data` -In the execution state network the `custom_payload` field of the `Ping` and `Pong` messages is the serialization of an SSZ Container specified as `custom_data`: - -``` -custom_data = Container(data_radius: uint256) -custom_payload = SSZ.serialize(custom_data) -``` +In the execution state network the `custom_payload` field of the `Ping` and `Pong` messages is the serialization of an SSZ Container specified as [Type 2 Ping Custom Payload Extension](../ping-payload-extensions/extensions/type-2.md) #### POKE Mechanism diff --git a/transaction-gossip/transaction-gossip.md b/transaction-gossip/transaction-gossip.md index 084bf9e..83fb811 100644 --- a/transaction-gossip/transaction-gossip.md +++ b/transaction-gossip/transaction-gossip.md @@ -26,19 +26,7 @@ The network uses the PING/PONG/FINDNODES/FOUNDNODES/OFFER/ACCEPT messages from t The network uses the standard XOR distance function. -### PING payload - -``` -Ping.custom_payload := ssz_serialize(custom_data) -custom_data := Container(transaction_radius: uint256) -``` - -### PONG payload - -``` -Pong.custom_payload := ssz_serialize(custom_data) -custom_data := Container(transaction_radius: uint256) -``` +In the execution state network the `custom_payload` field of the `Ping` and `Pong` messages is the serialization of an SSZ Container specified as [Type 5 Ping Custom Payload Extension](../ping-payload-extensions/extensions/type-5.md) ## Content Keys