-
Notifications
You must be signed in to change notification settings - Fork 703
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Showing
2 changed files
with
180 additions
and
12 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,169 @@ | ||
| # Datastream Tool | ||
|
|
||
| This tool was designed to be used internally with the legacy zkEVM node, but most of its options are compatible with the CDK-Erigon node, as they share the same data stream format. The only non-compatible option is the data stream generation (generate-file). | ||
|
|
||
| The DataStream format spec can be found [here](https://github.com/0xPolygonHermez/cdk-erigon/blob/zkevm/docs/datastream/datastream.md). | ||
|
|
||
| ## Configuration | ||
|
|
||
| ``` | ||
| [Online] | ||
| URI = "localhost:6900" | ||
| StreamType = 1 | ||
| [Offline] | ||
| Port = 6901 | ||
| Filename = "datastream.bin" | ||
| Version = 3 | ||
| ChainID = 1440 | ||
| UpgradeEtrogBatchNumber = 0 | ||
| [StateDB] | ||
| User = "state_user" | ||
| Password = "state_password" | ||
| Name = "state_db" | ||
| Host = "localhost" | ||
| Port = "5432" | ||
| EnableLog = false | ||
| MaxConns = 200 | ||
| [MerkleTree] | ||
| URI = "localhost:50061" | ||
| MaxThreads = 20 | ||
| CacheFile = "merkle_tree_cache.json" | ||
| [Log] | ||
| Environment = "development" | ||
| Level = "error" | ||
| Outputs = ["stdout"] | ||
| ``` | ||
|
|
||
| - **Online Section**: | ||
| - It is used to connect to a remote data stream server. Currently only StreamType 1 exists. | ||
| - **Offline Section**: | ||
| - It is used to work with local datastream files. This section is also used during local data stream file generation. | ||
| - Port: The data stream library requires a port, this must be a free port in the machine where the tool is running. | ||
| - Filename: Full path and file name of the datastream to generate or query. | ||
| - Version: It will be added to the data stream file header. Current version is 3. It is only for information, but it is recommended to properly set it to the correct value. | ||
| - ChainID: L2 Chain ID | ||
| - UpgradeEtrogBatchMNumber: Only useful for chains that started before Fork ID Etrog. This value must be the batch number of the firtst etrog batch. The reason for this is that the first batch for an upgrade to Etrog contains a transaction generated by the rollup smart contract that must be handled in a special way as it has not been sequenced by the sequencer, but synchronized from L1. | ||
| - **StateDB Section**: | ||
| - Database access to the StateDB of the zkEVM node where the trusted state is stored. The information to generate the datastream will be extracted from here. | ||
| - **MerkleTree Section**: | ||
| - HashDB Service offered by the Prover image. It is needed to retrieve the IM State Root for Fork ID <= Etrog. If the network is only > Etrog, set MaxThreads to 0 to disable this feature. The CacheFile is useful to speed up generations, but will only help once it has been generated on the first run. | ||
| - **Log Section**: | ||
| - It is recommended to leave `Level` to `error` once the tool is up and running, but during configuration and until it is working, setting it to `debug` can be very helpful. | ||
|
|
||
| ## Options | ||
|
|
||
| To see avalible options type `make` once in the tool folder. | ||
|
|
||
| ``` | ||
| decode-batch Runs the tool to decode a given batch | ||
| decode-batch-offline Runs the offline tool to decode a given batch | ||
| decode-entry Runs the tool to decode a given entry number | ||
| decode-entry-offline Runs the offline tool to decode a given entry number | ||
| decode-l2block Runs the tool to decode a given L2 block | ||
| decode-l2block-offline Runs the offline tool to decode a given L2 block | ||
| dump-batch Runs the tool to dump a given batch to file | ||
| dump-batch-offline Runs the tool to dump a given batch to file offline | ||
| generate-file Runs the tool to populate the binary file | ||
| help Prints this help | ||
| truncate Runs the offline tool to truncate the stream file | ||
| ``` | ||
|
|
||
| All the decode options can work online, connecting to a node serving the stream, or offline, accessing the data stream files directly. | ||
|
|
||
| - **Decode Batch**: Decodes a Batch from a given number and shows all its data, l2blocks and transactions. | ||
| - **Decode Entry**: Decodes an entry and shows its content. Entry can be anything: bookmark, batch start, batch end, l2block, updateGER or transaction. | ||
| - **Decode L2Block**: Decodes a L2Block from a given number and shows all its data and transactions. | ||
| - **Truncate**: Truncates the file to a given entry number. Useful in case of unwinding the network. | ||
| - **Generate file**: Connects to StateDB and MerkleTree and generates the data stream files. | ||
| - **Dump batch**: Used to extract the binary data of a batch following the DS Spec. Useful during development of the integration with the Stateless Executor and Prover to generate test vectors. | ||
|
|
||
| ## Examples | ||
|
|
||
|
|
||
| ### Generate Data Stream files | ||
|
|
||
| `make generate-file` | ||
|
|
||
|
|
||
| ### Get contents of Batch 1 from the local files | ||
|
|
||
| `make decode-batch-offline 1` | ||
|
|
||
| ``` | ||
| Entry Type......: Batch Start | ||
| Entry Number....: 6 | ||
| Batch Number....: 1 | ||
| Batch Type......: BATCH_TYPE_INJECTED | ||
| Fork ID.........: 9 | ||
| Chain ID........: 6969 | ||
| Entry Type......: BookMark | ||
| Entry Number....: 7 | ||
| Type............: 2 (BOOKMARK_TYPE_L2_BLOCK) | ||
| Value...........: 1 | ||
| Entry Type......: L2 Block | ||
| Entry Number....: 8 | ||
| L2 Block Number.: 1 | ||
| Batch Number....: 1 | ||
| Timestamp.......: 1714380108 (2024-04-29 08:41:48 +0000 UTC) | ||
| Delta Timestamp.: 1714380108 | ||
| Min. Timestamp..: 0 | ||
| L1 Block Hash...: 0xdeaef97f8a5c6f056d08e162073d720c035a25adcaa2cd868124543ea54fb185 | ||
| L1 InfoTree Idx.: 0 | ||
| Block Hash......: 0xeeb2b1e810770dc1dcdd71a7ffa2a81ae77642d1e3c389919464100b3f70e366 | ||
| State Root......: 0xada6af5a8bf491712d5ba14c67283a7b516245cd571151c5ade13f82532a398d | ||
| Global Exit Root: 0xad3228b676f7d3cd4284a5443f17f1962b36e491b30a40b2405849e597ba5fb5 | ||
| Coinbase........: 0x5BD65BF6e084ECC10565EED59b838E82aBc28083 | ||
| Block Gas Limit.: 0 | ||
| Block Info Root.: 0x5dfdd92c4436374df99c4532ca7b8b1732faa10e9a6e31e0868bd4bfb6e8303d | ||
| Entry Type......: L2 Transaction | ||
| Entry Number....: 9 | ||
| L2 Block Number.: 1 | ||
| Index...........: 0 | ||
| Is Valid........: true | ||
| Data............: 0xf9010f80808401c9c38094ca127484cda2b723c4c03558b94749184d3cfa9880b8e4f811bff7000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000a40d5f56745a118d0906a34e69aec8c0db1cb8fa000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000c000000000000000000000000000000000000000000000000000000000000000001b8505ca1ab1e0845ca1ab1e | ||
| Effec. Gas Price: 255 | ||
| IM State Root...: 0xeb6b784c61931c8d188ce19298799000615de174e17850ae53feb496b4a5a6ca | ||
| Sender..........: 0x2CfbeDbE634712c0Cec2Cd929DcB8c23B0038F2A | ||
| Nonce...........: 0 | ||
| Entry Type......: Batch End | ||
| Entry Number....: 10 | ||
| Batch Number....: 1 | ||
| State Root......: 0xada6af5a8bf491712d5ba14c67283a7b516245cd571151c5ade13f82532a398d | ||
| Local Exit Root.: 0x0000000000000000000000000000000000000000000000000000000000000000 | ||
| ``` | ||
|
|
||
| ### Get content of L2Block 1 from an online Data Stream | ||
|
|
||
| `make decode-l2block 1` | ||
|
|
||
| ``` | ||
| Entry Type......: L2 Block | ||
| Entry Number....: 8 | ||
| L2 Block Number.: 1 | ||
| Batch Number....: 1 | ||
| Timestamp.......: 1714380108 (2024-04-29 08:41:48 +0000 UTC) | ||
| Delta Timestamp.: 1714380108 | ||
| Min. Timestamp..: 0 | ||
| L1 Block Hash...: 0xdeaef97f8a5c6f056d08e162073d720c035a25adcaa2cd868124543ea54fb185 | ||
| L1 InfoTree Idx.: 0 | ||
| Block Hash......: 0xeeb2b1e810770dc1dcdd71a7ffa2a81ae77642d1e3c389919464100b3f70e366 | ||
| State Root......: 0xada6af5a8bf491712d5ba14c67283a7b516245cd571151c5ade13f82532a398d | ||
| Global Exit Root: 0xad3228b676f7d3cd4284a5443f17f1962b36e491b30a40b2405849e597ba5fb5 | ||
| Coinbase........: 0x5BD65BF6e084ECC10565EED59b838E82aBc28083 | ||
| Block Gas Limit.: 0 | ||
| Block Info Root.: 0x5dfdd92c4436374df99c4532ca7b8b1732faa10e9a6e31e0868bd4bfb6e8303d | ||
| Entry Type......: L2 Transaction | ||
| Entry Number....: 9 | ||
| L2 Block Number.: 1 | ||
| Index...........: 0 | ||
| Is Valid........: true | ||
| Data............: 0xf9010f80808401c9c38094ca127484cda2b723c4c03558b94749184d3cfa9880b8e4f811bff7000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000a40d5f56745a118d0906a34e69aec8c0db1cb8fa000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000c000000000000000000000000000000000000000000000000000000000000000001b8505ca1ab1e0845ca1ab1e | ||
| Effec. Gas Price: 255 | ||
| IM State Root...: 0x0000000000000000000000000000000000000000000000000000000000000000 | ||
| Sender..........: 0x2CfbeDbE634712c0Cec2Cd929DcB8c23B0038F2A | ||
| Nonce...........: 0 | ||
| ``` |