diff --git a/docs/tutorials/join-dal-baker.md b/docs/tutorials/join-dal-baker.md deleted file mode 100644 index 5894d3070..000000000 --- a/docs/tutorials/join-dal-baker.md +++ /dev/null @@ -1,26 +0,0 @@ -# How to join the Tezos DAL as a baker, in 5 steps - -Tezos' [Data-Availability Layer](https://tezos.gitlab.io/shell/dal.html) (DAL for short), is an experimental feature which is, at the time of writing, not available on Tezos Mainnet but planned to be proposed in a protocol amendment in the near future. - -The DAL is a key component for the scalability of Tezos. In a nutshell, the DAL is about increasing the data bandwidth available for Tezos Smart Rollups thanks to a new parallel P2P network on which rollups could connect to fetch inputs without compromising their security. - -In order for the DAL to be as secure as the Tezos Layer 1 itself, bakers need to play a very important role in it. Currently, bakers on the L1 network are not only responsible for producing blocks but also for attesting that blocks are published on the L1 network. They are rewarded for this contribution through protocol incentives. Similarly, the role of bakers in the DAL would be to attest the publication of data on the DAL's P2P network. They would in turn be rewarded for this through (potentially different) protocol incentives. - -Given that setting up a new P2P network with several hundred active participants may take some time, the first proposed version of the DAL for Tezos Mainnet will not provide any participation incentives. This will let plenty of time for bakers to join the DAL network without risking any reward loss, ensuring a smooth transition. - -This incentive-free version of the DAL is already available on the Weeklynet test network. In this tutorial you will learn how to join Weeklynet as a baker and attest the publication of data on the DAL network. - -:::warning -This tutorial uses a very simple setup running all required daemons on the same machine. In a production environment, we advise against running a DAL attester node under the same IP address than a baker's node because the DAL node may leak the IP address and ease DOS attacks on the baker. See also [the DAL documentation page on baking](https://tezos.gitlab.io/shell/dal_bakers.html). -::: - -:::warning -The UX of the DAL components will be subject to changes with the feedback from the testers following this tutorial, so this tutorial will be updated accordingly. Feel free to file issues if it's not up-to-date. -::: - -- [Step 1: get a Weeklynet-compatible Octez version](./join-dal-baker/get-octez) -- [Step 2: run an Octez node on Weeklynet](./join-dal-baker/run-node) -- [Step 3: setting up a baker account on Weeklynet](./join-dal-baker/prepare-account) -- [Step 4: run an Octez DAL node on Weeklynet](./join-dal-baker/run-dal-node) -- [Step 5: run an Octez baking daemon on Weeklynet](./join-dal-baker/run-baker) -- [Conclusion](./join-dal-baker/conclusion) diff --git a/docs/tutorials/join-dal-baker.mdx b/docs/tutorials/join-dal-baker.mdx new file mode 100644 index 000000000..57e6918fd --- /dev/null +++ b/docs/tutorials/join-dal-baker.mdx @@ -0,0 +1,52 @@ +--- +title: Join the DAL as a baker, in 5 steps +authors: Tezos core developers +last_update: + date: 24 January 2024 +--- + +import LucidDiagram from '@site/src/components/LucidDiagram'; + +The Tezos data availability layer (DAL) is a key component for the scalability of Tezos. +In a nutshell, the DAL increases the data bandwidth available for Tezos Smart Rollups by providing a peer-to-peer network that they can use to fetch data without compromising security. + +:::note Experimental +The data availability layer is an experimental feature that is not yet available on Tezos Mainnet. +The way the DAL works may change significantly before it is generally available. +::: + +Just like layer 1, Tezos bakers ensure the security of the DAL. +Bakers not only produce blocks but also attest that other bakers' blocks are valid and properly published on layer 1. +In the same way, bakers attest that data published to the DAL is available. +In the current implementation of the DAL, bakers do not receive extra incentives for attesting DAL data, but they might in the future. +For now, bakers can join the DAL without risking any reward loss, ensuring a smooth transition. + +This incentive-free version of the DAL is currently available on the Weeklynet test network. +In this tutorial you learn how to join Weeklynet as a baker and attest the publication of data on the DAL network. + +For an introduction to how the DAL works, see the tutorial [Implement a file archive with the DAL and a Smart Rollup](./build-files-archive-with-dal). + +## Tutorial diagram + +In this tutorial, you set up the Octez client and several Octez daemons, including a layer 1 node, a baker, and a DAL baking node. +The following diagram shows these daemons with a blue background: + + + +:::warning +This tutorial uses a very simple setup running all required daemons on the same machine. In a production environment, we advise against running a DAL attester node under the same IP address than a baker's node because the DAL node may leak the IP address and ease DOS attacks on the baker. See also [the DAL documentation page on baking](https://tezos.gitlab.io/shell/dal_bakers.html). +::: + +:::warning +The UX of the DAL components will be subject to changes with the feedback from the testers following this tutorial, so this tutorial will be updated accordingly. Feel free to file issues if it's not up-to-date. +::: + +For more information about the DAL, see [Data-Availability Layer](https://tezos.gitlab.io/shell/dal.html) in the Octez documentation. +{/* TODO link to page on DAL when it's available: https://github.com/trilitech/tezos-developer-docs/pull/270 */} + +- [Step 1: Get a Weeklynet-compatible Octez version](./join-dal-baker/get-octez) +- [Step 2: Run an Octez node on Weeklynet](./join-dal-baker/run-node) +- [Step 3: Set up a baker account on Weeklynet](./join-dal-baker/prepare-account) +- [Step 4: Run an Octez DAL node on Weeklynet](./join-dal-baker/run-dal-node) +- [Step 5: Run an Octez baking daemon on Weeklynet](./join-dal-baker/run-baker) +- [Conclusion](./join-dal-baker/conclusion) diff --git a/docs/tutorials/join-dal-baker/conclusion.md b/docs/tutorials/join-dal-baker/conclusion.md index 58c297504..afb66d3ac 100644 --- a/docs/tutorials/join-dal-baker/conclusion.md +++ b/docs/tutorials/join-dal-baker/conclusion.md @@ -1,3 +1,8 @@ -# Conclusion +--- +title: "Conclusion" +authors: Tezos core developers +last_update: + date: 23 January 2024 +--- In this tutorial we have gone through all the steps needed to participate in the Weeklynet test network as a baker and DAL attester. We could further improve the setup by defining system services so that the daemons are automatically launched when the machine starts or when the network restarts on Wednesday. We could also plug a monitoring solution such as the Prometheus + Grafana combo; a Grafana dashboard template for DAL nodes is available in Grafazos. The interactions between our baker and the Weeklynet chain can be observed on the Explorus block explorer which is aware of the DAL and can in particular display which DAL slots are being used at each level. diff --git a/docs/tutorials/join-dal-baker/get-octez.md b/docs/tutorials/join-dal-baker/get-octez.md index a0f2d991a..3b8a1bcc6 100644 --- a/docs/tutorials/join-dal-baker/get-octez.md +++ b/docs/tutorials/join-dal-baker/get-octez.md @@ -1,13 +1,44 @@ -# Step 1: Get a Weeklynet-compatible Octez version +--- +title: "Step 1: Get a Weeklynet-compatible Octez version" +authors: Tezos core developers +last_update: + date: 23 January 2024 +--- -The Weeklynet test network is restarted once every Wednesday at 0h UTC, and for most of its lifetime (from level 512) it runs a development version of the Tezos protocol, called Alpha, which is not part of any released version of Octez. For this reason, baking on Weeklynet requires to run Octez either with Docker using a specific Docker image, or by building it from source using a specific git commit. +The Weeklynet test network restarts every Wednesday at 0h UTC, and for most of its lifetime (from level 512) it runs a development version of the Tezos protocol, called Alpha, which is not part of any released version of Octez. +To work with Weeklynet, you must use the exact same version of the Octez suite that Weeklynet is using. -To get this specific Docker image, or the hash of this specific commit, see https://teztnets.com/weeklynet-about. This page also contains the proper `octez-node config init` incantation to configure the Octez node with the current network parameters of Weeklynet, the URL of a public RPC endpoint, and a link to a faucet distributing free testnet tez. +The easiest way to do this is to use the Docker image that is generated each time Weeklynet is reset and recreated. +As another option, you can build the specific version of the Octez suite locally. +For instructions, see the Weeklynet page at https://teztnets.com/weeklynet-about. -For example, the commands to start a Docker image and configure the Octez node for the Weeklynet launched on January 17 2024, the instructions were: +To set up an environment and account in a Docker container, follow these steps: -``` -docker run -it --entrypoint=/bin/sh tezos/tezos:master_7f3bfc90_20240116181914 +1. From the [Weeklynet](https://teztnets.com/weeklynet-about) page, find the Docker command to create a container from the correct Docker image. +For example, the command to start a Docker image for the Weeklynet launched on January 17 2024 was: -octez-node config init --network https://teztnets.com/weeklynet-2024-01-17 -``` + ```bash + docker run -it --entrypoint=/bin/sh tezos/tezos:master_7f3bfc90_20240116181914 + ``` + + The image tag in this command changes each time the network is reset. + +1. In the container, initialize the Octez node with the command on the Weeklynet page, such as this example: + + ```bash + octez-node config init --network https://teztnets.com/weeklynet-2024-01-17 + ``` + + The specific command is on the Weeklynet page at https://teztnets.com/weeklynet-about. + +1. Copy the URL of the public RPC endpoint for Weeklynet, such as `https://rpc.weeklynet-2024-01-17.teztnets.com`. +This endpoint also changes each time the network is reset. + +1. Initialize the Octez client with that endpoint, as in this example: + + ```bash + octez-client -E https://rpc.weeklynet-2024-01-17.teztnets.com config init + ``` + +Now you have the Octez client and node configured to work with Weeklynet. +The next step is to start an Octez node; continue to [Step 2: Run an Octez node on Weeklynet](./run-node). diff --git a/docs/tutorials/join-dal-baker/prepare-account.md b/docs/tutorials/join-dal-baker/prepare-account.md index 064fbd00e..0dbe465f2 100644 --- a/docs/tutorials/join-dal-baker/prepare-account.md +++ b/docs/tutorials/join-dal-baker/prepare-account.md @@ -1,57 +1,65 @@ -# Step 3: Set up a baker account on Weeklynet +--- +title: "Step 3: Set up a baker account on Weeklynet" +authors: Tezos core developers +last_update: + date: 23 January 2024 +--- -Our baker needs an implicit account consisting of a pair of keys and an address. The simplest way to get them is to ask the Octez client to randomly generate them and associate them to the `my_baker` alias: +Our baker needs a user account consisting of a pair of keys and an address. +The simplest way to get an account that works with Weeklynet is to use the Octez client to randomly generate them and associate them to the `my_baker` alias: -``` +```bash octez-client gen keys my_baker ``` The address of the generated account can be obtained with the following command: -``` +```bash octez-client show address my_baker ``` Let's record this address in a shell variable, this will be useful for the some commands which cannot get addresses by their octez-client aliases. -``` +```bash MY_BAKER="$(octez-client show address my_baker | head -n 1 | cut -d ' ' -f 2)" ``` At this point, the balance of the `my_baker` account is still empty as can be seen with the following command: -``` -octez-client --endpoint "$ENDPOINT" get balance for my_baker +```bash +octez-client get balance for my_baker ``` -In order to get some consensus and DAL rights, we need to put some tez on the account. Fortunately, getting free testnet tez is easy thanks to the testnet faucet. To use it, we need to enter the generated address in the Weeklynet faucet linked from https://teztnets.com/weeklynet-about. We need at least 6k tez for running a baker but the more tez we have the more rights we will get and the shorter we will have to wait to produce blocks and attestations; that being said, baking with too much stake would prevent us from leaving the network without disturbing or even halting it so to avoid breaking the network for all other testers let's not be too greedy. 50k tez should be enough to get enough rights to easily check if our baker behaves as expected while not much disturbing the network when our baker will stop operating. +In order to get some consensus and DAL rights, we need to put some tez in the account. Fortunately, getting free testnet tez is easy thanks to the testnet faucet. To use it, we need to enter the generated address in the Weeklynet faucet linked from https://teztnets.com/weeklynet-about. We need at least 6k tez for running a baker but the more tez we have the more rights we will get and the shorter we will have to wait to produce blocks and attestations. That being said, baking with too much stake prevents us from leaving the network without disturbing or even halting it so to avoid breaking the network for all other testers let's not be too greedy. 50k tez is enough to get enough rights to easily check if our baker behaves as expected while not disturbing the network too much when our baker stops operating. -Once the tez are obtained from the faucet, we can check with the same `get balance` command that they have been received: +We can verify that the faucet sent the tez to the account with the same `get balance` command: -``` -octez-client --endpoint "$ENDPOINT" get balance for my_baker +```bash +octez-client get balance for my_baker ``` -At this point, the `my_baker` account owns enough stake to bake but has still no consensus nor DAL rights because we haven't declared to the Tezos protocol our intention to become a baker. This can be achieved with the following command: +At this point, the `my_baker` account owns enough stake to bake but has still no consensus or DAL rights because we haven't declared our intention to become a baker to the Tezos protocol. This can be achieved with the following command: -``` -octez-client --endpoint "$ENDPOINT" register key my_baker as delegate +```bash +octez-client register key my_baker as delegate ``` -Seven cycles later (about 1h40 on Weeklynet), our baker will start receiving rights. To see for instance its consensus attestation rights in the current cycle, we can use the following RPC: +Seven cycles later (about 1h40 on Weeklynet), our baker will start receiving rights. To see for instance its consensus attestation rights in the current cycle, we can use the following RPC call: -``` -octez-client --endpoint "$ENDPOINT" rpc get /chains/main/blocks/head/helpers/attestation_rights\?delegate="$MY_BAKER" +```bash +octez-client rpc get /chains/main/blocks/head/helpers/attestation_rights\?delegate="$MY_BAKER" ``` -To see the DAL attestation rights of all bakers, we can use the following RPC: +To see the DAL attestation rights of all bakers, we can use the following RPC call: -``` -octez-client --endpoint "$ENDPOINT" rpc get /chains/main/blocks/head/context/dal/shards +```bash +octez-client rpc get /chains/main/blocks/head/context/dal/shards ``` This command returns an array of DAL attestation rights. The 2048 shards which are expected to be attested at this level are shared between active bakers proportionally to their stake. Each baker is assigned a slice of shard indices represented in the output of this command by a pair consisting of the first index and the length of the slice. So to check if some rights were assigned to us we can filter the array to our baker by running this command: +```bash +octez-client rpc get /chains/main/blocks/head/context/dal/shards | grep "$MY_BAKER" ``` -octez-client --endpoint "$ENDPOINT" rpc get /chains/main/blocks/head/context/dal/shards | grep "$MY_BAKER" -``` + +When attestation rights are assigned to your baker, continue to [Step 4: Run an Octez DAL node on Weeklynet](./run-dal-node.md). diff --git a/docs/tutorials/join-dal-baker/run-baker.md b/docs/tutorials/join-dal-baker/run-baker.md index 4bf9a2061..feb32bc39 100644 --- a/docs/tutorials/join-dal-baker/run-baker.md +++ b/docs/tutorials/join-dal-baker/run-baker.md @@ -1,26 +1,35 @@ -# Step 5: Run an Octez baking daemon on Weeklynet +--- +title: "Step 5: Run an Octez baking daemon on Weeklynet" +authors: Tezos core developers +last_update: + date: 23 January 2024 +--- The baking daemon is launched almost as usual, the only difference is that we use the `--dal-node http://127.0.0.1:10732` option to tell it to connect to the DAL node that we just launched in the previous step. -``` + +```bash octez-baker-alpha run with local node "$HOME/.tezos-node" my_baker --liquidity-baking-toggle-vote on --adaptive-issuance-vote on --dal-node http://127.0.0.1:10732 >> "$HOME/octez-baker.log" 2>&1 ``` -We can check that the DAL is now subscribed to the relevant topics by retrying the following RPC, which should now return all the topics of the form `{"slot_index":,"pkh":"
"}` where `index` varies between `0` included and the number of slot indexes (`32` on Weeklynet) excluded: +We can check that the DAL is now subscribed to the relevant topics by retrying the following RPC call, which should now return all the topics of the form `{"slot_index":,"pkh":"
"}` where `index` varies between `0` included and the number of slot indexes (`32` on Weeklynet) excluded: -``` +```bash curl http://localhost:10732/p2p/gossipsub/topics ``` We can also look at the baker logs to see if it manages to inject the expected operations. At each level, the baker is expected to: -- receive a block proposal (log message: "received new proposal ... at level ..., round ...") -- inject a preattestation for it (log message: "injected preattestation ... for my_baker (<address>) for level ..., round ...") -- receive a block (log message: "received new head ... at level ..., round ...") -- inject an attestation for it (log message: "injected attestation ... for my_baker (<address>) for level ..., round ...") -- inject a DAL attestation indicating which of the shards assigned to the baker have been seen on the DAL network (log message: "injected DAL attestation ... for level ..., round ..., with bitset ... for my_baker (<address>) to attest slots published at level ..."); if no shard was seen (either because they did not reach the DAL node for some reason or simply because nothing was published on the DAL at the targeted level), the operation is skipped (log message: "Skipping the injection of the DAL attestation for attestation level ..., round ..., as currently no slot published at level ... is attestable.") +- Receive a block proposal (log message: "received new proposal ... at level ..., round ...") +- Inject a preattestation for it (log message: "injected preattestation ... for my_baker (<address>) for level ..., round ...") +- Receive a block (log message: "received new head ... at level ..., round ...") +- Inject an attestation for it (log message: "injected attestation ... for my_baker (<address>) for level ..., round ...") +- Inject a DAL attestation indicating which of the shards assigned to the baker have been seen on the DAL network (log message: "injected DAL attestation ... for level ..., round ..., with bitset ... for my_baker (<address>) to attest slots published at level ..."); if no shard was seen (either because they did not reach the DAL node for some reason or simply because nothing was published on the DAL at the targeted level), the operation is skipped (log message: "Skipping the injection of the DAL attestation for attestation level ..., round ..., as currently no slot published at level ... is attestable.") Optionally, we can also launch an accuser which will monitor the behaviour of the other Weeklynet bakers and denounce them to the Tezos protocol if they are caught double-signing any block or consensus operation. -``` +```bash octez-accuser-alpha run >> "$HOME/octez-accuser.log" 2>&1 ``` + +Now you have a complete DAL baking setup. +For a summary, see [Conclusion](./conclusion). diff --git a/docs/tutorials/join-dal-baker/run-dal-node.md b/docs/tutorials/join-dal-baker/run-dal-node.md index cfbd5fbca..077f6d1b8 100644 --- a/docs/tutorials/join-dal-baker/run-dal-node.md +++ b/docs/tutorials/join-dal-baker/run-dal-node.md @@ -1,21 +1,33 @@ -# Step 4: Run an Octez DAL node on Weeklynet +--- +title: "Step 4: Run an Octez DAL node on Weeklynet" +authors: Tezos core developers +last_update: + date: 23 January 2024 +--- -``` +Start the DAL node by running this command: + +```bash octez-dal-node run >> "$HOME/octez-dal-node.log" 2>&1 ``` This, too, may take some time to launch the first time because it needs to generate a new identity file, this time for the DAL network. -When running normally, the logs of the DAL node should contain one line per block applied by the layer 1 node looking like: +The DAL node connects to the DAL network but it is not yet receiving shards. -``` -: layer 1 node's block at level , round is final -``` +DAL nodes share shards and information about them over a peer-to-peer pub/sub network built on the Gossipsub protocol. +As layer 1 assigns shards to the bakers, the Gossipsub network manages topics that DAL nodes can subscribe to. +For example, if a user submits data to slot 1, the network has a list of topics: a topic to identify the slot 1 shards assigned to baker A, a topic to identify the slot 1 shards assigned to baker B, and so on for all the bakers that are assigned shards from slot 1. -The DAL node we have launched connects to the DAL network but it is not yet subscribed to any Gossipsub topic. We can observe this by requesting the topics it registered to, using the following RPC: +Then, the baker daemon automatically asks the DAL node to subscribe to the topic that provides the shards that it is assigned to. +You can see the topics that the DAL node is subscribed to by running this RPC call: -``` +```bash curl http://localhost:10732/p2p/gossipsub/topics ``` -In particular, it won't collect the shards assigned to our baker until it is subscribed to the corresponding topics. Don't worry, the baker daemon will automatically ask the DAL to subscribe to the relevant topics. +If you are using the `tezos/tezos` Docker image, you can install the `curl` program by running the command `sudo apk add curl`. + +In the results, each topic contains a shard and the address of the baker that is assigned to it. +Now you can run a baker to verify these shards. +Continue to [Step 5: Run an Octez baking daemon on Weeklynet](./run-baker). diff --git a/docs/tutorials/join-dal-baker/run-node.md b/docs/tutorials/join-dal-baker/run-node.md index 540bf3c20..0fc91b467 100644 --- a/docs/tutorials/join-dal-baker/run-node.md +++ b/docs/tutorials/join-dal-baker/run-node.md @@ -1,20 +1,39 @@ -# Step 2: Run an Octez node on Weeklynet +--- +title: "Step 2: Run an Octez node on Weeklynet" +authors: Tezos core developers +last_update: + date: 24 January 2024 +--- -Once the Octez node has been configured to join Weeklynet, we can launch it and make its RPC available: +Now that the Octez node is configured to join Weeklynet, we can launch it and make its RPC available. -``` -octez-node run --rpc-addr 127.0.0.1:8732 --log-output="$HOME/octez-node.log" -``` +1. Optional: If it has been a few days since Weeklynet was restarted on Wednesday morning, you can speed up the bootstrapping process by loading a snapshot: -At first launch, the node will generate a fresh identity file used to identify itself on the Weeklynet L1 network, it then bootstraps the chain which means that it downloads and applies all the blocks. This takes a variable amount of time depending on when during the week these instructions are followed but at worst, on a Tuesday evening, it takes a few hours. Fortunately, we can continue to set up our Weeklynet baking infrastructure while the node is bootstrapping, all we have to do for this is to use another, already bootstrapped, node as RPC endpoint for `octez-client` when we want to interact with the chain. + 1. Download a snapshot of Weeklynet from https://snapshot.tzinit.org based on the instructions on that site. + For example, the command to download the snapshot may look like this: -A public RPC endpoint URL for Weeklynet is linked from the https://teztnets.com/weeklynet-about page, let's record it in a shell variable: -``` -ENDPOINT="" -``` + ```bash + wget -O snapshot_file https://snapshots.eu.tzinit.org/weeklynet/rolling + ``` -For example, for the Weeklynet launched on January 17 2024, the endpoint was: + 1. Load the snapshot in the node by running this command: -``` -ENDPOINT=https://rpc.weeklynet-2024-01-17.teztnets.com -``` + ```bash + octez-node snapshot import snapshot_file + ``` + + If you see the error "The chain name contained in the snapshot file is not consistent with the network configured in the targeted data directory," the snapshot is for the previous instance of Weeklynet and no snapshot is available for this week. + Continue with the next step as usual. + +1. Start the node: + + ``` + octez-node run --rpc-addr 127.0.0.1:8732 --log-output="$HOME/octez-node.log" + ``` + +At first launch, the node generates a fresh identity file used to identify itself on the Weeklynet L1 network. +Then it bootstraps the chain, which takes a variable amount of time depending on how many blocks need to be loaded. +At worst, if the network has been running for nearly a week, it can take a few hours. + +Fortunately, we can continue to set up our Weeklynet baking infrastructure while the node is bootstrapping. +Continue to [Step 3: Set up a baker account on Weeklynet](./prepare-account).