diff --git a/docs/_sidebar.md b/docs/_sidebar.md index 1090ac27..3b21a1f3 100644 --- a/docs/_sidebar.md +++ b/docs/_sidebar.md @@ -30,7 +30,7 @@ - [EVM](/wiki/EL/evm.md) - [Transaction anatomy](/wiki/EL/transaction.md) - [JSON-RPC](/wiki/EL/JSON-RPC.md) - - [Data Structures](/wiki/el/data-structures.md) + - [Data Structures](/wiki/EL/data-structures.md) - [DevP2P] - [Precompiled Contracts](/wiki/EL/precompiled-contracts.md) - [Consensus Layer](/wiki/CL/overview.md) @@ -54,7 +54,6 @@ - [Scaling](/wiki/research/scaling/scaling.md) - [Core Changes](/wiki/research/scaling/core-changes/core-changes.md) - [EIP-4844](/wiki/research/scaling/core-changes/eip-4844.md) - - Sharding - Statelessness - Purge - [MEV](/wiki/research/PBS/mev.md) @@ -71,6 +70,9 @@ - ASE - EOF - Portal Network + - Preconfirmations + - [Preconfirmations](/wiki/research/Preconfirmations/Preconfirmations.md) + - [Based Sequencing with Preconfs](/wiki/research/Preconfirmations/BasedSequencingPreconfs.md) - [Cryptography](/wiki/Cryptography/intro.md) - [ECDSA](/wiki/Cryptography/ecdsa.md) - [Keccak256](/wiki/Cryptography/keccak256.md) @@ -87,4 +89,4 @@
-
+ \ No newline at end of file diff --git a/docs/eps/week10-dev.md b/docs/eps/week10-dev.md index 23183c2c..35375d65 100644 --- a/docs/eps/week10-dev.md +++ b/docs/eps/week10-dev.md @@ -1,6 +1,6 @@ # Study Group Week 10 | Precompiles -Week 10 dev talk is focused on history expiry and dives into Portal Network, an overlay network for light clients enabling alternative access to current and historical data. +Week 10 dev talk is diving into EVM precompiles and their integration in execution clients. Join the presentation by [Danno Ferrin](https://twitter.com/shemnon), on [Monday, April 22, 4PM UTC](https://savvytime.com/converter/utc-to-germany-berlin-united-kingdom-london-china-shanghai-ny-new-york-city-japan-tokyo-australia-sydney-india-delhi-argentina-buenos-aires/apr-22-2024/4pm). @@ -28,4 +28,4 @@ Additionally, you can get ready by studying the following resources: - [EVM Precompiled and System Contract Catalog](https://github.com/shemnon/precompiles/) - [RollCall breakout on precompiles](https://www.youtube.com/watch?v=tg01COfxi_M) -- [Custom RPCs and Precompiles for Hyperledger Besu](https://www.youtube.com/watch?v=djL5nczlYFw) \ No newline at end of file +- [Custom RPCs and Precompiles for Hyperledger Besu](https://www.youtube.com/watch?v=djL5nczlYFw) diff --git a/docs/eps/week10-research.md b/docs/eps/week10-research.md new file mode 100644 index 00000000..6f774743 --- /dev/null +++ b/docs/eps/week10-research.md @@ -0,0 +1,27 @@ +# Study Group Week 10 | Consensus upgrades + +Week 10 research talk will cover variety of Beacon Chain upgrades currently researched, analyzed and considered for inclusion. + +Join the presentation by [](https://twitter.com/), on [Wednesday, April 24, 4PM UTC](https://savvytime.com/converter/utc-to-germany-berlin-united-kingdom-london-china-shanghai-ny-new-york-city-japan-tokyo-australia-sydney-india-delhi-argentina-buenos-aires/apr-24-2024/4pm). + +The talk will be streamed live on [StreamEth](https://streameth.org/65cf97e702e803dbd57d823f/epf_study_group) and [Youtube](https://www.youtube.com/@ethprotocolfellows/streams), links will be provided before the call in the [Discord server](https://discord.gg/addwpQbhpq). Discord also serves for the discussion and questions during the stream. + +## Pre-reading + +Before starting with the week 10 development content, make yourself familiar with resources in previous weeks, especially week 3 and week 5. You should have understanding of Beacon Chain and current consensus research topics. + +Additionally, you can get ready by studying the following resources: +- [A simple single slot finality](https://ethresear.ch/t/a-simple-single-slot-finality-protocol/14920) + +## Outline + +- Single Slot Finality +- MaxEB +- Inclusion lists +- PeerDAS +- SSLE + +## Additional reading and exercises + +- [Paths toward single-slot finality, 2022](https://notes.ethereum.org/@vbuterin/single_slot_finality) +- [Notes on SSF, Lincoln Murr](https://publish.obsidian.md/single-slot-finality/Welcome+to+My+Research!) \ No newline at end of file diff --git a/docs/eps/week3.md b/docs/eps/week3.md index be567662..de1d859c 100644 --- a/docs/eps/week3.md +++ b/docs/eps/week3.md @@ -6,7 +6,7 @@ Watch the presentation on overview of the CL with Alex Stokes on [StreamEth](htt [recording](https://streameth.org/embed/?playbackId=66a30awapcuiok0z&vod=true&streamId=&playerName=Consensus+Layer+Overview+%7C+Alex+Stokes+%7C+Week+3 ':include :type=iframe width=100% height=520 frameborder="0" allow="fullscreen" allowfullscreen') -For written summary of week 2 presentation, check the [notes](https://github.com/eth-protocol-fellows/protocol-studies/files/14850973/Week.3.EPFsg.Consensus.Layer.Overview.pdf) +For written summary of week 3 presentation, check the [notes](https://github.com/eth-protocol-fellows/protocol-studies/files/14850973/Week.3.EPFsg.Consensus.Layer.Overview.pdf) For archive of the discussion during the talk, check [this thread](https://discord.com/channels/1205546645496795137/1214219045205835776/1214219052969492541) in our Discord server. @@ -40,5 +40,6 @@ Additionally, you can read and get ready by studying the following resources: - [Slashing scenario explanation by Dankrad Feist](https://dankradfeist.de/ethereum/2022/03/24/run-the-majority-client-at-your-own-peril.html) - [Beacon Chain design mistakes by Justin Drake](https://www.youtube.com/watch?v=10Ym34y3Eoo) - [Casper and Consensus from Devcon 3](https://archive.devcon.org/archive/watch/3/casper-and-consensus/?tab=YouTube) +- [Anatomy of a slot](https://www.youtube.com/watch?v=EswDO0kL_O0) After learning about both EL and CL, run a client pair. Spin a pair of one execution and consensus client, read their logs to learn how they operate. diff --git a/docs/eps/week4.md b/docs/eps/week4.md index 99b9c3ed..1ec8004c 100644 --- a/docs/eps/week4.md +++ b/docs/eps/week4.md @@ -9,7 +9,7 @@ Watch the presentation on Ethereum core testing infrastracture on [Youtube](http [recording](https://www.youtube.com/embed/PQVW5dJ8J0c?si=fv5ww4_6zInGXpjO ':include :type=iframe width=100% height=560 frameborder="0" allow="fullscreen" allowfullscreen encrypted-media gyroscope picture-in-picture web-share') -For written summary of week 2 presentation, check the [notes](https://github.com/eth-protocol-fellows/protocol-studies/files/14850974/Week.4.EPFsg.Test.Security.Overview.pdf) +For written summary of week 4 presentation, check the [notes](https://github.com/eth-protocol-fellows/protocol-studies/files/14850974/Week.4.EPFsg.Test.Security.Overview.pdf) For archive of the discussion during the talk, check [this thread](https://discord.com/channels/1205546645496795137/1216771776810455160/1216771782040621118) in our Discord server. diff --git a/docs/eps/week7-research.md b/docs/eps/week7-research.md index 6f16c78c..55c1d8e2 100644 --- a/docs/eps/week7-research.md +++ b/docs/eps/week7-research.md @@ -12,7 +12,7 @@ Before starting with the week 7 development content, make yourself familiar with Additionally, you can read and get ready by studying the following resources: -- [Data structures in Ethereum](/wiki/protocol/data-structures.md) +- [Data structures in Ethereum](/wiki/EL/data-structures.md) - [Guillaume Ballet - The Verge](https://www.youtube.com/watch?v=F1Ne19Vew6w) ## Outline diff --git a/docs/eps/week9-research.md b/docs/eps/week9-research.md index 6034b52d..0858d515 100644 --- a/docs/eps/week9-research.md +++ b/docs/eps/week9-research.md @@ -13,7 +13,7 @@ Before starting with the week 9 research content, make yourself familiar with re Additionally, you can get ready by studying the following resources: - [Statelessness and history expiry, Ethereum.org](https://ethereum.org/en/roadmap/statelessness/) - [Portal Network web](https://www.ethportal.net/) -- [EL Data structure](https://epf.wiki/#/wiki/protocol/data-structures) +- [EL Data structure](https://epf.wiki/#/wiki/EL/data-structures) ## Outline diff --git a/docs/wiki/Cryptography/KZG.md b/docs/wiki/Cryptography/KZG.md new file mode 100644 index 00000000..ab61b6b6 --- /dev/null +++ b/docs/wiki/Cryptography/KZG.md @@ -0,0 +1,564 @@ +# KZG Polynomial Commitment Scheme + +## [TLDR](#tldr) +The KZG (Kate, Zaverucha, and Goldwasser) commitment scheme is like a cryptographic vault for securely locking away polynomials (mathematical equations) so that you can later prove you have them without giving away their secrets. It's like making a sealed promise that you can validate without ever having to open it up and show the contents. Using advanced math based on elliptic curves, it enables efficient, verifiable commitments that are a key part of making blockchain transactions more private and scalable. This scheme is especially important for Ethereum's upgrades, where it helps to verify transactions quickly and securely without compromising on privacy. + +KZG is a powerful cryptographic tool that supports a wide range of applications within the Ethereum ecosystem and other cryptographic applications. Its distinctive features are leveraged in proving schemes to enhance scalability and privacy in various applications. + + +## [Motivation](#motivation) + +### [ZKSNARKs](#zksnarks) +Learning about Polynomial Commitment Schemes (PCS) is important because they play a key role in creating Zero-Knowledge Succinct Non-Interactive Arguments of Knowledge (ZKSNARKs). ZKSNARKs are special cryptographic methods that allow someone (the prover) to show to someone else (the verifier) that they know a specific piece of information (like a number) without revealing that information. This is done by using PCS and Interactive Oracle Proofs (IOP) together. + +*Modern ZKSNARK = Functional Commitment Scheme + Compatible Interactive Oracle Proof (IOP)* + + +### [Use cases in Ethereum Ecosystem](#use-cases-in-ethereum-ecosystem) +KZG commitment scheme has emerged as a pivotal technology in the Ethereum ecosystem, particularly in the context of Proto-Danksharding and its anticipated evolution into Danksharding. This commitment scheme is a cornerstone of many Zero-Knowledge (ZK) related applications within Ethereum, enabling efficient and secure verification of data without revealing the underlying information. + +Ethereum-based applications utilizing the KZG (Kate, Zaverucha, and Goldberg) scheme include: + +- **Proto-Danksharding (EIP-4844)**: This proposal aims to reduce the cost of posting data on Ethereum's Layer 1 for rollups by using KZG for its polynomial commitment scheme. It introduces a "blob-carrying transaction" type to accommodate substantial data blobs, with only a commitment to the data blob being accessible from the execution layer. + +- **Data Availability Sampling**: PCS enable a critical feature known as Data Availability Sampling (DAS) in Ethereum roadmap, which allows validators to confirm the correctness and availability of data blobs without downloading the entire data. This capability is facilitated by the unique properties of PCS, enabling efficient verification processes in blockchain applications like Ethereum's Danksharding. + +- **PSE's Summa, Proof of Solvenecy Protocol**: Ethereum Foundation's PSE group project, Summa, leverages KZG commitments in its Proof of Solvency protocol. This allows centralized exchanges and custodians to demonstrate that their total assets exceed their liabilities, all while keeping user balance information private. + +- **Scroll's zkRollups**: Scroll, a native zkEVM Layer 2 for Ethereum, uses KZG to generate commitments to a collection of polynomials that encapsulate computations. This allows verifiers to request evaluations at random points to validate the accuracy of the computation represented by the polynomials. + +- **Jellyfish**: Jellyfish employs the KZG commitment scheme to generate commitments to polynomials during the commitment phase. It leverages the homomorphic properties of KZG for efficient evaluation of the polynomial at any point without revealing its coefficients. + +- **Hyperplonk**: Hyperplonk utilizes the multi-linear KZG commitment, indicating its application in scenarios requiring multi-linear polynomial commitments. + + +## [Goal](#goal) +Now that we are motivated to learn PCS, let us get started with defining what is our goal i.e. what is the exact problem we want to solve with KZG scheme. + +Say we have a function or polynomial $f(x)$ defined as $f(x) = f_0 + f_1x + f_2x^2 + \ldots + f_dx^t$. The degree of $f(x)$ is $t$. + +Our main goal with KZG scheme is that we want to prove to someone that we know this polynomial without revealing the polynomial, i.e. coefficients of the polynomial. + +In practice what we exactly do is that we prove that we know a specific evaluation of this polynomial at a point $x=a$. + +We write this, $f(a)$, for some $x=a$. + +## [Prerequisite Knowledge](#prerequisite-knowledge) +There are some important concepts we need to know before we can move further to understand KZG scheme. Fortunately, we can get an Engineering level understanding of the KZG scheme from just enough high school mathematics. We will try to gain some intuition on advanced concepts and their important properties without knowing them intimately. This can help us see the KZG protocol flow without bogged down by the advanced mathematics. + +We need to know: + +### [Modular Arithmetic](#modular-arithmetic) +An analog clock illustrates modular arithmetic as hours cycle back after reaching their limit. For KZG, it's enough to know simple arithmetic—adding, subtracting, multiplying, and dividing—along with using the modulus operation, just like a clock resets after 12 or 24 hours. + + +### [Finite Field of order prime p](#finite-field-of-order-prime) +A finite field of order prime $p$, we denote it by $\mathbb F_p$, is a special set of numbers, { $\{1, 2, 3, \ldots, p-1\}$ }, where you can do all the usual math operations (addition, subtraction, multiplication, and division, except by zero) and still follow the rules of arithmetic. + +The "order" of this set is the number of elements it contains, and for a finite field of order prime $p$, this number is a prime number. The most common way to create a $\mathbb F_p$ is by taking the set of all integers greater than or equal to $0$ and dividing them by $p$, keeping only the remainders. This gives us a set of numbers from $0$ to $p-1$ that can be used for arithmetic operations. For example, if $p = 5$, the set would be {0, 1, 2, 3, 4}, and you can add, subtract, multiply, and divide these numbers in a way that follows the rules of arithmetic. This set is a finite field of order 5, we denote this by $\mathbb F_5$, because it has exactly 5 elements, and it's a prime number. + +When we do modular arithmetic operations in the finite field $\mathbb F_p$, we have a nice "wrap around" property i.e. the field behaves as if it "wraps around" after reaching $(p - 1)$. + +In general, when we define a finite field, we define, the order $p$ of the field and an arithmetic operation like addition or multiplication. If it is addition, we denote the field by $(\mathbb F_p, +)$. If it is multiplication, we denote it by $(\mathbb F^*_p, +)$. The `*` is telling us to exclude the zero element from our field so that we can satisfy all the required properties of the finite field i.e. mainly we can divide the numbers and find inverse of all elements. If we include the zero element, we can't find the inverse of zero element. + +In the next section, we will learn how generators of a Group enable the KZG commitment scheme to function as an efficient, secure, and verifiable method of committing to polynomials, making it a powerful tool for cryptographic protocols, particularly in blockchain technologies where these properties are very important. + +### [Group](#group) +A Group is conceptually similar to a finite field, although with a few minor variations. An important difference is that in a group, we only have one arithmetic operation on the set, typically addition or multiplication as opposed to finite field with both addition and multiplication. Similarly to finite field, group elements must have an inverse and meet all its requirements, explained in the example below. + +The notation is ($\mathbb G, +)$ for a Group with addition as the group operation, ($\mathbb G^*, .)$ for Group with multiplication operation; the `*` is telling to exclude zero element to avoid division by zero. + +In the next section we use an example to define a Group. This will help develop an intuition on when we call a set of numbers a Group. + +### [Generator of a Group](#generator-of-a-group) +A generator is an element within a group that, when combined with itself repeatedly through the group's operation, can eventually produce every other element within the group. + +In mathematical sense, if you have a group ($\mathbb G, .)$ and an element $g$ in $\mathbb G$ we say that $g$ is a generator of $\mathbb G$ if the set of all powers of $g$, $(g, g^2, g^3, ...)$, is equal to $\mathbb G$ for a finite group, or covers all elements of $\mathbb G$ through this repeated operation in the case of an infinite group. + +This concept is best explained with an example. + +We will work with ($\mathbb G_7, +)$ of group elements { ${0,1,2,3,4,5,6}$ } and ($\mathbb G^*_7, .)$ of group elements { ${1,2,3,4,5,6}$ } with modulo $7$ to find the generator of the Groups. + +**Generator of Additive Group** + +Our set ($\mathbb G_7, +)$ with elements { ${0,1,2,3,4,5,6}$ } is a Group because it satisfies the definition of a Group. + +- **Closure:** When you add any two numbers in the set and take the remainder when divided by $7$, you end up with a result that's still in the set. +- **Associativity:** For any numbers $a, b$ and $c$ in the set, $(a+b)+c$ is always the same as $a+(b+c)$, even with modulo $7$. +- **Identity element:** The number $0$ acts as an identity element because when you add $0$ to any number in the set, you get the same number back. +- **Inverse elements:** Every number in the set has an inverse such that when you add them together, you end up back at the identity element $0$. For example, the inverse of $3$ is $4$ because $3 + 4 = 7$, which is $0$ modulo $7$. + +Now, for the generator. Since our group has a prime order $7$, any element except for the identity element $0$ is a generator. Let's pick the element $1$ as our generator i.e $g = 1$. Since we are working with an additive group, our group elements with generator g will be $\{0, g, 2g, 3g, 4g, 5g, 6g\}$. + + +Starting with $1$ and adding it to itself modulo $7$, we get: +- $1 + 1 = 2$ (which is $2*1$ modulo 7) +- $1 + 1 + 1 = 3$ (which is $3*1$ modulo 7) +- $1 + 1 + 1 + 1 = 4$ (which is $4*1$ modulo 7) +- $1 + 1 + 1 + 1 + 1 = 5$ (which is $5*1$ modulo 7) +- $1 + 1 + 1 + 1 + 1 + 1 = 6$ (which is $6*1$ modulo 7) +- $1 + 1 + 1 + 1 + 1 + 1 + 1 = 7$, which is $0$ modulo 7 (which is $7*1$ modulo 7) + +As you can see, by repeatedly adding $1$ modulo $7$, we can generate every other element in the group. Hence, $1$ is a generator of the group ($\mathbb G_7, +)$. Similarly, we could pick any number in $2, 3, 4, 5, 6$ as our generator, and by performing repeated addition modulo $7$, we would still generate the entire group. This is a special property of groups with a prime number of elements. + + +**Generator of Multiplicative Group** +For the multiplicative group of integers modulo a prime $p$, the group ($\mathbb G_p, .$) consists of the integers { ${1, 2, 3, \ldots, p-1}$ }, where the operation is multiplication modulo $p$. We'll choose a small prime to make it simple, say $p = 7$. So, our group ($\mathbb G^*_7, .)$ under multiplication modulo $7$ consists of the elements { ${1, 2, 3, 4, 5, 6}$ }. Remember, division by zero element is excluded, that's why we have `*` in the notation. + +Here's the group structure: + +- **Closure:** The product of any two elements, when reduced modulo $7$, is still an element of the set. +- **Associativity:** For any numbers $a, b, c$ in the set, $(a \cdot b) \cdot c$ is always the same as $a \cdot (b \cdot c)$, even when considering modulo $7$. +- **Identity element:** The number $1$ acts as an identity element because when you multiply any number in the set by $1$, you get the same number back. +- **Inverse elements:** Every number in the set has a multiplicative inverse in the set such that when you multiply them together, you get the identity element $1$. For example, the multiplicative inverse of $3$ is $5$ because $3 \cdot 5 = 15$, which is $1$ modulo $7$. + +Let's verify that each element is indeed a generator by multiplying it repeatedly modulo $7$: + +- Starting with $2$, we multiply by $2$ each time and take the result modulo $7$: + - $2^1 = 2$ + - $2^2 = 4$ + - $2^3 = 8 \equiv 1 \mod 7$ + - $2^4 = 16 \equiv 2 \mod 7$ (and here we cycle back to the beginning, showing that $2$ is not a generator) + +- Let's try $3$: + - $3^1 = 3$ + - $3^2 = 9 \equiv 2 \mod 7$ + - $3^3 = 27 \equiv 6 \mod 7$ + - $3^4 = 81 \equiv 4 \mod 7$ + - $3^5 = 243 \equiv 5 \mod 7$ + - $3^6 = 729 \equiv 1 \mod 7$ (and since we've reached the identity after hitting all elements, $3$ is a generator) + +You can verify that $5$ is also a generator for our multiplicative group ($\mathbb G^*_7, .)$ modulo $7$. + +### [Why Primes for Modulo Operations in Fields](#why-primes-for-modulo-operations-in-fields) +Choosing a prime number as the modulus for operations in a finite field offers several benefits and simplifies various aspects of field arithmetic: + +1. **Well-defined Division:** In a finite field, every non-zero element must have a multiplicative inverse. If the modulus is prime, every number in the set { ${1, 2, 3, \ldots, p-1}$ } has a multiplicative inverse modulo $p$. This property allows for well-defined division operations within the field, which wouldn't be possible if the modulus wasn't prime (except in special cases like Galois fields of order $p^n$, where $p$ is prime). + +2. **Simplicity of Construction:** When the modulus is a prime number, the field's construction is straightforward. The elements of the field are simply the set of integers { ${1, 2, 3, \ldots, p-1}$ }, and the field operations (addition, subtraction, multiplication, and division) are performed modulo $p$. For non-prime moduli, constructing a field requires more complex structures, such as polynomial rings. + +3. **Guaranteed Field Properties:** The use of a prime modulus guarantees the satisfaction of required field properties. These include - the existence of additive and multiplicative identities, the existence of additive and multiplicative inverses for every element, and the commutative, associative, and distributive laws for addition and multiplication. A prime modulus ensures all these properties are met. + +4. **Uniform Distribution of Non-zero Elements:** In a finite field with a prime modulus, the non-zero elements have a uniform distribution with respect to multiplication. This means that the multiplication table of the field does not have any 'gaps' and every element appears exactly once in each row and column of the multiplication table (except the row and column for the zero element). + +5. **Simplified Algorithms:** Many algorithms in number theory and cryptography are simpler and more efficient when working with prime fields. For example, finding multiplicative inverses can be done efficiently using the Extended Euclidean Algorithm, and there's no need for complex polynomial arithmetic that is necessary in non-prime fields. + +6. **Cryptographic Security:** In the context of cryptography, the difficulty of certain problems, such as the discrete logarithm problem, is well-understood in prime fields. This difficulty is crucial for the security of cryptographic systems. For composite moduli (especially when the factors are not known), the structure can introduce vulnerabilities or make the problem's difficulty less predictable. +7. **Optimization in Computation:** Some prime numbers, like 31 or primes of the form $2^n - 1$, are easily optimized by CPUs for multiplication operations. This optimization can lead to faster computation times, which is beneficial in applications where performance is a critical factor. + +Using a prime number as the modulus for finite fields simplifies the field arithmetic and ensures that all field properties are satisfied, which is essential for both theoretical and practical applications, particularly in cryptography. + + +### [Cryptographic Assumptions](#cryptographic-assumptions) +In order to work with KZG commitment scheme, we need two additional assumptions. We won't go deep into why these assumptions are needed but we will give an intuition to why these cryptographic assumptions are needed to make KZG more secure. + +**Discrete Logarithm** + +Say we have a generator $g$ in the group $\mathbb G^\*_p$ and $a$ is any element in the finite field $\mathbb F^*_p$ and $g^a$ is some element in the group $\mathbb G^\*_p$. The Discrete Logarithm assumption says that it is practically impossible to find $a$, for given $g$ and $g^a$. This means we can't easily find the exponent $a$ that will give us these elements. + +**Developing an intuition for Discrete Logarithm Problem** + +Imagine you have a special kind of lock that works with numbers (let's call this lock a "generator", and we'll name it $g$). This lock is part of a magic set of locks and keys, all living in a magical land called $\mathbb G^\*_p$. Now, you pick a secret number $a$ and use it to turn your lock $g$ a certain number of times. The lock ends up in a new position, let's call this $g^a$. + +If someone walks by and sees your lock at $g^a$, even if they know it started at $g$ and the magical land it belongs to, figuring out how many times you turned it (finding your secret number $a$) is incredibly difficult. + +In simpler terms, the Discrete Logarithm problem tells us that even though it's easy to turn the lock around if you know your secret number, going backwards — seeing the result and trying to guess the secret number — is like finding a needle in a haystack. This concept is crucial in cryptography, ensuring that some secrets are incredibly hard to uncover. + +**Strong Diffie-Hellman** + +Say we have a generator $g$ in the group $\mathbb G^\*_p$ and $a, b$ are any elements in the finite field $\mathbb F^*_p$ and $g^a$, $g^b$ are some elements in the group $\mathbb G^\*_p$. The Strong Diffie-Hellman assumption says that $g^a$ and $g^b$ are indistinguishable from $g^{ab}$. This means we can't extract any extra information about $g^{ab}$ given $g^a$ and $g^b$. + + +**Developing an intuition for Strong Diffie-Hellman** + +Imagine you're in a world, famous for its magical cookies, and there's a secret ingredient (our "generator", $g$) that makes them special. Two master bakers, Alice and Bob, each know a unique twist to using this ingredient, represented by their own secret recipes $a$ and $b$, respectively. + +When Alice bakes her cookies using her secret recipe, she creates a special batch $g^a$. Bob does the same with his recipe, resulting in another unique batch $g^b$. + +Now, suppose Alice and Bob decide to collaborate and combine their secret recipes to create a super-secret batch of cookies $g^{ab}$. The Strong Diffie-Hellman assumption is saying that even if someone has tasted both Alice's and Bob's individual batches, they can't decipher what their combined super-secret batch would taste like. The flavors of the combined recipe are indistinguishable from any other batch without knowing the exact combination of Alice's and Bob's recipes. + +So, in essence, the Strong Diffie-Hellman assumption tells us that just knowing the outcomes of individual secrets (recipes) doesn't help anyone crack the result of combining those secrets. This is a cornerstone of secure communication, ensuring that even if someone knows the separate pieces, the combined secret remains safe and unguessable. + + +### [Pairing Function](#pairing-function) +Say we have a generator $g$ in the group $\mathbb G^\*_p$ and $a, b$ are any elements in the finite field $\mathbb F^*_p$ and $g^a$, $g^b$ are some elements in the group $\mathbb G^\*_p$. + +A pairing function is a mathematical function that takes two inputs and produces a single output by mapping distinct pairs of inputs to a distinct value. It has two important properties, bilinearity and non-degeneracy. + +- Bilinearity means, we can move around in a reversible way. +- Non-degeneracy means, if we apply pairing function to the same element, it doesn't result in the identity element of the Group. + +Let's define these properties a bit more rigorously. + +A pairing function $e:$ $\mathbb G_1 X \mathbb G_2 \rightarrow \mathbb G_T$ such that it satisfies, + +Bilinear property: $e(g^a, g^b) = e(g, g^{ab}) = e(g^{ab}, g) = e(g,g)^{ab}$ + +Non-degenerate property: $e(g,g) \neq 1$, means the output is not an identity element. + +When $\mathbb G_1$ and $\mathbb G_2$ are the same Group, we call this symmetric pairing function. Otherwise, it is an asymmetric pairing function. + +Here are some great resources to learn more about pairing functions from a practical POV[^3][^8][^9]. + +**Developing an intuition for Pairing function** + +Imagine two separate islands, each inhabited by a unique species of magical creatures. The first island is home to Unicorns, each with a distinct color, and the second island is inhabited by Dragons, each with a unique fire color. A pairing function is like a magical bridge that connects a Unicorn with a Dragon, creating a unique, new magical creature, a Dracorn, that embodies characteristics of both. + +Here's how to think about this pairing function without getting bogged down by technicalities: + +- **Two Groups:** Think of the Unicorns and Dragons as belonging to two different groups (in mathematical terms, these are usually called groups $\mathbb G_1$ and $\mathbb G_2$). +- **Pairing Function:** The magical bridge acts as the pairing function. When a Unicorn and a Dragon meet on this bridge, the pairing function combines them into a Dracorn. This Dracorn has a special glow that uniquely corresponds to the combination of that specific Unicorn and Dragon (reversible). +- **Unique Outcome:** Just like every Unicorn and Dragon pair produces a Dracorn with a unique glow, in mathematics, a pairing function takes one element from each group and produces a unique output in a third group (often denoted as $\mathbb G_T$). + +**Why is this magical?** Because even though there are countless possible combinations of Unicorns and Dragons, each combination (pairing) produces a unique Dracorn. This is powerful in cryptography because it allows for complex operations that underpin many security protocols, ensuring that each combination is distinct and traceable to its original pair. + +**In simpler terms,** imagine you have two sets of keys (Unicorns and Dragons), and when you combine one key from each set, you get a unique lock (Dracorn). The magic is in how predictable yet secure this combination is, allowing for secure interactions that rely on the certainty of these unique outcomes without revealing the original keys. + +Pairing functions enable advanced cryptographic techniques, such as those used in certain types of digital signatures and encryption, by allowing this kind of "cross-group" interaction to occur securely and predictably. + +## [Properties of Commitments](#properties-of-commitments) +Commitment schemes are like the secret-keeping wizards of the digital world. They let someone make a promise about a piece of information (we'll call this the secret message) in a way that ties them to their promise without letting anyone else know what the secret is. Here's how it works: + +- **Making the Promise (Commitment):** You decide on a secret message and use a special spell (the commitment scheme) to create a magic seal (the commitment). This seal proves you have a secret, but it keeps the secret hidden. +- **Keeping It Secret (Hiding):** Even though you've made this seal, nobody else can see what your secret message is. It's like you've locked it in a chest and only you have the key. +- **Proving You're Honest (Binding):** The magic of the commitment is that you can't change your secret message later without breaking the seal. This means once you've made your commitment, you're bound to it. + +Later, when the time comes to reveal your secret, you can show the original message and prove that it matches the seal you made before. This lets someone else (the Verifier) check and confirm that your secret message is the same one you committed to in the beginning, proving that you kept your word. + +The Binding and Hiding properties are extremely important and they tie back to the above cryptographic assumptions we made with the Discrete Logarithm and Strong Diffie-Hellman assumptions. + +But for now, we don't need to go deep into the technicalities. In case, you want to learn more, here is a great resource for PCS from Prof. Dan Boneh[^4]. + +With this background, we are ready to explain KZG protocol flow and understand its construction. + + +## [KZG Protocol Flow](#kzg-protocol-flow) +Let us reiterate on what is the problem we are solving with KZG protocol. + +We want prove that we know a specific evaluation of a function or polynomial at a point $x=a$ without revealing it. + +In the KZG commitment scheme, the roles of the Trusted Third Party, Prover, and Verifier are critical to its function and security. Here's how each contributes to the process: + +1. **Trusted Third Party (Setup Authority):** This entity is responsible for the initial setup phase of the KZG scheme. They generate the public parameters (PP) or Common Reference String (CRS) that will be used in the commitments and proofs, based on a secret that only they know. This secret is crucial for the construction of commitments but must be discarded (or kept extremely secure) after the setup to ensure the system's integrity. The trust in this party is fundamental because if the secret is mishandled or leaked, it could compromise the entire system. The role of this party concludes once they have generated the CRS and distributed it to both the Prover and the Verifier. After this point, they are not involved in any further steps of the protocol, whether it be in proving or verifying. + +2. **Prover:** The Prover is the one who wants to commit to a certain piece of data (like a polynomial) without revealing it. Using the CRS provided by the Trusted Third Party, the Prover computes a commitment to their data. When it's time to prove certain properties of their data (like a polynomial evaluation at a specific point), the Prover can generate a proof based on their commitment. This proof shows that their data has certain properties without revealing the data itself. + +3. **Verifier:** The Verifier is the party interested in checking the Prover's claims about their secret data. The Verifier uses the proof provided by the Prover, along with the CRS from the Trusted Third Party, to verify that the Prover's claim about their data is true. This is done without the Verifier ever directly accessing the secret data. The strength of the KZG scheme ensures that if the proof verifies correctly, the Verifier can be confident in the Prover's claim, assuming the Trusted Third Party has correctly performed their role and the secret has not been compromised. + +This interaction between the three parties allows for secure and efficient verification of data properties in a variety of cryptographic applications, including blockchain protocols and secure computation, providing a balance between transparency and privacy. + +Below is a detailed sequence diagram that explains the flow in a typical KZG protocol. + +```mermaid +sequenceDiagram + autonumber + participant TP as Trusted Party + Actor P as Prover + Actor V as Verifier + + rect rgb(255, 190, 152) + note right of TP: Generates a ∈ F_p,
computes PP =
and DELETES a + TP->>P: Sends PP + TP->>V: Sends PP + rect rgb(128,182,223) + note right of P: P Chooses f(x) ∈ F_p[X] and computes C_f = C(f(a)) = f(a).g ∈ F_p using PP. + P->>V: Sends C_f + V-->>P: V Asks to open at b ∈ F_p + rect rgb(224,128,135) + note right of P: P Computes Q_b(x) = (f(x) - f(b)) / (x - b) and computes C_Q = C(Q_b) = Q_b(a).g. + P->>V: Sends (b, f(b), C_Q) + V->>P: Checks if e(C_f - f(b).g, g) == e(C_Q, ag - bg) + end + end + end +``` + +### [Trusted Setup](#trusted-setup) +A trusted third party picks a random element $a \in \mathbb{F}_p$. They compute the public parameter (PP) or common reference string (CRS), as < $g, {a^1}.g, {a^2}.g, \ldots, {a^t}.g$ >. Then, they **delete** $a$. This step of deleting $a$ is extremely important to secure the system. + +Then, the trusted party sends the CRS to the Prover and the Verifier. + +In practice, this process is wrapped around a multi-party computation (MPC) where a secret is generated in such a way that, as long as at least one participant remains honest, the randomness of the secret can be guaranteed. + +The trusted setup is a one-time procedure that generates a piece of data necessary for the cryptographic protocol to function. This data must be used every time the protocol is run, but once generated and the secrets are forgotten, no further participation from the creators of the ceremony is required. The trust in the setup comes from the fact that the secrets used to generate the data are securely discarded after the setup, ensuring that the data remains secure for future use. + +Modern protocols often use a powers-of-tau setup, which involves even thousands of participants. The security of the final output depends on the honesty of at least one participant who does not publish their secret. This approach is considered "close enough to trustless" in practice, making it a practical solution for cryptographic protocols that require a trusted setup. + +Ethereum has a very detailed documentation of the Trusted Setup ceremony for more detail[^2]. + +### [Initial Configuration](#initial-configuration) +Say the Prover has a function or polynomial $f(x)$ defined as $f(x) = f_0 + f_1x + f_2x^2 + \ldots + f_dx^t$ in a finite field $\mathbb F_p$. The degree of $f(x)$ is $t$ which is less than $p$, the order of the finite field $\mathbb F_p$. + +We often denote this as $f(x) \in \mathbb{F}_p[x]$. + +$\mathbb{G}_p$ is an Elliptic Curve group of order $p$ with a generator $g$. + +Often, the prime order $p$ is chosen such that $p \gt 2^k$, for some security parameter k. The prime number $p$ is very large in practice. + +Prover also picks a pairing function that satisfies both bilinear and non-degenerate properties. The pairing is denoted as below: + +$e:$ $\mathbb G_1 X \mathbb G_2 \rightarrow \mathbb G_T$ + +To simplify this step, Prover picks a polynomial $f(x) \in \mathbb{F}_p[x]$, the degree of $f(x)$ is at most $t$ which is less than $p$, the order of the finite field $\mathbb{F}_p$. Prover also picks a pairing function $e$ on the Elliptic Curve group $\mathbb{G}_p$. + + +### [Commitment of the Polynomial](#commitment-of-the-polynomial) +Say, the commitment of the polynomial $f(x)$ is denoted as $C_f$. The commitment is like hash function. + +So $C_f = {f(a)} \cdot g = {(f_0 + f_1a + f_2a^2 + \ldots + f_ta^t)} \cdot g$. Here $f(a)$ is the polynomial evaluated at $x=a$. + +Though, the Prover doesn't know $a$, he or she can still compute the commitment of the polynomial at $x=a$. + +So we have, $C_f = {f(a)} \cdot g = {(f_0 + f_1a + f_2a^2 + \ldots + f_ta^t)} \cdot g$. + +$C_f = {f_0} \cdot g + {f_1a} \cdot g + {f_2a^2} \cdot g + \ldots + {f_ta^t} \cdot g $. + +$C_f = {f_0} \cdot g + {f_1} \cdot (ag) + {f_2} \cdot ({a^2}g) + \ldots + {f_t} \cdot ({a^t}g)$. + +From the CRS, the Prover knows these values < $g, {a^1}.g, {a^2}.g, \ldots, {a^t}.g$ >, he or she can compute this value as commitment of the polynomial, $C_f$ and sends to the Verifier. + +### [Opening of the Polynomial](#opening-of-the-polynomial) +Upon receiving a commitment to a polynomial, denoted by $C_f$, from the Prover, the Verifier takes the next step in the protocol by selecting a random point, which we'll call $b$, from the field $\mathbb F_p$. The Verifier then requests the Prover to open or reveal the value of the polynomial at this specific point. + +**What does 'opening the polynomial' mean?** +Opening the polynomial at $x=b$ involves calculating the value of the polynomial at that point, which is mathematically represented as $f(b)$. This is done by evaluating the polynomial using the chosen point $b$: + +$f(b) = f_0 + f_1b + f_2b^2 + \ldots + f_tb^t$. + +Let's assume that this computation results in $f(b) = d$. The Prover's task is now to provide the Verifier with an Evaluation Proof, which is evidence that $f(b)$ truly equals $d$. + +Let's unpack this step by step. + +**Calculating the Evaluation Proof:** +The Prover determines the Quotient polynomial, which we will denote as $Q(x)$, and computes a commitment to it. This step is essential for creating a verifiable proof. Since we know $f(b)=d$, the polynomial $(f(x)−d)$ will have a root at $x=b$, meaning that $(f(x)−d)$ is divisible by $x−b$ with no remainder—this is a consequence of Little Bezout’s Theorem[^1]. + +Expressed in mathematical terms, the Quotient polynomial is: +$Q(x) = \frac{f(x) - f(b)}{x - b} = \frac{f(x) - d}{x - b}$ + +The commitment to the Quotient Polynomial, $Q(x)$, is represented by $C_Q$. Using the Common Reference String (CRS) provided during the Trusted Setup, the Prover calculates $C_Q$: +$C_Q = {Q(a)} \cdot g$. + +The Prover can calculate $C_Q$ as long as $(f(x) - f(b))$ is divisible by $(x−b)$. If this were not the case, $Q(x)$ would not be a proper polynomial i.e. the Quotient polynomial will have a denominator and some negative exponents, and the Prover could not compute the Evaluation Proof $C_Q$ using only the CRS. + +Finally, the Prover sends the tuple < $b, f(b), C_Q$ > to the Verifier, completing this stage of the protocol. + + +### [Verification Proof](#verification-proof) +Let's first summarize what data does the Verifier has so far in the protocol. + +**Data in hand:** The Verifier knows: +- The commitment of the polynomial, $C_f$. +- The opening point $b$. +- The value of the polynomial at $b$, denoted as $f(b)$. +- The commitment to the Quotient polynomial at $b$, denoted as $C_Q = {Q(a)} \cdot g$. + +**Properties of a commitment scheme:** +- **Completeness:** A commitment scheme is said to be **complete** if anything which is true is provable. +- **Soundness:** It is said to be **sound** if everything which is provable is true - i.e. anything which is false cannot be proven by the scheme. + +**Quotient polynomial and verification:** + +Recall that the Quotient polynomial is given by +$Q(x) = \frac{f(x) - f(b)}{x - b} = \frac{f(x) - d}{x - b}$. + +So, $(x - b) \cdot Q(x) = f(x) - d$ + +Evaluating this at $x=a$, we get +$(a - b) \cdot Q(a) = f(a) - d$ + +Multiplying both sides by the generator $g$, we get + +$(a−b) \cdot Q(a) \cdot g = f(a) \cdot g − d \cdot g$ + +Now, the Verifier knows that $C_Q = Q(a) \cdot g$ and $C_f = f(a) \cdot g$. + +So substituting, we get + +$(a−b) \cdot C_Q = C_f − d \cdot g$ + +If the verifier can confirm the validity of the above equality, it means the commitment has been verified. However, since the verifier is unaware of the value of $a$, they cannot directly validate the truth of this equality. + +However, the Verifier can use Elliptic Curve Pairings as outlined above to verify the equality constraint even without knowing $a$. Remember that the pairing function is denoted as: + +$e:$ $\mathbb G_1 X \mathbb G_2 \rightarrow \mathbb G_T$ such that it satisfies, + +Bilinear property: $e(g^a, g^b) = e(g, g^{ab}) = e(g^{ab}, g) = e(g,g)^{ab}$ + +Non-degenerate property: $e(g,g) \neq 1$, means the output is not an identity element. + +Let us for now, use a symmetric pairing function where $e:$ $\mathbb G X \mathbb G \rightarrow \mathbb G_T$ + +The Prover has to check he equality $(a−b) \cdot C_Q = C_f − d \cdot g$. + +The pairing function takes any two elements from the group $\mathbb G$ and maps it to an element in $\mathbb G_T$. + +- A commitment, like $C_f$ or $C_Q$, is obtained by multiplying a number (a scalar) with the group's generator, $g$. +- Since both $C_f$ and $C_Q$ are the result of this operation, they belong to the group $\mathbb G$. +- When we multiply $C_Q$ by the difference of two numbers $a$ and $b$, which is also a scalar, the result, $(a−b) \cdot C_Q$, stays within the group $\mathbb G$. +- Similarly, $C_f$ is a group element, and so is $d \cdot g$ because it's the generator multiplied by a scalar. +- Subtracting $d \cdot g$ from $C_f$ gives us another element in the group, $C_f − d \cdot g$. +- All these resulting elements are part of the group $\mathbb G$ and can be used in the pairing function. + +So, applying the pairing function on the both sides using the generator $g$ as the second parameter, the equality constraint becomes, + +$e((a−b) \cdot C_Q, g) = e(C_f − d \cdot g, g)$ + +We still can't calculate $a-b$ as nobody knows $a$. But we can use the bilinear property of the pairing function + +$e(g^a, g^b) = e(g, g^{ab}) = e(g^{ab}, g) = e(g,g)^{ab}$ + +So we can rewrite the equality constraint as + +$e(C_Q, (a−b) \cdot g) = e(C_f − d \cdot g, g)$ + +$e(C_Q, a \cdot g − b \cdot g) = e(C_f − d \cdot g, g)$ + +Though the Verifier doesn’t know $a$, he or she knows $a \cdot g$ from the Common Reference String. So now the Verifier can check whether the above equality is true or not. This ends the verification of the Evaluation Proof. + +**Full Opening VS Partial Opening of the polynomial** + +- **Full Open Process:** + - The Prover sends the complete polynomial to the Verifier. + - Using the CRS, the Verifier independently computes the polynomial's commitment. + - The Verifier then checks if this independently computed commitment matches the one originally sent by the Prover. + +- **Partial Open Process in KZG:** + - Instead of opening the whole polynomial, the Prover can opt for a partial open. + - This means the Prover reveals the polynomial's value at a single specific point. + - This partial revelation is known as the Evaluation Proof. + +## [KZG by Hands](#kzg-by-hands) +Now, let us practically derive the steps in KZG protocol using a small finite field. We can compute all finite field operations and pairing operations by hand and get a feel for the KZG protocol flow and verifying polynomial commitments. + +### [KZG by Hands - Initial Configuration](#kzg-by-hands---initial-configuration) +- We will work with the finite field $(\mathbb F_{11}, + )$. So, the prime order $p = 11$. This means all finite field operations are done modulo 11. +- The finite field set is {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10}. +- The generator $g = 2$ in $(\mathbb G_{11}, +)$. +- This means that the group operation is addition with modulo 11. +- The Prover selects the polynomial $f(x) = 3x^2 + 5x + 7$. +- Then we have the degree of the polynomial $f(x)$ as $t = 2$. +- The pairing function $e(x, y) = xy$ over $(\mathbb G_{11}, +)$. + +### [KZG by Hands - Trusted Setup](#kzg-by-hands---trusted-setup) +- The Trusted Party chooses a secret number randomly. Say, $a = 3$ is the secret number. +- They generate the public parameter or the common reference string (CRS) < $g, {a^1}.g, {a^2}.g, \ldots, {a^t}.g$ >. +- This is equal to < $2, 3 \cdot 2, {3^2} \cdot 2$ > which is equal to < $2, 6, 7$ > after applying modulo 11. +- The Trusted Party **deletes** the secret number $a$. +- The Trusted Party sends the CRS to the Prover and Verifier. + +### [KZG by Hands - Commitment of the polynomial](#kzg-by-hands---commitment-of-the-polynomial) +- The Prover calculates the commitment of the polynomial, $C_f$. +- $C_f = f(a) \cdot g = {f_0} \cdot g + {f_1} \cdot (ag) + {f_2} \cdot ({a^2}g) $. +- $C_f = 7 \cdot g + 5 \cdot (ag) + 3 \cdot a^2g = 7.2 + 5.6 + 3.7 = 65 = 10$ (mod 11). +- The Provers sends the commitment of the polynomial $C_f = 10$ to the Verifier. + +### [KZG by Hands - Opening of the Polynomial](#kzg-by-hands---opening-of-the-polynomial) +- The Verifier asks the Prover to open the polynomial at $x = 1$. +- The Prover computes the Quotient polynomial $Q(x) = \frac{f(x) - f(1)}{x - 1} = \frac{f(x) - d}{x - b}$. +- Compute $f(1) = d = 3.1^2 + 5.1 + 7 = 4$ (mod $11$). +- $Q(x) = \frac{3x^2 + 5x + 7 - 4}{x - 1} = \frac{3x^2 + 5x + 3}{x - 1}$. +- Divide the Leading Term: $3x^2$ divided by $x$ gives us $3x$. We write $3x$ above the division bar. +- Multiply the Divisor by the Quotient's Leading Term: Multiply $x - 1$ by $3x$ to get $3x^2 - 3x$. +- Subtract from the Polynomial: Subtract $3x^2 - 3x$ from $3x^2 + 5x$ to get $8x$. +- Bring Down the Next Term: Bring down the $+3$ to get $8x + 3$. +- Divide the Next Term: $8x$ divided by $x$ is $8$. Write $+8$ above the division bar next to $3x$. +- Multiply Again: Multiply $x - 1$ by $8$ to get $8x - 8$. +- Subtract Subtract: $8x - 8$ from $8x + 3$ to get $11$. +- Apply Modulo $11$: We reduce each term modulo $11$. Since $11$ is $0$ modulo $11$, the remainder is $0$. +- The Prover computes the commitment of $C_Q = Q(a) \cdot g = 3 \cdot ag + 8 \cdot g = 3.6 + 8.2 = 34 = 1$ (mod 11). +- The Prover sends to the Verifier < $1, f(1), C_Q$ > = < $1, 4, 1$ >. + +### [KZG by Hands - Verification](#kzg-by-hands---verification) +- The Verifier must check the pairing constraint $e(C_Q, a \cdot g − b \cdot g) = e(C_f − d \cdot g, g)$ +- L.H.S (left hand side): $e(1, 6 - 1.2) = e(1, 4) = 1.4 = 4 (mod 11)$ +- R.H.S (right hand side): $e(10 - 4.2, 2) = e(2, 2) = 2.2 = 4 (mod 11)$. +- This proves the equality constraint is true, hence the Evaluation Proof is verified. + +## [Security of KZG](#security-of-kzg) +**Deleting toxic waste during the Trusted Setup Ceremony** + +- Imagine the Prover somehow finds out the secret number $a$ or the Trusted Party leaks $a$ to a malicious Prover. +- The Prover computes $f_1(x) = 3x^2 + 5x + 7$ at $x=3$. So we get, $f_1(2) = 3.3^2 + 5.3 + 7 = 49 = 5 mod(11)$ +- The Prover computes $f_2(x) = 2x^2 + 7x + 10$ at $x=3$. So we get, $f_2(2) = 2.3^2 + 7.3 + 10 = 49 = 5 mod(11)$ +- This breaks the binding property of the commitment scheme leading to fraudulent proofs by the malicious Prover. +- Hence, it is extremely important to **delete** the secret number $a$ by the Trusted Party after generating the CRS. + +## [Asymmetric Pairing Functions](#asymmetric-pairing-functions) +An asymmetric pairing function is denoted as: + +$e:$ $\mathbb G_1 X \mathbb G_2 \rightarrow \mathbb G_T$. + +Let the generators of $\mathbb G_1$ be $g_1$ and $\mathbb G_2$ be $g_2$. + +The Prover has to check the equality $(a−b) \cdot Q(a) = f(a) − d$. + +Multiplying both sides by $g_1$, we get + +$(a−b) \cdot Q(a) \cdot g_1 = f(a) \cdot g_1 − d \cdot g_1$ + +$(a−b) \cdot C_Q = C_f − d \cdot g_1$ + +Applying the asymmetric pairing function on both sides, we get + +$e((a−b) \cdot C_Q, g_2) = e(C_f − d \cdot g_1, g_2)$ + +Using the bilinear property, we get + +$e(C_Q, (a−b) \cdot g_2) = e(C_f − d \cdot g_1, g_2)$ + +$e(C_Q, a \cdot g_2 − b \cdot g_2 ) = e(C_f − d \cdot g_1, g_2)$ + +Here $a \cdot g_2$ will be the part of CRS of $\mathbb G_2$ and everything else can be either computed or part of CRS of $\mathbb G_1$. + +## [Unwavering Compactness](#unwavering-compactness) +The KZG Polynomial Commitment Scheme ensures that both commitments and evaluation proofs are of a fixed size, regardless of the polynomial's length, offering consistent and space-efficient cryptographic operations[^5][^6][^7]. + +One key benefit of the KZG Polynomial Commitment Scheme is its efficient use of space. No matter the length or complexity of the polynomial we're working with, the commitment to that polynomial—essentially its cryptographic "footprint"—is always a single, fixed-size element within a mathematical group, $\mathbb G$. This means that as the polynomial grows in degree, the size of the commitment does not increase. The same principle applies to the evaluation proof, which is the evidence we provide to show that our commitment is accurate. Whether we're verifying just one value or many at once (in batch mode), the proof will always be of a consistent size. This consistency in size translates to predictable and efficient storage requirements, an important feature for practical applications in cryptography. + +## [KZG Batch Mode](#kzg-batch-mode) +KZG commitments can also be opened and verified at multiple points or using multiple polynomials or any combination of them. This is called batch mode in practice. + +### [Single Polynomial, Multiple Points](#single-polynomial-multiple-points) +In batch mode, the Verifier requests the Prover to validate a set of points $B =$ { $b_1, b_2, b_3, \ldots, b_n$ } with $n < t$, where $t$ is the degree of the polynomial $f(x)$. For these points, the Prover computes the values $f(b_1) = d_1, f(b_2) = d_2, \ldots, f(b_n) = d_n$ and forms the set $D =$ { $d_1, d_2, d_3, \ldots, d_n$ }. + +The Prover then creates a Polynomial $P(x) = (x - b_1)(x - b_2)\ldots(x - b_n)$. Given that $n < t$, it's possible to divide $f(x)$ by $P(x)$, resulting in $f(x) = P(x)Q(x) + R(x)$, where $Q(x)$ is the quotient polynomial and $R(x)$ is the remainder. This division suggests that $f(x)$ can be represented as such, not implying direct divisibility by $Q(x)$. + +The commitment for $Q(x)$, denoted as $C_Q$, alongside the set $B$, is sent to the Verifier by the Prover. Optionally, the Prover may also send the remainder polynomial $R(x)$ to the Verifier. However, the Verifier has the capability to independently compute $R(x)$, considering that for any $b_i$ in $B$, $P(x)$ evaluates to zero, leading to $f(x) = R(x)$ for all $b_i$ in $B$. + +As the degree of $Q(x)$ is $n$ and $R(x)$'s degree is less than $n$, the Verifier, knowing $R(x)$'s evaluation at $n$ points, can determine $R(x)$ via Lagrange’s Interpolation[^10]. + +The Verifier also computes the polynomials $P(x)$ and $R(x)$, alongside their commitments $C_P = P(a) \cdot g$ and $C_R = R(a) \cdot g$. They proceed to verify the Batch Evaluation by ensuring that $f(b_i) = R(b_i)$ for all $b_i$ in $B$ and that the equality $f(x) = P(x)Q(x) + R(x)$ holds. + +The Verifier needs to evaluate the above constraint to verify the proof. However, since the secret opening at $x = a$ is unknown, hence she or he cannot evaluate it directly. But like before, the Verifier can use pairings to solve this. + +To verify, the Verifier checks: +- $f(b_i) = R(b_i)$ for each $b_i$ in $B$, comparing the Prover's provided $D$ values with their computation of $R(x)$ at each $b_i$. + +- The equality $f(x) \cdot g - R(x) \cdot g = P(x)Q(x) \cdot g$ when evaluated at $x = a$, simplifying to $C_f - C_R = P(a) \cdot C_Q$ using known commitments and the secret $a$. + +Despite not knowing $a$, the Verifier utilizes pairings to assess the proof: +- Since both $C_f$ and $C_R$ belong to $\mathbb G$, their difference does too. +- Given $C_Q$'s membership in $\mathbb G$ and $P(a)$ as a scalar, $P(a) \cdot C_Q$ remains within $\mathbb G$. + +Applying the pairing function yields: + +$e(C_f − C_R, g) = e(P(a) \cdot C_Q, g)$ + +Applying the bilinearity property, we get + +$e(C_f - C_R, g) = e(C_Q, C_P)$ + +where $C_P = P(a) \cdot g$. Given this, the Verifier can confirm the truth of the equality, thereby verifying the proof. + + +## [References](#references) +[^1]: https://en.wikipedia.org/wiki/Polynomial_remainder_theorem +[^2]: https://github.com/ethereum/kzg-ceremony +[^3]: https://www.rareskills.io/post/bilinear-pairing +[^4]: https://www.youtube.com/watch?v=WyT5KkKBJUw +[^5]: https://www.iacr.org/archive/asiacrypt2010/6477178/6477178.pdf +[^6]: https://dankradfeist.de/ethereum/2020/06/16/kate-polynomial-commitments.html +[^7]: https://www.youtube.com/watch?v=uGeIDNEwHjs&t=520s +[^8]: https://www.youtube.com/watch?v=8WDOpzxpnTE +[^9]: https://vitalik.eth.limo/general/2017/01/14/exploring_ecp.html +[^10]: https://en.wikipedia.org/wiki/Lagrange_polynomial \ No newline at end of file diff --git a/docs/wiki/EL/data-structures.md b/docs/wiki/EL/data-structures.md index 686647f9..4d3ad431 100644 --- a/docs/wiki/EL/data-structures.md +++ b/docs/wiki/EL/data-structures.md @@ -31,14 +31,12 @@ More on [Merkle Trees in Ethereum](https://blog.ethereum.org/2015/11/15/merkling ## Primer on Patricia Tree -Patricia Tries (also called Radix tree) are n-ary trees which unlike Merkle Trees,is used for storage of data instead of verification. +Patricia Tries (also called Radix tree) are n-ary trees which unlike Merkle Trees, are used for storage of data instead of verification. Simply put, Patricia Tries is a tree data structure where all the data is store in the leaf nodes, and each non-leaf nodes is a character of a unique string identifying the data. Using the unique string we navigate through the character nodes and finally reach the data. Hence, it is very efficient at data retrieval. Patricia tries are designed to be more space-efficient than traditional trie structures by eliminating redundant nodes with single children. They achieve compactness by sharing prefixes among keys. This means that common prefixes are shared among different keys, reducing the overall storage requirements. -### **TODO: Patricia Tree Diagram** - ## Merkle Patricia Trie in Ethereum Ethereum's primary data structure for storing the execution layer state is a **Merkle Patricia Trie** (pronounced "try"). It is named so, since it is a Merkle tree that uses features of PATRICIA (Practical Algorithm To Retrieve Information Coded in Alphanumeric), and because it is designed for efficient data retrieval of items that comprise the Ethereum state. @@ -52,6 +50,12 @@ There are three types of nodes within the MPT: Every single node has a hash value. The node's hash is calculated as the SHA-3 hash value of its contents. This hash also acts as a key to refer that specific node. Nibbles serve as the distinguishing unit for key values in the MPT. It represents a single hexadecimal digit. Each trie node can branch out to as many as 16 offshoots, ensuring a concise representation and efficient memory usage. +##### **TODO: Patricia Tree Diagram** + +# Ethereum + +Ethereum's primary data structure for storing the execution layer state is a **Merkle Patricia Trie** (pronounced "try"). It is named so, since it is a Merkle tree that uses features of PATRICIA (Practical Algorithm To Retrieve Information Coded in Alphanumeric), and because it is designed for efficient data retrieval of items that comprise the Ethereum state. + Ethereum state is stored in four different modified merkle patricia tries (MMPTs): - Transaction Trie @@ -129,3 +133,5 @@ The transition to new verkle tree database poses a major challenge. To securely - [More on Verkle Tree](https://notes.ethereum.org/@vbuterin/verkle_tree_eip#Simple-Summary) - [Verge transition](https://notes.ethereum.org/@parithosh/verkle-transition) - [Implementing Merkle Tree and Patricia Trie](https://medium.com/coinmonks/implementing-merkle-tree-and-patricia-trie-b8badd6d9591) • [archived](https://web.archive.org/web/20210118071101/https://medium.com/coinmonks/implementing-merkle-tree-and-patricia-trie-b8badd6d9591) + +[More on Merkle Patricia Trie](https://ethereum.org/developers/docs/data-structures-and-encoding/patricia-merkle-trie) diff --git a/docs/wiki/EL/evm.md b/docs/wiki/EL/evm.md index 31f6119d..e49b3c89 100644 --- a/docs/wiki/EL/evm.md +++ b/docs/wiki/EL/evm.md @@ -70,7 +70,7 @@ Ethereum has two kinds of accounts: - **External account:** An account [controlled by an associated private key](/wiki/Cryptography/ecdsa.md) and empty EVM code. - **Contract account:** An account controlled by an associated non-empty EVM code. The EVM code as part of such an account is colloquially known as a _smart contract._ -Refer [Ethereum data structures](wiki/protocol/data-structures.md) for details on how the world state is implemented. +Refer [Ethereum data structures](wiki/EL/data-structures.md) for details on how the world state is implemented. ## Virtual machine paradigm diff --git a/docs/wiki/research/Preconfirmations/BasedSequencingPreconfs.md b/docs/wiki/research/Preconfirmations/BasedSequencingPreconfs.md new file mode 100644 index 00000000..e1612fb5 --- /dev/null +++ b/docs/wiki/research/Preconfirmations/BasedSequencingPreconfs.md @@ -0,0 +1,193 @@ +# Ethereum Based Sequencing with Preconfirmations + +## [Overview](#overview) + +Ethereum's evolving ecosystem is set to introduce new paradigms for rollups and chain interactions, emphasizing seamless transitions and enhanced user experiences. This wiki article introduces a framework for Ethereum sequencing and preconfirmations, originally proposed by Justin Drake[^1][^4], a step toward realizing this vision, offering a unified platform for all Ethereum chains and rollups. + + +## [Motivation](Motivation) + +### [United Chains of Ethereum](#united-chains-of-ethereum) + +The vision for Ethereum is not just a network of isolated chains but a cohesive ecosystem where all rollups and chains coexist without friction, termed the "United Chains of Ethereum." This concept envisions a scenario where users can move between different states (rollups) with ease, akin to crossing state lines without the need for passports or the imposition of tariffs. Such an environment would not only enhance user experience but also foster a more integrated and efficient blockchain ecosystem. + +![United Chains of Ethereum](/docs/wiki/research/img/preconfs/united-chains-of-ethereum.jpg) + +_Figure: United Chains of Ethereum, Credit Justin Drake_ + +### [Ethereum's Services for Rollups](#ethereums-services-for-rollups) + +- **Current Services:** Ethereum currently provides two critical services to rollups: settlement and data availability. These services lay the foundation for rollups to operate effectively on Ethereum's decentralized platform. + +- **Introduction of Ethereum Sequencing:** Ethereum sequencing[^2][^3], is proposed to complement the existing ones, offering a new resource that rollups can leverage to further optimize their operations. Although sequencing has always been inherent to Ethereum, its potential as a dedicated service for rollups represents an innovative application, akin to the adaptive use of core data for new functionalities. + +### [Current Sequencing Options](#current-sequencing-options) + +![Sequencing Types](/docs/wiki/research/img/preconfs/based-sequencing-problems-space.png) + +_Figure: Different Sequencing Options and their Problem Space, Credit Justin Drake_ + + +#### [Decentralized Sequencing](#decentralized-sequencing) + +**Overview:** Decentralized sequencing distributes the responsibility of transaction ordering among multiple nodes rather than a single central authority. This method enhances security and resistance to censorship, as no single node can dictate the transaction order by itself. + +**Problems and Challenges:** +- **Complexity in Coordination:** Since multiple nodes are involved in transaction ordering, achieving consensus can be challenging and complex, particularly when the nodes have varying incentives. +- **Network Integrity Maintenance:** Ensuring that all participating nodes follow the protocol without any malicious behavior can be difficult to enforce. +- **Front-Running and MEV:** Miners or validators might exploit their ability to order transactions to extract maximal extractable value (MEV), which can lead to unfair transaction processing and a negative user experience. +- **Resilience to Censorship:** Although decentralized sequencing makes censorship more difficult, it doesn't eliminate the possibility, especially if a collusion of nodes occurs. + +#### [Shared Sequencing](#shared-sequencing) + +**Concept:** Shared sequencing is a form of decentralized sequencing where the task of ordering transactions is shared among several entities, typically across different layers or platforms. This approach is designed to further decentralize the process and reduce the influence any single participant might have over the sequence of transactions. + +**Application:** In Ethereum, shared sequencing could involve various rollups solutions that coordinate to manage transaction order. This coordination can help ensure that transactions are processed efficiently and fairly, reducing the potential for bottlenecks or biased sequencing practices. + +**Benefits:** Shared sequencing aims to promote scalability by distributing the load of transaction processing and enhancing the network’s throughput. It also strives for neutrality and fairness in transaction handling, critical for maintaining trust in a decentralized ecosystem. + +**Problems and Challenges:** +- **MEV Sharing:** Coordinating MEV sharing, like the approach Espresso is investigating, requires sophisticated mechanisms to fairly distribute MEV across participating rollups and chains[^5]. +- **Deposits Sharing:** Solutions like zkSync's deposit sharing are innovative but require widespread adoption and trust among different rollups to function effectively, potentially leading to centralization of trust[^6]. +- **Execution Sharing:** Implementation of execution sharing strategies, such as Polygon's aggregation layer, requires standardization and integration across different rollups to ensure compatibility and trustless atomicity[^7]. + +**Based Sequencing:** + +**Concept:** A specialized form of decentralized sequencing that uses the base layer of a Ethereum, Beacon chain, to manage transaction ordering. This method leverages the security and consensus mechanisms of the Beacon chain to ensure that transactions are sequenced in a trustless manner. + +**Focus:** Based sequencing aims to integrate the robust security features of the Beacon chain into transaction sequencing, reducing dependency on external sequencers or centralized systems. It aligns with Ethereum's decentralized principles by using the existing Ethereum infrastructure to secure transaction order. + +**Integration with Shared Sequencing:** Based sequencing can be a pivotal part of a larger shared sequencing strategy, providing a reliable, secure foundation that other layers or rollups can build upon. It ensures that at least one layer of the transaction ordering process is closely tied to the highly secure, well-tested consensus mechanisms of the Ethereum blockchain. + +**Problems and Challenges:** +- **Proposer Responsibility:** Proposers must opt into based sequencing by posting collateral, adding financial risk and responsibility to their role. +- **Inclusion List Management:** The concept of inclusion lists must be maintained and managed carefully to ensure fair transaction inclusion. +- **Consensus Mechanism Dependence:** Based sequencing is inherently tied to the underlying consensus mechanism, which means any issues with the consensus could directly affect transaction sequencing. +- **Preconfirm Complexity:** Implementing preconfirm mechanisms, where users get assurance of transaction execution from proposers, adds complexity to transaction processing and requires a new level of trust and interaction between users and proposers. + +## [Technical Construction](#technical-construction) + +### [Based Sequencing](#based-sequencing) + +- **Mechanism:** The proposal for based sequencing involves utilizing the beacon chain's look-ahead period to invite proposers to opt into providing sequencing services by posting collateral. This approach leverages Ethereum's existing structure to introduce a new layer of functionality for rollups. + +- **Look-Ahead Period:** By capitalizing on the beacon chain's ability to predict the next set of proposers, the system can prepare and designate specific proposers to take on the additional role of sequencers, ensuring that rollups have predictable and reliable sequencing services. + + +### [Preconfirm Mechanism](#preconfirm-mechanism) + +In the [Preconfirmations](/docs/wiki/research/Preconfirmations/Preconfirmations.md) article, I explained the details on how Preconfirmations work and the promise acquisition process flow[^2][^3]. + +- **User Interaction with Proposers:** Users can identify which proposers within the look-ahead period have opted for based sequencing and request preconfirmations from them. These preconfirmations are akin to promises that the user's transaction will be included and executed in the future, with penalties applied for non-fulfillment. + +- **Slashing for Non-Fulfillment:** The system imposes penalties, or slashing, for proposers who fail to fulfill their preconfirmations. This adds a layer of accountability, ensuring that proposers are incentivized to honor their commitments. + +### [Look-Ahead Preconf Construction](#look-ahead-preconf-construction) + +![Look-ahead preconf construction](/docs/wiki/research/img/preconfs/lookahead-preconfs.png) + +_Figure: Look-ahead mechanism for Preconfirmations, Credit Justin Drake_ + + +- **Lookahead Period:** On the Ethereum Beacon chain, there is a lookahead period where upcoming proposers for block slots are known ahead of time. This period can typically include a set number of the next 32 slots. +- **Preconfirmation Request:** A user who wants to make a transaction sends a preconfirmation request to a proposer who is scheduled to create a block in the near future (within the lookahead period). The request includes the transaction details and possibly a fee offer. +- **Promise Issuance:** Upon receiving the preconfirmation request, the chosen proposer – referred to as a preconfer – assesses the transaction and decides whether or not to make a promise. If the proposer agrees, they issue a promise to the user, committing to include and execute the transaction in a future block when their turn to propose comes up. This promise is backed by collateral that the proposer has posted, which can be slashed if they fail to honor their promise. +- **Inclusion of the Preconfirmed Transaction:** When the proposer's slot (n+1 in the above figure) arrives, they must include and execute the preconfirmed transaction as they promised. If the proposer fails to do so without a valid reason, they risk being slashed. +- **Sharing of the Preconfirmation:** The promise made by the proposer may need to be communicated to others in the network, especially if there are multiple proposers who might include the transaction before the proposer’s slot arrives. This communication can be facilitated through various means, including MEV boost relays, to ensure that the transaction is settled and included appropriately. +- **Execution of the Transaction:** Once the proposer’s turn comes, and if they have not been preempted by an earlier proposer, they include the preconfirmed transaction in the block they are proposing. This ensures that the transaction is executed on-chain as was promised to the user. + +### [Communication through MEV Boost](#communication-through-mev-boost) + +The integration of preconfirmations with MEV Boost represents a critical aspect of the technical construction, facilitating the efficient flow of information between users, proposers, builders, and the Ethereum network. By routing preconfirmation details through MEV Boost, the system ensures that builders are aware of preconfirmed transactions and can construct blocks accordingly. This process not only optimizes the inclusion of transactions but also maintains the integrity and value of the constructed blocks, aligning with the overarching goals of the Ethereum sequencing and preconfirmation framework. + +## [Preconfirmations Flow through MEV Boost](#preconfirmations-flow-through-mev-boost) + + +```mermaid +sequenceDiagram + participant U as User + participant P as Proposer + participant R as MEV Boost/Relays + participant B as Builders + participant BC as Beacon Chain + + U->>+P: Request Preconfirmation + P-->>-U: Promise Preconfirmation + P->>+R: Send Preconfirmation Promise + R->>+B: Relay Promise to Builders + B-->>-R: Build Block with Preconfirmed Tx + R->>+P: Send Block to Proposer + P->>+BC: Propose Block on Beacon Chain + BC-->>-U: Execute and Finalize Tx +``` + +*Figure: Preconfirmations Flow through MEV Boost* + + +The process of how preconfirmations would flow through MEV Boost within the context of Ethereum's base layer sequencing and preconfirmations involves several key steps and entities and it is valuable to discuss in details. This mechanism aims to ensure that transactions preconfirmed by proposers (who have opted into providing sequencing services) are communicated effectively to builders through Relays in MEV Boost and ultimately included in the constructed blocks. Here's a detailed step-by-step explanation of the process: + +- **User Requests Preconfirmation:** + - A user identifies proposers within the beacon chain's look-ahead period who have opted into providing based sequencing by posting collateral. + - The user then sends a preconfirmation request to one of these proposers, seeking assurance that their transaction will be included and executed in a future slot. + +- **Proposer Provides Preconfirmation:** + - The selected proposer evaluates the request and, if accepted, provides the user with a preconfirmation. This preconfirmation is essentially a promise to include and execute the user's transaction in a specified future slot, subject to certain conditions and penalties for non-fulfillment. + +- **Proposer to MEV Boost Communication:** + - Once a proposer issues a preconfirmation, they communicate this information to MEV Boost. MEV Boost acts as an intermediary that facilitates the communication between proposers (now acting as sequencers for their respective slots), builders, and ultimately, the Ethereum network. + +- **MEV Boost Relays Preconfirmations to Builders:** + - MEV Boost relays the preconfirmation details to builders, who are responsible for constructing the blocks. Builders receive information about all preconfirmed transactions, which they must consider while building their blocks. + +- **Builders Construct Blocks Considering Preconfirmations:** + - With the preconfirmation details at hand, builders construct blocks that honor these preconfirmations. This involves including the preconfirmed transactions in the block for the specified slot and ensuring that the execution conditions promised in the preconfirmations are met. + +- **Blocks Are Proposed to the Network:** + - Once builders construct a block that respects all preconfirmations and optimizes for other factors (like MEV), the block is proposed to the Ethereum network. The proposer for the relevant slot, who initially issued the preconfirmation, is responsible for ensuring that this block gets submitted. + +- **Execution and Settlement:** + - If the block is successfully included in the blockchain, the preconfirmed transactions are executed as promised, fulfilling the proposer's commitment to the user. If a proposer fails to fulfill the preconfirmation, penalties (slashing) may be applied depending on the nature of the fault (e.g., liveness fault, safety fault). + +**Additional Considerations:** + +- **Slashing Mechanism:** The process incorporates a slashing mechanism to penalize proposers if they fail to honor their preconfirmations. This ensures a level of accountability and trust in the system. +- **Dynamic Communication:** The flow of information through MEV Boost allows for dynamic adjustments based on real-time conditions, such as changes in transaction priority or network congestion. + +## [Future Areas of Research](#future-areas-of-research) + +Previous discussions on Ethereum Based Sequencing with Preconfirmations[^4] revealed that the design space of this framework involves many complex topics and left with several open questions and concerns raised by community. Below are the some of the areas of research and complexities involved: + +- **Suboptimal Block Value**: Preconfirmations could lead to less valuable blocks for validators, as constraints imposed by preconfirmed transactions could limit MEV opportunities. +- **Complexity with Multiple Preconfirms**: Managing and coordinating multiple preconfirms can complicate the execution state and challenge the uniformity of transaction sequencing. +- **Pricing and Economic Incentives**: Determining the right price for preconfirmation tips is complex, as preconfirms could affect the expected MEV and thus the economic incentives for proposers and users. +- **Execution Guarantees**: Variability in the execution guarantees of preconfirms might require different levels of sophistication from proposers, with more complex preconfirms potentially necessitating higher capabilities. +- **Centralization Risks**: Some expressed concerns that the preconfirmation system could lead to centralization, with a few entities controlling the sequence of transactions. +- **Liveness and Safety Faults**: Understanding and implementing the proper response to liveness and safety faults within the system, including the correct attribution of faults and management of associated slashing, is complex. +- **Infrastructure Requirements**: The need for validators to run full nodes, manage bandwidth, and provide Denial-of-Service protection adds to the operational complexity. +- **Collateral Posting**: Managing the posting and efficiency of collateral for preconfirms is a significant consideration, particularly concerning the scaling of collateral relative to the value of transactions. +- **User Experience**: How users experience the process, including the speed and reliability of preconfirmations, and the transparency of the system. +- **Relay Trust**: Trust in relays and their role in the preconfirmation process, considering that relays must balance the interests of various network participants and manage the associated risks. +- **Communication Channels**: Establishing secure and efficient channels for communication between users, proposers, relays, and builders. +- **Lookahead and Selection Mechanisms**: The lookahead mechanism's impact on preconfer selection and whether an alternative selection mechanism would be more advantageous. +- **Layer 1 and Layer 2 Coordination**: Coordinating between Beacon Chain proposers and Layer 2 sequencers, particularly with rollups designating their own sequencers, can be challenging. +- **Legal and Regulatory Considerations**: Potential legal and regulatory implications of the preconfirmation process, especially regarding financial transactions. +- **Technological Adaptability**: The need for the system to adapt to new technologies, like the eventual integration of execution tickets, which could alter the preconfirmation landscape. + +## Resources +- [Ethereum Sequencing](https://docs.google.com/presentation/d/1v429N4jdikMIWWkcVwfjMlV2LlOXSawFCMKoBnZVDNU/) +- [Based preconfirmations](https://ethresear.ch/t/based-preconfirmations/17353) +- [Preconfirmations](/docs/wiki/research/Preconfirmations/Preconfirmations.md) +- [Ethereum Sequencing and Preconfirmations Call #1](https://youtu.be/2IK136vz-PM) +- [Espresso Shared Sequencing](https://hackmd.io/@EspressoSystems/SharedSequencing) +- [Zksync Deposit Sharing](https://docs.zksync.io/zk-stack/components/shared-bridges.html) +- [Polygon Aggregate Layer](https://polygon.technology/blog/aggregated-blockchains-a-new-thesis) + + +## References +[^1]: https://docs.google.com/presentation/d/1v429N4jdikMIWWkcVwfjMlV2LlOXSawFCMKoBnZVDNU/ +[^2]: https://ethresear.ch/t/based-preconfirmations/17353 +[^3]: https://epf.wiki/#/wiki/research/Preconfirmations/Preconfirmations +[^4]: https://youtu.be/2IK136vz-PM +[^5]: https://hackmd.io/@EspressoSystems/SharedSequencing +[^6]: https://docs.zksync.io/zk-stack/components/shared-bridges.html +[^7]: https://polygon.technology/blog/aggregated-blockchains-a-new-thesis diff --git a/docs/wiki/research/Preconfirmations/Preconfirmations.md b/docs/wiki/research/Preconfirmations/Preconfirmations.md new file mode 100644 index 00000000..2e3cd1bd --- /dev/null +++ b/docs/wiki/research/Preconfirmations/Preconfirmations.md @@ -0,0 +1,155 @@ + +# Ethereum Based Preconfirmations + +## [Overview](#overview) + +Based preconfirmations (preconfs) represent a significant advancement in Ethereum transaction processing, offering users swift and reliable execution. Through a combination of on-chain infrastructure, proposer accountability mechanisms, and flexible promise acquisition processes, preconfs stand to significantly enhance the user experience in Ethereum interactions. This technology not only reduces transaction latency but also introduces a layer of security and efficiency previously unseen in the ecosystem[^1]. + + +## [Construction of Preconf Promises](#construction-of-preconf-promises) + +Preconfirmation promises, or "preconfs," rely on two foundational on-chain infrastructure components[^2][^3]: + +- **Proposer Slashing:** Proposers can opt into additional slashing conditions to ensure reliability and accountability. This approach draws inspiration from EigenLayer's model, which employs restaking as a means of enforcing these slashing mechanisms. + +- **Proposer Forced Inclusions:** To ensure the seamless execution of transactions, proposers have the authority to mandate the inclusion of specific transactions on-chain. This power is crucial in situations where the separation between proposers and builders (PBS) renders self-building uneconomical. The implementation of this mechanism typically involves the use of inclusion lists. + + +When a Beacon Chain validator decides to become a "preconfer," they are essentially agreeing to adhere to two distinct slashing conditions related to preconf promises. In return for their service, preconfers issue signed promises to users and are compensated with tips for successfully fulfilling these promises. The hierarchy among preconfers is determined based on their position in the slot order within an epoch, with precedence given to those with earlier slot assignments. + +A transaction that secures a preconf promise gains the eligibility for immediate inclusion and execution on-chain by any proposer positioned before the issuer of the promise (preconfer). The primary obligation of the preconfer is to honor all such promises during their designated slot, utilizing the inclusion list to facilitate this process[^3]. + +There are two main types of promise-related faults, each carrying the potential for slashing: + +1. **Liveness Faults:** These faults occur when a preconfer fails to include a promised transaction on the chain because their designated slot was missed. + +2. **Safety Faults:** These arise when the preconfer includes transactions on-chain that directly contradict the promises made, despite not missing their slot. + + +To ensure that transactions with preconf promises are given priority, a specific execution queue is established for transactions lacking such promises. This arrangement guarantees that preconfed transactions are executed ahead of others. + +Preconfers are not limited to a single type of preconfirmation promise. They can offer a spectrum of promises, ranging from strict execution guarantees based on specific state roots to simpler promises of transaction inclusion. This flexibility allows preconfers to cater to a wide array of user needs and preferences. + + +## [Key Elements of Preconfs](#key-elements-of-preconfs) + +The process towards securing a preconfirmation promise for transactions within the Ethereum network is initiated by establishing a connection with the next available preconfer. This process entails a series of critical steps and factors, including[^1]: + +- **Endpoints:** Preconfers may offer direct API endpoints or utilize decentralized peer-to-peer (p2p) networks for the exchange of promises, striking a balance between quick response times and widespread availability. + +- **Latency:** Utilizing direct communication channels, the process aims to achieve preconfirmation times as swift as 100 milliseconds, ensuring rapid transaction handling. + +- **Bootstrapping:** A substantial participation rate among L1 validators as preconfers is crucial. This ensures that within the proposer lookahead window, there is always a significant chance of encountering a preconfer ready to issue promises. + +- **Liveness Fallback:** Users can enhance their transaction's reliability by securing promises from several preconfers, thus safeguarding against the possibility of any single preconfer failing to fulfill their promise due to missed slots. + +- **Parallelization:** The system accommodates various promise types, ranging from strict commitments on post-execution state to more flexible, intent-based promises. + +- **Replay Protection:** Ensures transactions are protected from replay attacks, vital for maintaining the integrity and security of preconf transactions. + +- **Single Secret Leader Election (SSLE):** This mechanism allows for the confidential identification of preconfers within the lookahead period, enabling them to validate their status without revealing their identity prematurely. + +- **Delegated Preconf:** Offers a provision for proposers constrained by limited network bandwidth or processing capability, allowing them to delegate their preconfirmation duties to ensure efficient processing of promises. + +- **Fair Exchange:** The system addresses the fair exchange dilemma between users and preconfers concerning promise requests and the collection of preconf tips. Solutions include public streaming of promises for transparency, intermediation by trusted relays, or the use of cryptographic fair exchange protocols to balance the interests of all parties involved. + +- **Tip Pricing:** The negotiation of preconf tips considers the transaction's potential impact on a proposer’s ability to extract MEV. Through mutual agreement or the assistance of trusted relays, users and preconfers can determine fair compensation for preconfirmations. + +- **Negative Tips:** Preconfers may accept negative tips for transactions that enhance their MEV opportunities, such as those affecting DEX prices and creating arbitrage prospects. + + +Each of these elements plays a crucial role in the functionality and efficiency of based preconfirmations, ensuring transactions are not only processed swiftly but also securely and fairly within the Ethereum ecosystem. + + +## [Preconfs Acquisition Process Flow](#preconfs-acquisition-process-flow) + + +![Promise acquisition flow](/docs/wiki/research/img/preconfs/Promise-acquisition-flow-JustinDrake.png) + + +*Figure: Preconf Promise acquisition process flow. Source: Justin Drake* + + +*Here is a sequence diagram explaining the interactions in a typical Preconfs Acquisition Process Flow.* + +```mermaid +sequenceDiagram + participant User as User + participant Preconfer as Preconfer + participant Blockchain as Ethereum + + rect rgb(191, 223, 255) + User->>+Preconfer: Identify & send promise request + Note over User,Preconfer: User selects the next preconfer based on slot
position and sends a transaction promise request. + + Preconfer->>-User: Evaluate request + Note over User,Preconfer: Preconfer assesses the request, considering network
conditions, preconf tip, and MEV potential. + end + rect rgb(177,176,159) + Preconfer->>+User: Issue preconf promise + Note over User,Preconfer: If accepted, preconfer sends back a
signed preconf promise to the user. + end + rect rgb(191, 223, 255) + User->>Preconfer: Transfer preconf tip + Note over User,Preconfer: User transfers the agreed preconf tip
to the preconfer as compensation. + end + rect rgb(177,176,159) + Preconfer->>+Blockchain: Include & execute transaction + Note over Preconfer,Blockchain: In the assigned slot, preconfer includes the
transaction in their block proposal for execution. + + Blockchain->>-User: Verify transaction execution + Note over User,Blockchain: Once executed on-chain, both user and preconfer verify the fulfillment of the promise. + end +``` + + +The promise acquisition process in the context of Ethereum's sequencing and pre-confirmation mechanism is a critical aspect, ensuring transactions receive a preconfirmation or "promise" from a proposer or sequencer. This process involves several steps, each integral to securing a promise that a transaction will be included and executed on-chain within a specified time frame. The above figure shows the preconf promise acquisition flow through a sequence of interactions[^2]. Below is a detailed explanation of the acquisition process flow: + +**1. User Identifies Next Preconfer** + +- **Starting Point:** A user or a smart contract initiates the process by identifying the next available preconfer (a proposer who has opted in to provide preconfirmation services) within the Ethereum network's proposer lookahead window. + +- **Selection Criteria:** The selection is based on the proposer's slot position in the proposer lookahead, where proposers have declared their capability and willingness to issue preconfirmations by posting collateral. + +**2. Promise Request Sent to Preconfer** + +- **Initiation:** The user sends a promise request to the identified preconfer. This request includes details of the transaction for which the preconfirmation is sought, along with any specific conditions or requirements. + +- **Communication Channel:** The request can be sent through various off-chain communication channels established by the preconfer, such as a dedicated API endpoint or a peer-to-peer messaging system. + +**3. Preconfer Evaluates the Request** + +- **Assessment:** Upon receiving the request, the preconfer evaluates it based on several factors, including the current network conditions, the preconf tip amount proposed by the user, and the overall risk of executing the transaction. + +- **Decision Making:** The preconfer decides whether to accept or reject the promise request. This decision may involve calculating the potential MEV and assessing whether the transaction aligns with the preconfer's criteria. + +**4. Issuance of Preconf Promise** + +- **Promise Generation:** If the preconfer decides to accept the request, they generate a signed preconf promise. This promise includes the preconfer's commitment to ensuring the transaction's inclusion and execution within their upcoming slot, adhering to the agreed conditions. + +- **Communication of Promise:** The preconf promise is then communicated back to the user, providing them with a guarantee of transaction execution. The communication method used is like that of the initial request, ensuring secure and verifiable delivery. + +**5. Payment of Preconf Tip** + +- **Tip Transfer:** Upon receipt of the preconf promise, the user transfers the agreed preconf tip to the preconfer. This tip serves as compensation for the service provided and incentivizes the preconfer to honor the commitment. + +- **Escrow Mechanisms:** In some implementations, the tip may be held in escrow until the promise is fulfilled, adding an extra layer of security for the user. + +**6. Inclusion and Execution of Transaction** + +- **On-Chain Fulfillment:** The preconfer includes the preconfirmed transaction in their proposed block during their assigned slot, executing it according to the terms outlined in the preconf promise. + +- **Verification of Fulfillment:** Once the transaction is included and executed on-chain, both the preconfer and the user can verify that the promise has been fulfilled, completing the process. + + +**Additional Considerations:** + +- **Fallback Mechanisms:** In case of unexpected issues or if the first preconfer fails to include the transaction, users may have fallback options, such as requesting promises from multiple preconfers in parallel. + +- **Dispute Resolution:** The system may include mechanisms for resolving disputes in cases where there's disagreement about whether the promise was adequately fulfilled. + + +## References +[^1]: https://ethresear.ch/t/based-preconfirmations/17353 +[^2]: https://www.youtube.com/watch?v=2IK136vz-PM +[^3]: https://notes.ethereum.org/@JustinDrake/rJ2eXRcKa diff --git a/docs/wiki/research/img/preconfs/Preconfirmations-Acquisition-Flow.png b/docs/wiki/research/img/preconfs/Preconfirmations-Acquisition-Flow.png new file mode 100644 index 00000000..8ab79710 Binary files /dev/null and b/docs/wiki/research/img/preconfs/Preconfirmations-Acquisition-Flow.png differ diff --git a/docs/wiki/research/img/preconfs/Promise-acquisition-flow-JustinDrake.png b/docs/wiki/research/img/preconfs/Promise-acquisition-flow-JustinDrake.png new file mode 100644 index 00000000..dc8ea3e6 Binary files /dev/null and b/docs/wiki/research/img/preconfs/Promise-acquisition-flow-JustinDrake.png differ diff --git a/docs/wiki/research/img/preconfs/based-sequencing-problems-space.png b/docs/wiki/research/img/preconfs/based-sequencing-problems-space.png new file mode 100644 index 00000000..af56229a Binary files /dev/null and b/docs/wiki/research/img/preconfs/based-sequencing-problems-space.png differ diff --git a/docs/wiki/research/img/preconfs/lookahead-preconfs.png b/docs/wiki/research/img/preconfs/lookahead-preconfs.png new file mode 100644 index 00000000..b186d14d Binary files /dev/null and b/docs/wiki/research/img/preconfs/lookahead-preconfs.png differ diff --git a/docs/wiki/research/img/preconfs/united-chains-of-ethereum.jpg b/docs/wiki/research/img/preconfs/united-chains-of-ethereum.jpg new file mode 100644 index 00000000..e4c41b96 Binary files /dev/null and b/docs/wiki/research/img/preconfs/united-chains-of-ethereum.jpg differ diff --git a/wordlist.txt b/wordlist.txt index c723be9f..0f3e4164 100644 --- a/wordlist.txt +++ b/wordlist.txt @@ -473,4 +473,4 @@ XMSS XORed xy Yellowpaper -zk +zk \ No newline at end of file