Update Melodot Milestone 2

[DarkingLee]

Project Abstract

Melodot is an incentive-compatible data availability layer. Melodot introduces the role of farmers by combining PoSpace, alleviating the system's dependence on the minimum number of honest nodes assumption, and completing an incentive-compatible distributed generation and data storage scheme. Consensus validators now act more like light clients, improving future scalability.

Grant level

• Level 1: Up to $10,000, 2 approvals
• Level 2: Up to $30,000, 3 approvals
• Level 3: Unlimited, 5 approvals (for >$100k: Web3 Foundation Council approval)

Application Checklist

• The application template has been copied and aptly renamed (project_name.md).
• I have read the application guidelines.
• Payment details have been provided (bank details via email or BTC, Ethereum (USDC/DAI) or Polkadot/Kusama (USDT) address in the application).
• The software delivered for this grant will be released under an open-source license specified in the application.
• The initial PR contains only one commit (squash and force-push if needed).
• The grant will only be announced once the first milestone has been accepted (see the announcement guidelines).
• I prefer the discussion of this application to take place in a private Element/Matrix channel. My username is: @_______:matrix.org (change the homeserver if you use a different one)

[github-actions]

CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅

[DarkingLee]

Dear W3f Committee,

We deeply apologize for the delay in our delivery. At present, we have completed the majority of the work. Through the actual development process, we've made some changes to our initial milestone design. We hope to receive your understanding and approval.

In our initial design, validators were set to directly process user-submitted data during runtime. To more swiftly approach our goal and ensure compatibility with future versions, we altered our design. Now, users only need to submit the metadata of the data, while validators will fetch the actual data from the DHT network and verify its correctness (in the next milestone, validators will only sample). We no longer have the necessity to handle large amounts of data and polynomial commitments during runtime; instead, verification is performed within the node. As a result, we have streamlined the extensions for frame-executive and frame-system, implementing most functionalities within the node and melo_store. Specifically, we made the following major modifications:

1. Initially, our design relied on the frame-executive. However, we found that this approach wasn't worthwhile. We managed to achieve our goals with minimal modifications to the original code. Additionally, we incorporated test support for the new block headers, ensuring that all existing test cases continue to function properly.
2. system_ext is no longer responsible for generating and verifying data availability; it only needs to create new block headers.
3. melo_store has been enhanced to allow validators to conduct a preliminary availability check on data availability transactions in the current block through OCW.
4. The node offers an RPC interface permitting users to submit both availability data and metadata. Once the node verifies that the commitments and proofs of the metadata align with the data, it will publish the data to the DHT and push the transaction into the transaction pool.
5. We have removed multi-roles from Docker, as it seems unnecessary for the current milestone.

[DarkingLee]

I have read and hereby sign the Contributor License Agreement.

[takahser]

@DarkingLee thanks for the update. The reasoning for the amendments sounds logical. However, I feel like the scope of the milestone might have been somewhat reduced by these modifications. For example, by avoiding large data handling on-chain and moving validations off-chain, has this reduced the complexity of the on-chain logic/pallets required? Or with multi-role Docker setup removed, does this reduce the initial setup and configurations required?

[DarkingLee]

Hi @takahser. Thank you for your prompt comment. In fact, these changes have expanded the scope of the milestone. In the original design, the pallet only needed to process user data through RuntimeCall. After decoupling data and transactions, we've added RPC in the node to accept data and transactions, monitor and handle the transaction pool, push and retrieve data from the DHT network, and handle off-chain storage. The melo_store has added off-chain workers to manage data availability, and allowing validators to report unavailable data. Both the melo_store pallet and the node have become considerably more complex. Although the complexity of the Docker configuration has indeed been slightly reduced, it's moderate. Overall, the intricacy and logic of the milestone delivery have increased significantly compared to before.

[takahser]

@DarkingLee thanks for the response. I'm approving it and relay it to the rest of the committee so they can have a look as well. 👍