Ocelloids XCM Transfer Monitoring Service

[mfornos]

Project Abstract

This grant proposal is a follow-up to the Ocelloids Monitoring SDK, previously delivered and available here: w3f/Grant-Milestone-Delivery#934

The objective of this grant is to develop an open-source monitoring service using the Ocelloids SDK. This service will monitor XCM transfers across selected parachains. The primary purpose is to offer service providers integrating with a single chain (Asset Hub as starting point) and monitoring effects on other chains that are connected via HRMP and that use XCM as their message format. The service will support connecting to the configured networks through light clients in order to reduce infrastructure overhead for service providers. Users will have access to a self-hosted HTTP API to subscribe to XCM transfers and manage their subscriptions. A public Docker image will be published to facilitate service deployment.

Grant level

• Level 1: Up to $10,000, 2 approvals
• Level 2: Up to $30,000, 3 approvals
• Level 3: Unlimited, 5 approvals (for >$100k: Web3 Foundation Council approval)

Application Checklist

• The application template has been copied and aptly renamed (project_name.md).
• I have read the application guidelines.
• Payment details have been provided (bank details via email or BTC, Ethereum (USDC/DAI) or Polkadot/Kusama (USDT) address in the application).
• The software delivered for this grant will be released under an open-source license specified in the application.
• The initial PR contains only one commit (squash and force-push if needed).
• The grant will only be announced once the first milestone has been accepted (see the announcement guidelines).
• I prefer the discussion of this application to take place in a private Element/Matrix channel. My username is: @_______:matrix.org (change the homeserver if you use a different one)

[github-actions]

CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅

[mfornos]

I have read and hereby sign the Contributor License Agreement.

[Noc2]

Thanks a lot for the application. I'm happy to go ahead with it, but let me also ping @joepetrowski in case he has any additional input.

[nikw3f]

Thankyou very much for the application. This is very much needed for the XCM supporting chains. I have a few questions:

• Have you considered scalability challenges with the monitoring service, especially when the number of XCM transfers significantly increases?
• Will there be any user-friendly interface or dashboard where users can easily track their XCM transfers, or is this strictly an API-based solution?
• How does the team plan to manage changes or updates to the supported networks, XCM protocol spec can change in future?

[mfornos]

Hi @nikw3f

to answer your questions:

Have you considered scalability challenges with the monitoring service, especially when the number of XCM transfers significantly increases?

Regarding the number of transfers, we do not foresee it causing scalability problems. This is because their rate is constrained by the block production of the full node with lowest block time, which runs on a single machine and is computationally more demanding than a light client. So the primary consideration would be ensuring that the hardware is properly dimensioned to accommodate the load.

However, the number of light clients running within a single service instance could pose scalability issues. To address this concern, a straightforward solution would be to deploy multiple service instances, each handling a specific set of networks.

Will there be any user-friendly interface or dashboard where users can easily track their XCM transfers, or is this strictly an API-based solution?

The primary objective of this grant is to provide an API for subscribing to XCM transfer events related to accounts of interest. This API is designed to serve custodians, exchanges, and institutions integrating with Asset Hub, enabling them to receive notifications via a web hook. These are the requirements that were provided to us for this specific use case.

However, implementing a user-friendly UI could be considered if there is interest from an end-user perspective.

How does the team plan to manage changes or updates to the supported networks, XCM protocol spec can change in future?

As an open-source project, our ideal approach is to encourage interested parties to contribute by updating the runtime specifications in our repository. Alternatively, we are prepared to make these updates ourselves as needed.

In any case, we are committed to ensuring that our code remains compatible with any future updates to the XCM protocol. If there are changes or updates to the protocol version, we will take the necessary steps to adapt our code accordingly.

[nikw3f]

Thanks. Happy to approve.

[joepetrowski]

I support the concept but a few things left to be clarified. Namely:

1. Must be light-client friendly, not just "if it's not too hard". I'd rather see the grant fail to meet its delivery timeline but have documented light client issues for the light client developers to solve than to meet the timeline but without light client support.
2. It's not entirely clear if you plan to run this as a paid subscription service or if you plan to release docker images or binaries that others can run themselves. The latter is a requirement IMO.

Also, for clarity I generally prefer "XCM program" over "XCM message", since if you spell out the acronym into "cross-consensus message format {message, program}" you see why.

[mfornos]

Hi @joepetrowski,
Thank you for the feedback.

We have updated the application incorporating your suggestions and hopefully clarifying the doubts.

Regarding your concerns:

Must be light-client friendly, not just "if it's not too hard". I'd rather see the grant fail to meet its delivery timeline but have documented light client issues for the light client developers to solve than to meet the timeline but without light client support.

We will focus on Asset Hub, Astar and Acala since they are already compatible with smoldot.

It's not entirely clear if you plan to run this as a paid subscription service or if you plan to release docker images or binaries that others can run themselves. The latter is a requirement IMO.

We plan to publish public docker images for the service, as included in the deliverables.

[joepetrowski]

@mfornos thanks for clarifications, better now.

[mfornos]

@joepetrowski we've updated the document to add asset teleports support and some minor fixes.

[github-actions]

Congratulations and welcome to the Web3 Foundation Grants Program! Please refer to our Milestone Delivery repository for instructions on how to submit milestones and invoices, our FAQ for frequently asked questions and the support section of our README for more ways to find answers to your questions.

Before you start, take a moment to read through our announcement guidelines for all communications related to the grant or make them known to the right person in your organisation. In particular, please don't announce the grant publicly before at least the first milestone of your project has been approved. At that point or shortly before, you can get in touch with us at [email protected] and we'll be happy to collaborate on an announcement about the work you’re doing.

Lastly, please remember to let us know in case you run into any delays or deviate from the deliverables in your application. You can either leave a comment here or directly request to amend your application via PR. We wish you luck with your project! 🚀