Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cyborg Network Grant Application #1933

Closed
wants to merge 4 commits into from
Closed

Conversation

SHA888
Copy link
Contributor

@SHA888 SHA888 commented Aug 26, 2023

Project Abstract

Cyborg Network is aimed at pioneering a novel approach to decentralized cloud computing within the Polkadot Ecosystem by establishing a marketplace that harnesses crowdsourced infrastructure. As a premier venture of this nature in the Polkadot landscape, our network effortlessly allows providers to integrate their machines, rewarding them for their invaluable participation.

Grant level

  • Level 1: Up to $10,000, 2 approvals
  • Level 2: Up to $30,000, 3 approvals
  • Level 3: Unlimited, 5 approvals (for >$100k: Web3 Foundation Council approval)

Application Checklist

  • The application template has been copied and aptly renamed (project_name.md).
  • I have read the application guidelines.
  • Payment details have been provided (bank details via email or BTC, Ethereum (USDC/DAI) or Polkadot/Kusama (USDT) address in the application).
  • The software delivered for this grant will be released under an open-source license specified in the application.
  • The initial PR contains only one commit (squash and force-push if needed).
  • The grant will only be announced once the first milestone has been accepted (see the announcement guidelines).
  • I prefer the discussion of this application to take place in a private Element/Matrix channel. My username is: @_______:matrix.org (change the homeserver if you use a different one)

* copy application template

* add overview and team section

* Update diagrams

* add missing information

* fix missing urls

* update cover image

* add development roadmap and future plans

* corrections

* finalize changes

* fix url linkage

* update competitor links

* update additional information

* remove unecessary link

* Update overview and dev roadmap

* Fix bullets

* update payment address

---------

Co-authored-by: Kresna Sucandra <[email protected]>
@github-actions
Copy link
Contributor

github-actions bot commented Aug 26, 2023

CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅

@SHA888 SHA888 changed the title Proposal (#1) Cyborg Network Grant Application Aug 26, 2023
@SHA888
Copy link
Contributor Author

SHA888 commented Aug 26, 2023

I have read and hereby sign the Contributor License Agreement.

@SHA888
Copy link
Contributor Author

SHA888 commented Aug 26, 2023

recheck

@beekay2706
Copy link
Contributor

I have read and hereby sign the Contributor License Agreement.

@beekay2706
Copy link
Contributor

recheck

Copy link
Collaborator

@Noc2 Noc2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for the application. I have one initial question: How do you plan to decentralize the databases and storage solutions while ensuring that the data is available?

@Noc2 Noc2 added the changes requested The team needs to clarify a few things first. label Aug 28, 2023
@beekay2706
Copy link
Contributor

Thanks a lot for the application. I have one initial question: How do you plan to decentralize the databases and storage solutions while ensuring that the data is available?

For this grant, our main objective is to enhance the blockchain components to ensure the system's efficiency. In the initial stages, we will host the database ourselves to guarantee data availability, while CyberHub will be centralized. This will let us prioritize user experience. As we progress, we will transition these centralized components to more decentralized methods, such as:

To begin with, CyberHub will be hosted on Cyborg's crowdsourced servers as a step towards decentralization. After the launch, we intend to develop our own decentralized dynamic relational database, named Polkasql. This move aims to establish a fully decentralized system that can effectively address real-world issues.

This decision is based on advice from Ben Fielding, Co-founder of Gensyn, given during one of his sessions at the PBA.

Copy link
Collaborator

@Noc2 Noc2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the quick reply. I will mark the application as ready for review. But could you provide more details on how you plan to decentralize your setup? I personally think that this is the main problem to solve, otherwise, it makes in my opinion not much sense to use blockchain in the first place.

@Noc2 Noc2 added ready for review The project is ready to be reviewed by the committee members. and removed changes requested The team needs to clarify a few things first. labels Aug 29, 2023
@beekay2706
Copy link
Contributor

Thanks for the quick reply. I will mark the application as ready for review. But could you provide more details on how you plan to decentralize your setup? I personally think that this is the main problem to solve, otherwise, it makes in my opinion not much sense to use blockchain in the first place.

Thank you so much for the quick response,

To tackle the issue of decentralization, we've outlined a multi-faceted plan that we believe will effectively address the core concerns. Our plan centers around several key principles:

Geotagged Instances: One of our central ideas involves the division of Cyberhub into geotagged instances. Each instance will be deployed on an edge server located within the same geographical region as the users it serves. This approach has a twofold advantage: it helps mitigate latency issues by minimizing the distance between the user and the server, thereby enhancing user experience; and it lays the foundation for decentralization by distributing the system's nodes across diverse locations.

Collator Nodes for Quick Response: To ensure swift response times, we intend to position a Collator node in close proximity to each Cyberhub instance. This configuration will optimize the processing of transactions and data, resulting in an improved overall system performance. The presence of Collator nodes near every Cyberhub instance is integral to achieving the quick and efficient response times that users expect.

Direct Pallet Access for Database: Our commitment to decentralization extends to the database architecture as well. A comprehensive migration of our database is in the works, aimed at enabling direct access to data from the pallets. This migration will eliminate intermediaries like Cyberhub, thereby enhancing data availability and contributing to the decentralization objective. Users will be able to interact with the blockchain's data in a more direct and distributed manner.

Edge Server Considerations: It's important to note that while our edge servers (hosted by providers) hold potential, they also present challenges due to their non-deterministic nature. Acknowledging this, we're exploring a cautious approach. We're contemplating allocating a portion of Cyberhub's functionality to collator node operators. This allocation will serve as a preliminary testing phase, enabling us to gather real-world insights and assess the viability of a full migration to edge servers.

These are some of the strategies that we have planned at the moment, but I firmly believe that the solution will improve a lot as we come closer to building these and testing for real world ready uses cases

Copy link
Member

@semuelle semuelle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @SHA888. Thanks for the application. To add to what David already said, it seems quite ambitious to build all these components, a complex communication protocol and to onboard providers and users and then to decentralise the components. It might make more sense to start this well decentralised from the start. In the current architecture, the parachain is the only decentralised bit and its role is minor. A couple more questions:

  • You want to deliver CyberHub as an open source project, but the instances will all run be run by you?
  • How do the pallets interact with the CyberHub, as suggested in deliverable 5? It sounds more like the backend would have to pull data from the blockchain.
  • What does 'facilitate uninterrupted communication' (deliverable 1) mean exactly? I don't see how broadcasting events does that.
  • How is the database 'streamlined'? What does this mean from a technical perspective?
  • What do you mean by 'direct access to data from the pallets' with regard to the database? What does this mean from a technical perspective?

@beekay2706
Copy link
Contributor

I appreciate your thought-provoking follow-up questions.

I've provided responses to the queries you raised, which I hope have offered a clearer understanding of our approach. We are highly receptive to seeking development advice to enhance our entire roadmap further. Your insights are invaluable in shaping our ongoing development.

Starting Well-Decentralized from the Beginning: You raised a valid point about the ambition of our project and the order of its components. We appreciate your perspective, and it's a consideration we've carefully weighed. While the initial stages of our project involve building various components and establishing a communication protocol, our ultimate vision is indeed to achieve a high level of decentralization. We share your belief that starting decentralized from the outset can be advantageous. However, building the entire infrastructure in a completely decentralized manner can be technically challenging and may also pose adoption hurdles. Hence, we've chosen a phased approach, starting with centralized instances that will gradually transition to a more decentralized setup as we gain insights and confidence in the system's performance.

Open Source Project with Centralized Instances: Yes, our intent is to make CyberHub an open-source project to foster community collaboration. However, in the initial stages, the instances will be operated by us to ensure the stability and reliability of the network. The transition to a more decentralized model will be a gradual process that aligns with our overall strategy

Pallet Interaction with CyberHub: In our current design, Pallets emit events that are captured by Cyberhub to trigger the necessary actions. Since Cyberhub also operates as an oracle, it can post the outcomes back onto the blockchain for validation by the Pallets. This bidirectional interaction ensures seamless coordination between Pallets and Cyberhub. We are open to suggestions on improving this outlook.

'Facilitate Uninterrupted Communication': Within our context, this phrase pertains to the health monitoring of provider machines. We achieve this by periodically sending random checks and verifying their responses. The Pallet Edge Connect component instructs Cyberhub to dispatch ping messages to all Cyborg Smart Client instances. The responses are then recorded on the blockchain for verification, ensuring the continuous and reliable operation of provider machines.

Streamlined Database: From a technical standpoint, a streamlined database involves optimizing its structure and access methods. This optimization encompasses the use of efficient indexing, data compression techniques, and data organization strategies. These measures collectively serve to reduce latency and resource consumption, ultimately enhancing the overall performance of the database.

Direct Access to Data from Pallets: In technical terms, enabling direct access to data from Pallets implies that the Pallets within the blockchain have the capability to interact directly with the off-chain database. In further phase, this interaction allows them to retrieve and manipulate data without the need for

@semuelle semuelle self-requested a review September 13, 2023 14:51
Copy link
Member

@semuelle semuelle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the thorough reply and sorry for the long wait. We have a bit of a backlog, so it takes a bit of time to circle back to ongoing discussions.

I'm struggling to interpret the diagrams in the application and the architecture.

  • What do you need an oracle for?
  • Does this oracle run alongside your server? But the CyberHub could be run by anyone, correct?
  • What/who is Platform in the 'Provider Connection' diagram?
  • Isn't Inventory stored on-chain?

Perhaps you could add some example message diagrams for common functions to make it a bit more clear what's happening where and through what/whom.

@beekay2706
Copy link
Contributor

Apologies for the delay,

Thank you for your follow-up inquiry. It is imperative for us to provide a comprehensive overview of our application to ensure a clear understanding of all its components. Allow me to elucidate this with an illustrative scenario:

Imagine two individuals, Alice and Bob, each with distinct professional backgrounds. Alice, a software developer, resides in the same geographical area as Bob, the owner of a real-time data processing software company. Alice possesses an unused computer, while Bob requires an edge server to execute real-time data processing tasks for his clientele.

Cyborg Network offers a platform that brings entities like Alice and Bob together within this scenario. Users access our network through our platform, an application layer integrated with the Cyborg parachain. This platform comprises two distinct interfaces, akin to the seller and buyer interfaces on platforms like Amazon, each tailored to meet specific requirements.

Alice, assuming the role of a provider, accesses the platform by connecting her Decentralized Identity (Kilt DID) and navigating to the provider end. Here, she can download the Cyborg Smart Client (CSC) binary and execute it within her chosen virtual machine (VM). Upon execution, the CSC establishes a WebSocket connection with the nearest Cyberhub instance, acting as the conduit between the blockchain and the off-chain computing realm. The CSC then transmits Alice's server information to the Cyberhub, which forwards it to the blockchain for verification. Upon successful verification, the blockchain emits an event instructing the Cyberhub to add Alice's machine to the off-chain database. This decision to maintain the database off-chain is strategic, as it prevents overloading the substrate RocksDB with an excessive number of calls, considering the anticipated exponential scaling over time. Alice's machine is now part of the inventory, ready for hosting.

Bob, the prospective customer, accesses the platform from the customer interface. Here, he connects his Decentralized Identity (DID) and articulates his specific requirements. The platform guides him to Alice's machine, the ideal solution in this scenario. Bob reviews the machine's specifications and performance score on the platform and proceeds to upload his execution binaries for deployment on Alice's machine.

At this juncture, Cyberhub steps in, transmitting Bob's command to the locally running CSC on Alice's machine. The binary executes, and the CSC relays the computation's result back to the Cyberhub. The Cyberhub, in turn, records this outcome on the blockchain using the trusted Oracle housed within the Cyberhub instances. Initially, we oversee this trusted Oracle, but our long-term plan involves extending this responsibility to our collator node operators and the broader public, contingent upon their trust scores, reflecting their network activities.

Subsequently, on the platform, Bob can verify the computation results and grant approval. He then initiates payment for the service rendered, while Alice receives a reward for her valuable contribution to the network.

We genuinely appreciate any suggestions to refine and enhance this workflow. We would also be happier to explain the whole flow in a call if needed.

@semuelle semuelle self-requested a review September 27, 2023 12:52
Copy link
Member

@semuelle semuelle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @SHA888 & @beekay2706. Sorry for the long wait, we are a bit swamped with applications at the moment.

For this grant, our main objective is to enhance the blockchain components to ensure the system's efficiency

Do you mean "enhance" or "build"? The application also covers a backend server and database, so it's a bit confusing.

After the launch, we intend to develop our own decentralized dynamic relational database, named Polkasql. This move aims to establish a fully decentralized system that can effectively address real-world issues.

Do you have any more information to share about this database and which "real-workd issues" aren't solved by existing solutions?

we intend to position a Collator node in close proximity to each Cyberhub instance

I assume this is related to the initial setup with Cyberhubs run by you? Or do you plan to position collator nodes near anyone running a Cyberhub?

our edge servers (hosted by providers) hold potential, they also present challenges due to their non-deterministic nature

Can you elaborate on this bit?

@beekay2706
Copy link
Contributor

Hi @SHA888 & @beekay2706. Sorry for the long wait, we are a bit swamped with applications at the moment.

For this grant, our main objective is to enhance the blockchain components to ensure the system's efficiency

Do you mean "enhance" or "build"? The application also covers a backend server and database, so it's a bit confusing.

After the launch, we intend to develop our own decentralized dynamic relational database, named Polkasql. This move aims to establish a fully decentralized system that can effectively address real-world issues.

Do you have any more information to share about this database and which "real-workd issues" aren't solved by existing solutions?

we intend to position a Collator node in close proximity to each Cyberhub instance

I assume this is related to the initial setup with Cyberhubs run by you? Or do you plan to position collator nodes near anyone running a Cyberhub?

our edge servers (hosted by providers) hold potential, they also present challenges due to their non-deterministic nature

Can you elaborate on this bit?

We extend our gratitude for your dedicated time and careful consideration of our grant application. Allow me to provide comprehensive responses to your queries, offering further insight into our objectives:

1. Enhancement of Blockchain Components: When we refer to "enhance" in the context of our blockchain components, we intend to convey the notion that these components harmoniously integrate with off-chain elements like Cyberhub and the database. While we comprehend how these systems operate independently, the true value of integration with the blockchain lies in the seamless synergy it brings to our system.

2. Polkasql Database and Addressing Existing Gaps: Currently, within our ecosystem, we have not identified a dynamic SQL-like database solution that adequately caters to our specific business cases. Our aim is to promote decentralized dynamic storage solutions, addressing a gap where existing solutions built upon IPFS often lack practical application. For instance, we encounter challenges when attempting to efficiently store and manage data from an E-commerce website using existing storage solutions within the Polkadot ecosystem. Polkasql is designed to fill this void by providing a purpose-built solution. Its implementation eliminates our reliance on a centralized database instance to manage our business logic, aligning with our overarching goal of decentralization and optimizing the user experience.

3. Management and Distribution of Cyberhub Oracle Instances: Regarding the management of the Cyberhub Oracle instances, we acknowledge the importance of its role as a trusted data source for pushing data into the blockchain. Initially, we plan to manage these instances ourselves and gradually distribute them in phases. To ensure efficient distribution, we have devised two strategies:

a. Allocation to Trusted Third-Party Node Operators: We intend to allocate Cyberhubs to trusted third-party node operators, fostering decentralization within the network.

b. Reputation-Based Allocation: In our network, all accounts will be associated with their KILT Decentralized Identities (DIDs). We will meticulously track historical network activity and behavior, based on which we will assign reputation scores. Over time, say, within a year, we will allocate Cyberhub instances to parties with higher reputation scores. This serves as a measure to establish decentralized trust within the system.

4. Non-Deterministic Nature and Ensuring Smooth Operations: The term "non-deterministic nature" denotes the inherent unpredictability of providers who might engage in detrimental behavior within the system. For instance, providers might abruptly turn off their machines while an ongoing process is active, potentially resulting in losses for customers. To mitigate such adversarial behavior, we will employ game-theoretic concepts as a means of ensuring the continued smooth operation of the network.

Copy link
Member

@semuelle semuelle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the clarifications, @beekay2706. I am a bit skeptical about using "trusted third parties" and the whole "decentralise-later" approach, especially given the lack of technical details around the future plans. I will, however, share this again with the rest of the committee to see if they have other opinions or questions.

@nikw3f
Copy link
Contributor

nikw3f commented Oct 10, 2023

Hey team. I had few queries: Given that providers are integrating their machines and being leveraged as servers to cater to customer demands, what measures are in place to ensure the security and privacy of both the providers' machines and the data being processed? Additionally, can you elaborate on the "protected computing environment" you're developing? How does Cyborg Network differentiate itself from competitors like phala network in terms of both technical architecture and the value proposition for end users?

@beekay2706
Copy link
Contributor

Thanks for the clarifications, @beekay2706. I am a bit skeptical about using "trusted third parties" and the whole "decentralise-later" approach, especially given the lack of technical details around the future plans. I will, however, share this again with the rest of the committee to see if they have other opinions or questions.

Thanks @semuelle We are open to receiving suggestions that can enhance our approach. One reason we are leaning toward this approach is because many of the potential customers we've engaged with so far have limited or no experience with web 3.0 applications. Consequently, we are taking steps to provide a user-friendly interface reminiscent of web 2.0, to ensure a smooth user experience while we diligently work on building a decentralized system from the ground up.

Additionally, it's worth noting that we are currently operating with a relatively small team due to funding constraints, which adds up to the need for a pragmatic, phased approach that prioritizes usability and gradually transitions towards full decentralization to accommodate the learning curve and resource limitations.

@beekay2706
Copy link
Contributor

Hey team. I had few queries: Given that providers are integrating their machines and being leveraged as servers to cater to customer demands, what measures are in place to ensure the security and privacy of both the providers' machines and the data being processed? Additionally, can you elaborate on the "protected computing environment" you're developing? How does Cyborg Network differentiate itself from competitors like phala network in terms of both technical architecture and the value proposition for end users?

Hey @nikw3f , Thanks for the questions

I hope the following explanation gives you a better context of our approach, I'm also happy to clarify any more concerns that might arise henceforth.

Providers are required to download and run our Cyborg Smart Client (CSC) software with root privileges within the virtual machine they wish to allocate to the network. The CSC establishes an isolated sandboxed environment that restricts the provider's access to the customer's computing resources. Furthermore, we are exploring the utilization of Trusted Execution Environments (TEE) to enhance the overall security of the system.

Providers are additionally required to stake a bond, which is linked to the Machine NFTs of the computers they connect. In the event that a provider intentionally shuts down a machine, the bond will be deducted or "slashed" to compensate the customer accordingly.

It's worth noting that while Phala Network primarily focuses on offering off-chain computing capabilities for smart contracts in the web 3.0 Dapps space, our emphasis is on creating a system that can be utilized by any web 2.0 application. This approach is aligned with the needs of a significant portion of our potential user base, who are predominantly in the web2 sector.

In the realm of competitors, Akash Network, from the Cosmos ecosystem, stands out as a closer contender. However, despite their strong token economy and valuation, they face scalability challenges due to their reliance on Cosmos and also struggle with user experience issues.

Cyborg Network has a long term vision vision of establishing a decentralized edge computing network. To realize this vision, we recognize the importance of developing a seamless cloud network.

To illustrate a potential future use case, we've engaged in discussions with a company specializing in operating security cameras at traffic signals. They use AI systems to detect seat belt usage among drivers and enforce penalties for inappropriate behavior while driving. Presently, they process data by sending it to a cloud server, which works for their current needs. However, they are incurring high costs due to long-distance data transfers. In the future, they aim to expand their capabilities to handle more data points in real-time, such as scanning faces and comparing them to potential criminals in police databases. Achieving this requires edge servers located near the cameras to process data instantly and send alerts promptly. Moreover, it helps mitigate the substantial costs associated with transferring data to the cloud. Traditional web 2.0 cloud solutions present challenges in establishing and managing servers worldwide, making blockchain-based edge computing an ideal solution. It allows for the seamless crowdsourcing of servers in a trustless and secure manner, making our product particularly appealing to them.

@takahser
Copy link
Collaborator

@SHA888 Personally, I agree with the overall sentiment that it's unclear how you're planning to decentralize Cyborg in the future, especially when it comes to the data storage components of the system. This is a risk I'm not willing to take, hence I've decided not to approve this proposal either. Furthermore, the W3F Grants Committee has discussed your proposal internally and unanimously decided not to go ahead with it.
Nevertheless, we wish you much success with your project. We thank you for your interest in our program and welcome you to apply again in the future.

@takahser takahser closed this Oct 12, 2023
@beekay2706 beekay2706 mentioned this pull request Mar 26, 2024
12 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready for review The project is ready to be reviewed by the committee members.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants