Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add mech tool for TEE AI agent #250

Merged
merged 5 commits into from
Sep 11, 2024

Conversation

tolak
Copy link
Contributor

@tolak tolak commented Sep 9, 2024

Proposed changes

This PR contains the 1st phase integration of adding trusted execution environment (TEE) AI agent to Valory mech hub. A TEE AI agent represents a program that running inside TEE (more specifically, Intel SGX), to provide the verifiable AI computation back and forth. Here is a diagram to show how the workflow would be like:

image

This PR implement a custom tool that can forward the openAI request to Phala TEE AI agent contract, where the latter then forward the request to openAI to get the LLM response. The workflow behind can be described as:

  • User choose the dedicated tool and send the prompt to Mech service
  • In the tool, forward the request to Phala TEE worker (can be accessed by URL https://wapo-testnet.phala.network/ipfs/QmeUiNKgsHiAK3WM57XYd7ssqMwVNbcGwtm8gKLD2pVXiP) through the http request
  • Inside the TEE worker, after fetching the agent contract (an example can be found at here) deployed on IFPS in advance, the worker then execute the contract.
  • The Agent contract being executed then request OpenAI with the given API key, which is passed from Mech service

[TODO in 2nd phase integration] The tool give users the option to access openAI service through TEE, which can bring verifiability to the mech ecosystem. However, we still rely on the service provided by openAI and the user data (like prompt) must expose to them, which didn't bring full privacy for users. To solve this, we will provide the GPU TEE network in a short future to host LLM model, at that time, the user request can be encrypted and send to TEE, and get the encrypted response with user's account key.

image

Fixes

If it fixes a bug or resolves a feature request, be sure to link to that issue.

Types of changes

What types of changes does your code introduce? (A breaking change is a fix or feature that would cause existing functionality and APIs to not work as expected.)
Put an x in the box that applies

  • Non-breaking fix (non-breaking change which fixes an issue)
  • Breaking fix (breaking change which fixes an issue)
  • Non-breaking feature (non-breaking change which adds functionality)
  • Breaking feature (breaking change which adds functionality)
  • Refactor (non-breaking change which changes implementation)
  • Messy (mixture of the above - requires explanation!)

Checklist

Put an x in the boxes that apply.

  • I have read the CONTRIBUTING doc
  • I am making a pull request against the main branch (left side). Also you should start your branch off our main.
  • Lint and unit tests pass locally with my changes
  • I have added tests that prove my fix is effective or that my feature works

Further comments

If this is a relatively large or complex change, kick off the discussion by explaining why you chose the solution you did and what alternatives you considered, etc...

@tolak tolak changed the title Add mech tool for TEE ai agent Add mech tool for TEE AI agent Sep 9, 2024
packages/valory/customs/tee_openai_request/__init__.py Outdated Show resolved Hide resolved
engine = tool.replace(PREFIX, "")

params = {
"openaiApiKey": api_key,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is quite risky, and would mean that we expose the agent's OpenAI API key to an external service.
Any way we can avoid this?

Copy link
Contributor Author

@tolak tolak Sep 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is safe, here is why:

  • The "external service" here is the AI Agent contract, the execution of the contract is verifiable and guaranteed by consensus system of Phala blockchain, like the view of smart contract, we trust the behavior by trust the code here. Check the doc if you are interested in how it work
  • The execution is happening inside TEE, in other words, happen in an isolated area in the CPU and nobody (including the host OS, hypervisor, etc) can see the context. The key is private and only know for the AI agent contract

@0xArdi
Copy link
Collaborator

0xArdi commented Sep 11, 2024

@tolak to resolve failing checks, can you run:

autonomy packages lock

Choose dev on the following prompt:

[INFO] A new package found with package ID (custom, valory/tee_openai_request:0.1.0)
Select package type (dev, third_party): dev

@0xArdi 0xArdi merged commit ac2bd3a into valory-xyz:main Sep 11, 2024
6 of 7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants