Support current workspace in http proxy (#691)

* support _token and remove prefix in login

* allow pass workspace when login

* Support current workspace for http endpoint

* fix login optional

* increase page size for list workspaces

* Bump version for hypha-rpc 0.20.38

* Update change logs and login instructions

CHANGELOG.md

@@ -1,5 +1,13 @@
 # Hypha Change Log
 
+### 0.20.38
+ - Allow passing workspace and expires_in to the `login` function to generate workspace specific token.
+ - When using http endpoint to access the service, you can now pass workspace specific token to the http header `Authorization` to access the service. (Previously, all the services are assumed to be accessed from the same service provider workspace)
+
+### 0.20.37
+ - Add s3-proxy to allow accessing s3 presigned url in case the s3 server is not directly accessible. Use `--enable-s3-proxy` to enable the s3 proxy when starting Hypha.
+ - Add `artifact-manager` service to provide comprehensive artifact management, used for creating gallery-like service portal. The artifact manager service is backed by s3 storage and supports presigned url for direct access to the artifacts. This is a replacement of the previous `card` service.
+
 ### 0.20.36
 
  - Upgrade hypha-rpc to support updating reconnection token (otherwise it generate token expired error after some time)

docs/getting-started.md

@@ -220,7 +220,7 @@ svc = await get_remote_service("http://localhost:9527/ws-user-scintillating-lawy
 Include the following script in your HTML file to load the `hypha-rpc` client:
 
 ```html
-<script src="https://cdn.jsdelivr.net/npm/[email protected].37/dist/hypha-rpc-websocket.min.js"></script>
+<script src="https://cdn.jsdelivr.net/npm/[email protected].38/dist/hypha-rpc-websocket.min.js"></script>
 ```
 
 Use the following code in JavaScript to connect to the server and access an existing service:
@@ -250,7 +250,7 @@ By default all the clients connected to Hypha server communicate via the websock
 
 ### User Login and Token-Based Authentication
 
-To access the full features of the Hypha server, users need to log in and obtain a token for authentication. The new `login()` function provides a convenient way to display a login URL, once the user click it and login, it can then return the token for connecting to the server.
+To access the full features of the Hypha server, users need to log in and obtain a token for authentication. The `login()` function provides a convenient way to display a login URL, once the user click it and login, it can then return the token for connecting to the server.
 
 Here is an example of how the login process works using the `login()` function:
 
@@ -294,7 +294,27 @@ The output will provide a URL for the user to open in their browser and
 
  perform the login process. Once the user clicks the link and successfully logs in, the `login()` function will return, providing the token.
 
-The `login()` function also supports additional arguments:
+#### Additional Arguments for Login
+
+You can specify the `workspace` and `expires_in` arguments in the `login()` function so that the token is generated for a specific workspace and expires after a certain period of time.
+
+```python
+token = await login(
+    {
+        "server_url": SERVER_URL,
+        "workspace": "my-workspace",
+        "expires_in": 3600,
+    }
+)
+```
+
+If a token is generated for a specific workspace, when calling `connect_to_server`, you need to specify the same workspace as well:
+
+```python
+server = await connect_to_server({"server_url": "https://ai.imjoy.io", "token": token, "workspace": "my-workspace"})
+```
+
+The `login()` function also supports other additional arguments:
 
 ```python
 token = await login(

docs/migration-guide.md

@@ -15,7 +15,7 @@ To connect to the server, instead of installing the `imjoy-rpc` module, you will
 pip install -U hypha-rpc # new install
 ```
 
-We also changed our versioning strategy, we use the same version number for the server and client, so it's easier to match the client and server versions. For example, `hypha-rpc` version `0.20.37` is compatible with Hypha server version `0.20.37`.
+We also changed our versioning strategy, we use the same version number for the server and client, so it's easier to match the client and server versions. For example, `hypha-rpc` version `0.20.38` is compatible with Hypha server version `0.20.38`.
 
 #### 2. Change the imports to use `hypha-rpc`
 
@@ -128,10 +128,10 @@ loop.run_forever()
 To connect to the server, instead of using the `imjoy-rpc` module, you will need to use the `hypha-rpc` module. The `hypha-rpc` module is a standalone module that provides the RPC connection to the Hypha server. You can include it in your HTML using a script tag:
 
 ```html
-<script src="https://cdn.jsdelivr.net/npm/[email protected].37/dist/hypha-rpc-websocket.min.js"></script>
+<script src="https://cdn.jsdelivr.net/npm/[email protected].38/dist/hypha-rpc-websocket.min.js"></script>
 ```
 
-We also changed our versioning strategy, we use the same version number for the server and client, so it's easier to match the client and server versions. For example, `hypha-rpc` version `0.20.37` is compatible with Hypha server version `0.20.37`.
+We also changed our versioning strategy, we use the same version number for the server and client, so it's easier to match the client and server versions. For example, `hypha-rpc` version `0.20.38` is compatible with Hypha server version `0.20.38`.
 
 #### 2. Change the connection method and use camelCase for service function names
 
@@ -149,7 +149,7 @@ Here is a suggested list of search and replace operations to update your code:
 Here is an example of how the updated code might look:
 
 ```html
-<script src="https://cdn.jsdelivr.net/npm/[email protected].37/dist/hypha-rpc-websocket.min.js"></script>
+<script src="https://cdn.jsdelivr.net/npm/[email protected].38/dist/hypha-rpc-websocket.min.js"></script>
 <script>
 async function main(){
     const server = await hyphaWebsocketClient.connectToServer({"server_url": "https://hypha.amun.ai"});
@@ -197,7 +197,7 @@ We created a tutorial to introduce this new feature: [service type annotation](.
 Here is a quick example in JavaScript:
 
 ```html
-<script src="https://cdn.jsdelivr.net/npm/[email protected].37/dist/hypha-rpc-websocket.min.js"></script>
+<script src="https://cdn.jsdelivr.net/npm/[email protected].38/dist/hypha-rpc-websocket.min.js"></script>
 
 <script>
 async function main(){

docs/service-type-annotation.md

@@ -134,7 +134,7 @@ if __name__ == "__main__":
 **JavaScript Client: Service Usage**
 
 ```html
-<script src="https://cdn.jsdelivr.net/npm/[email protected].37/dist/hypha-rpc-websocket.min.js"></script>
+<script src="https://cdn.jsdelivr.net/npm/[email protected].38/dist/hypha-rpc-websocket.min.js"></script>
 <script>
 async function main() {
     const server = await hyphaWebsocketClient.connectToServer({"server_url": "https://hypha.amun.ai"});

helm-charts/aks-hypha.md

@@ -146,7 +146,7 @@ replicaCount: 1
 image:
   repository: ghcr.io/amun-ai/hypha
   pullPolicy: IfNotPresent
-  tag: "0.20.37"
+  tag: "0.20.38"
 
 serviceAccount:
   create: true

helm-charts/hypha-server/Chart.yaml

@@ -15,7 +15,7 @@ type: application
 # This is the chart version. This version number should be incremented each time you make changes
 # to the chart and its templates, including the app version.
 # Versions are expected to follow Semantic Versioning (https://semver.org/)
-version: 0.20.37
+version: 0.20.38
 
 # This is the version number of the application being deployed. This version number should be
 # incremented each time you make changes to the application. Versions are not expected to

helm-charts/hypha-server/values.yaml

@@ -8,7 +8,7 @@ image:
   repository: ghcr.io/amun-ai/hypha
   pullPolicy: IfNotPresent
   # Overrides the image tag whose default is the chart appVersion.
-  tag: "0.20.37"
+  tag: "0.20.38"
 
 imagePullSecrets: []
 nameOverride: ""

hypha/VERSION

@@ -1,3 +1,3 @@
 {
-    "version": "0.20.37.post3"
+    "version": "0.20.38"
 }

hypha/apps.py

@@ -360,7 +360,7 @@ async def start(
         workspace = context["ws"]
         user_info = UserInfo.model_validate(context["user"])
 
-        async with self.store.get_workspace_interface(workspace, user_info) as ws:
+        async with self.store.get_workspace_interface(user_info, workspace) as ws:
             token = await ws.generate_token()
 
         if not user_info.check_permission(workspace, UserPermission.read):
@@ -520,7 +520,9 @@ async def list_running(self, context: Optional[dict] = None) -> List[str]:
     async def list_apps(self, context: Optional[dict] = None):
         """List applications in the workspace."""
         try:
-            apps = await self.artifact_manager.read(prefix="applications", context=context)
+            apps = await self.artifact_manager.read(
+                prefix="applications", context=context
+            )
             return apps["collection"]
         except KeyError:
             return []

hypha/core/__init__.py

@@ -115,6 +115,8 @@ class UserTokenInfo(BaseModel):
     """Represent user profile."""
 
     token: constr(max_length=1024)  # type: ignore
+    workspace: Optional[str] = None
+    expires_in: Optional[int] = None
     email: Optional[EmailStr] = None
     email_verified: Optional[bool] = None
     name: Optional[constr(max_length=64)] = None  # type: ignore

hypha/core/auth.py

@@ -36,6 +36,7 @@
 AUTH0_NAMESPACE = env.get("AUTH0_NAMESPACE", "https://amun.ai/")
 JWT_SECRET = env.get("JWT_SECRET")
 LOGIN_SERVICE_URL = "/public/services/hypha-login"
+LOGIN_KEY_PREFIX = "login_key:"
 
 if not JWT_SECRET:
     logger.warning(
@@ -67,6 +68,9 @@ def get_user_info(credentials):
         scope=scope,
         expires_at=expires_at,
     )
+    # make sure the user has admin permission to their own workspace
+    user_workspace = info.get_workspace()
+    info.scope.workspaces[user_workspace] = UserPermission.admin
     return info
 
 
@@ -292,7 +296,7 @@ def create_scope(
 
 
 def update_user_scope(
-    user_info: UserInfo, workspace_info: WorkspaceInfo, client_id: str
+    user_info: UserInfo, workspace_info: WorkspaceInfo, client_id: str = None
 ):
     """Update the user scope for a workspace."""
     user_info.scope = user_info.scope or ScopeInfo()
@@ -355,13 +359,20 @@ def create_login_service(store):
         auth0_issuer=AUTH0_ISSUER,
     )
 
-    async def start_login():
+    async def start_login(workspace: str = None, expires_in: int = None):
         """Start the login process."""
-        key = "login_key:" + str(random_id(readable=False))
+        key = str(random_id(readable=False))
         # set the key and with expire time
-        await redis.setex(key, MAXIMUM_LOGIN_TIME, "")
+        await redis.setex(LOGIN_KEY_PREFIX + key, MAXIMUM_LOGIN_TIME, "")
         return {
-            "login_url": f"{login_service_url.replace('/services/', '/apps/')}/?key={key}",
+            "login_url": f"{login_service_url.replace('/services/', '/apps/')}/?key={key}"
+            + (
+                f"&workspace={workspace}"
+                if workspace
+                else "" + f"&expires_in={expires_in}"
+                if expires_in
+                else ""
+            ),
             "key": key,
             "report_url": f"{login_service_url}/report",
             "check_url": f"{login_service_url}/check",
@@ -378,23 +389,25 @@ async def index(event):
 
     async def check_login(key, timeout=MAXIMUM_LOGIN_TIME, profile=False):
         """Check the status of a login session."""
-        assert await redis.exists(key), "Invalid key, key does not exist"
+        assert await redis.exists(
+            LOGIN_KEY_PREFIX + key
+        ), "Invalid key, key does not exist"
         if timeout <= 0:
-            user_info = await redis.get(key)
+            user_info = await redis.get(LOGIN_KEY_PREFIX + key)
             if user_info == b"":
                 return None
             user_info = json.loads(user_info)
             user_info = UserTokenInfo.model_validate(user_info)
             if user_info:
-                await redis.delete(key)
+                await redis.delete(LOGIN_KEY_PREFIX + key)
             return (
                 user_info.model_dump(mode="json")
                 if profile
                 else (user_info and user_info.token)
             )
         count = 0
         while True:
-            user_info = await redis.get(key)
+            user_info = await redis.get(LOGIN_KEY_PREFIX + key)
             if user_info != b"":
                 user_info = json.loads(user_info)
                 user_info = UserTokenInfo.model_validate(user_info)
@@ -403,7 +416,7 @@ async def check_login(key, timeout=MAXIMUM_LOGIN_TIME, profile=False):
                         f"Login session expired, the maximum login time is {MAXIMUM_LOGIN_TIME} seconds"
                     )
                 if user_info:
-                    await redis.delete(key)
+                    await redis.delete(LOGIN_KEY_PREFIX + key)
                     return (
                         user_info.model_dump(mode="json")
                         if profile
@@ -417,6 +430,8 @@ async def check_login(key, timeout=MAXIMUM_LOGIN_TIME, profile=False):
     async def report_login(
         key,
         token,
+        workspace=None,
+        expires_in=None,
         email=None,
         email_verified=None,
         name=None,
@@ -425,19 +440,41 @@ async def report_login(
         picture=None,
     ):
         """Report a token associated with a login session."""
-        assert await redis.exists(key), "Invalid key, key does not exist or expired"
+        assert await redis.exists(
+            LOGIN_KEY_PREFIX + key
+        ), "Invalid key, key does not exist or expired"
+        # workspace = workspace or ("ws-user-" + user_id)
         kwargs = {
             "token": token,
+            "workspace": workspace,
+            "expires_in": expires_in or None,
             "email": email,
             "email_verified": email_verified,
             "name": name,
             "nickname": nickname,
             "user_id": user_id,
             "picture": picture,
         }
-        user_info = UserTokenInfo.model_validate(kwargs)
-        user_info = user_info.model_dump(mode="json")
-        await redis.setex(key, MAXIMUM_LOGIN_TIME, json.dumps(user_info))
+
+        user_token_info = UserTokenInfo.model_validate(kwargs)
+        if workspace:
+            user_info = parse_token(token)
+            # based on the user token, create a scoped token
+            workspace = workspace or user_info.get_workspace()
+            # generate scoped token
+            workspace_info = await store.load_or_create_workspace(user_info, workspace)
+            user_info.scope = update_user_scope(user_info, workspace_info)
+            if not user_info.check_permission(workspace, UserPermission.read):
+                raise Exception(f"Invalid permission for the workspace {workspace}")
+
+            token = generate_presigned_token(user_info, int(expires_in or 3600))
+            # replace the token
+            user_token_info.token = token
+        await redis.setex(
+            LOGIN_KEY_PREFIX + key,
+            MAXIMUM_LOGIN_TIME,
+            user_token_info.model_dump_json(),
+        )
 
     logger.info(
         f"To preview the login page, visit: {login_service_url.replace('/services/', '/apps/')}"