diff --git a/docs/_sidebar.md b/docs/_sidebar.md index 9b4b3729..4e03f455 100644 --- a/docs/_sidebar.md +++ b/docs/_sidebar.md @@ -4,4 +4,5 @@ * [Service Type Annotation](/service-type-annotation) * [Serverless Functions](/serverless-functions) * [Serve ASGI Web Apps](/asgi-apps) + * [Launch Service](/launch-service) * [Development](/development) diff --git a/docs/asgi-apps.md b/docs/asgi-apps.md index 7f15cc91..da728ed7 100644 --- a/docs/asgi-apps.md +++ b/docs/asgi-apps.md @@ -1,83 +1,275 @@ -# Serve ASGI Web Applications +# Serving ASGI Applications with Hypha -ASGI is a standard to create async web applications. With hypha, you can register a service which serves an ASGI application, e.g. a FastAPI app which serve a web page, or a web API. +## Introduction -The following example shows how to use FastAPI to create a web application in the browser, e.g. https://imjoy-notebook.netlify.app/, and serve it through Hypha. +When deploying ASGI web applications like those built with FastAPI, developers often rely on traditional servers and tools like **Uvicorn**. Uvicorn is a popular choice for serving ASGI applications due to its simplicity and ease of use. Typically, you can start a server with the following command: -You need to first install fastapi: +```bash +uvicorn myapp:app --host 0.0.0.0 --port 8000 +``` + +However, this approach has limitations, particularly when you need to expose your application to the public. Hosting a server with Uvicorn typically requires: +- A public IP address. +- DNS management. +- SSL certificates for secure connections. +- Firewall configurations to ensure access while maintaining security. + +For many users, especially those working in environments with restrictive networks or behind firewalls, exposing a server publicly can be cumbersome and insecure. Moreover, maintaining server infrastructure and ensuring security compliance adds significant overhead. + +**Hypha** provides a solution to these challenges by allowing you to serve ASGI applications without the need for traditional server setups. With Hypha, you can instantly deploy and make your web application publicly accessible, bypassing the need for public IP addresses, DNS, or server management. This is especially beneficial in scenarios where you want to quickly share your application without worrying about the complexities of infrastructure. + +## The Hypha Serve Utility + +To streamline the process of deploying ASGI applications, Hypha offers a utility similar to Uvicorn, but with added benefits. The `hypha_rpc.utils.serve` utility allows you to serve your FastAPI (or any ASGI-compatible) application directly through the Hypha platform. This tool provides a simple command-line interface that makes deployment quick and easy. + +### Why Use Hypha Over Uvicorn? + +While Uvicorn is a powerful tool for local development and serving applications within controlled environments, Hypha's serve utility offers: +- **Ease of Access**: No need for public IP, DNS management, or SSL certificates. +- **Security**: Hypha handles secure connections and access control. +- **Simplicity**: Deploy your application with a single command, without worrying about infrastructure management. +- **Flexibility**: For advanced use cases, Hypha allows direct registration of ASGI services, giving you fine-grained control. + +### Serving Your ASGI App with Hypha + +Let's explore how to use the `hypha_rpc.utils.serve` utility to deploy your FastAPI application. + +## Prerequisites + +- **Hypha**: Ensure you have Hypha installed and configured. +- **FastAPI**: For this example, we'll use FastAPI, but any ASGI-compatible framework will work. +- **Python 3.9+**: The utility requires Python 3.9 or higher. + +## Installation -In Pyodide, you can install it using micropip: +If you haven't installed FastAPI yet, you can do so using pip: + +```bash +pip install fastapi +``` + +Ensure that `hypha_rpc` is installed and available in your environment. + +## Usage + +### Basic Example + +Here’s a simple example of how to serve a FastAPI application using the `hypha_rpc.utils.serve` utility. + +### Step 1: Create a FastAPI App + +Create a simple FastAPI app (e.g., `myapp.py`): ```python -import micropip -await micropip.install(["fastapi==0.70.0"]) +from fastapi import FastAPI +from fastapi.responses import HTMLResponse + +app = FastAPI() + +@app.get("/", response_class=HTMLResponse) +async def root(): + return """ + + Cat + cat + + """ + +@app.get("/api/v1/test") +async def test(): + return {"message": "Hello, it works!"} +``` + +### Step 2: Serve the App Using the Hypha Serve Utility + +You can serve your ASGI app with Hypha using the following command: + +```bash +python -m hypha_rpc.utils.serve myapp:app --id=cat --name=Cat --server-url=https://hypha.aicell.io --workspace=my-workspace --token=your_token_here +``` + +### Parameters + +- `myapp:app`: The `module:app` format where `myapp` is your Python file, and `app` is the FastAPI instance. +- `--id`: A unique identifier for your service. +- `--name`: A friendly name for your service. +- `--server-url`: The URL of the Hypha server. +- `--workspace`: The workspace you want to connect to within the Hypha server. +- `--token`: The authentication token for accessing the Hypha server. + +### Using the `--login` Option + +If you don’t have a token or prefer to log in interactively, you can use the `--login` option: + +```bash +python -m hypha_rpc.utils.serve myapp:app --id=cat --name=Cat --server-url=https://hypha.aicell.io --workspace=my-workspace --login ``` -Or in normal Python, you can install it using pip: +This will prompt you to log in, and it will automatically retrieve and use the token. + +### Disabling SSL Verification + +If you need to disable SSL verification (e.g., for testing purposes), you can use the `--disable-ssl` option: ```bash -pip install -U fastapi +python -m hypha_rpc.utils.serve myapp:app --id=cat --name=Cat --server-url=https://hypha.aicell.io --workspace=my-workspace --login --disable-ssl ``` -Then you can create a FastAPI app and serve it through Hypha: +### Accessing the Application + +Once the app is running, you will see a URL printed in the console, which you can use to access your app. The URL will look something like this: + +``` +https://hypha.aicell.io/{workspace}/apps/{service_id} +``` + +Replace `{workspace}` and `{service_id}` with the actual workspace name and service ID you provided. + +### Example Commands + +**Serve with Token:** + +```bash +python -m hypha_rpc.utils.serve myapp:app --id=cat --name=Cat --server-url=https://hypha.aicell.io --workspace=my-workspace --token=sflsflsdlfslfwei32r90jw +``` + +**Serve with Login and Disable SSL:** + +```bash +python -m hypha_rpc.utils.serve myapp:app --id=cat --name=Cat --server-url=https://hypha.aicell.io --workspace=my-workspace --login --disable-ssl +``` + +## Advanced: Registering the ASGI Service Directly + +For users who prefer more flexibility and control, Hypha also allows you to register the ASGI service directly through the platform, bypassing the utility function. This approach is useful if you need custom behavior or want to integrate additional logic into your application. + +### Example of Direct Registration + +Here’s an example of how you can directly register a FastAPI service with Hypha: ```python import asyncio from hypha_rpc import connect_to_server +from fastapi import FastAPI, HTMLResponse -from fastapi import FastAPI -from fastapi.responses import HTMLResponse +app = FastAPI() + +@app.get("/", response_class=HTMLResponse) +async def root(): + return """ + + Cat + cat + + """ + +@app.get("/api/v1/test") +async def test(): + return {"message": "Hello, it works!"} -# Connect to the Hypha server -server_url = "https://hypha.aicell.io" +async def serve_fastapi(args, context=None): + # context can be used for authorization, e.g., checking the user's permission + # e.g., check user id against a list of allowed users + scope = args["scope"] + print(f'{context["user"]["id"]} - {scope["client"]} - {scope["method"]} - {scope["path"]}') + await app(args["scope"], args["receive"], args["send"]) async def main(): - server = await connect_to_server({"server_url": server_url}) - - app = FastAPI() - - @app.get("/", response_class=HTMLResponse) - async def root(): - html_content = """ - - - Cat - - - cat - - - """ - return HTMLResponse(content=html_content, status_code=200) - - @app.get("/api/v1/test") - async def test(): - return {"message": "Hello, it works!", "server_url": server_url} - - async def serve_fastapi(args, context=None): - # context can be used for do authorization - # e.g. check user id against a list of allowed users - scope = args["scope"] - print(f'{context["user"]["id"]} - {scope["client"]} - {scope["method"]} - {scope["path"]}') - await app(args["scope"], args["receive"], args["send"]) + # Connect to Hypha server + server = await connect_to_server({"server_url": "https://hypha.aicell.io"}) + + svc_info = await server.register_service({ + "id": "cat", + "name": "cat", + "type": "ASGI", + "serve": serve_fastapi, + "config": {"visibility": "public"} + }) + + print(f"Access your app at: {server.config.workspace}/apps/{svc_info['id'].split(':')[1]}") + await server.serve() + +asyncio.run(main()) +``` + +## Running FastAPI in the Browser with Pyodide + +Hypha also supports running FastAPI applications directly in the browser using Pyodide. This feature is particularly useful when you want to create a lightweight, client-side application that can be served without any server infrastructure. + +### Step 1: Install FastAPI in Pyodide +To install FastAPI in a Pyodide environment, use `micropip`: + +```python +import micropip +await micropip.install(["fastapi==0.70.0"]) +``` + +### Step 2: Modify the FastAPI App for Pyodide + +The FastAPI app remains largely the same, but you should be aware of a few key differences when running in Pyodide: + +- Pyodide runs in a browser, so any blocking or long-running tasks should be handled with care. +- The installation of packages is done using `micropip` instead of `pip`. + +### Example Code for Pyodide + +Here’s an example of how to create and serve a FastAPI app in the browser using Pyodide: + +```python +import asyncio +import micropip +await micropip.install(["fast + +api==0.70.0"]) + +from hypha_rpc import connect_to_server +from fastapi import FastAPI, HTMLResponse + +app = FastAPI() + +@app.get("/", response_class=HTMLResponse) +async def root(): + return """ + + Cat + cat + + """ + +@app.get("/api/v1/test") +async def test(): + return {"message": "Hello, it works!"} + +async def serve_fastapi(args, context=None): + # context can be used for authorization, e.g., checking the user's permission + # e.g., check user id against a list of allowed users + scope = args["scope"] + print(f'{context["user"]["id"]} - {scope["client"]} - {scope["method"]} - {scope["path"]}') + await app(args["scope"], args["receive"], args["send"]) + +async def main(): + # Connect to Hypha server + server = await connect_to_server({"server_url": "https://hypha.aicell.io"}) + svc_info = await server.register_service({ "id": "cat", "name": "cat", "type": "ASGI", "serve": serve_fastapi, - "config":{ - "visibility": "public", - "require_context": True - } + "config": {"visibility": "public"} }) - print(f"Test it with the HTTP proxy: {server_url}/{server.config.workspace}/apps/{svc_info['id'].split(':')[1]}") + print(f"Access your app at: {server.config.workspace}/apps/{svc_info['id'].split(':')[1]}") await server.serve() -# Assuming if you are running in a Jupyter notebook -# which support top-level await, if not, you can use asyncio.run(main()) await main() ``` -This will create a web page which you can view in the browser, and also an API endpoint at `/api/v1/test`. +### Accessing the Application + +Just like in the standard Python environment, the application will be accessible through the URL provided by the Hypha server. + +## Conclusion + +Hypha offers a versatile and powerful way to serve ASGI applications, whether through the convenient `hypha_rpc.utils.serve` utility, by directly registering your ASGI service, or even by running FastAPI directly in the browser with Pyodide. By removing the need for traditional server setups, Hypha enables you to focus on your application without the overhead of managing infrastructure. Whether you’re deploying in a local environment or directly in a browser, Hypha simplifies the process, making it easier than ever to share your ASGI apps with the world. The added flexibility of direct service registration and browser-based FastAPI apps expands the possibilities for deploying and sharing your web applications. diff --git a/docs/getting-started.md b/docs/getting-started.md index 5664b9a3..810b9539 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -220,7 +220,7 @@ svc = await get_remote_service("http://localhost:9527/ws-user-scintillating-lawy Include the following script in your HTML file to load the `hypha-rpc` client: ```html - + ``` Use the following code in JavaScript to connect to the server and access an existing service: @@ -516,26 +516,3 @@ Multiple startup functions can be specified by providing additional `--startup-f python -m hypha.server --host=0.0.0.0 --port=9527 --startup-functions=./example-startup-function.py:hypha_startup ./example-startup-function2.py:hypha_startup ``` -#### Launching External Services Using Commands - -If you need to start services written in languages other than Python or requiring a different Python environment than your Hypha server, you can use the `launch_external_services` utility function available in the `hypha.utils` module. - -Here's an example of using `launch_external_services` within a startup function initiated with the `--startup-functions` option: - -```python -from hypha.utils import launch_external_services - -async def hypha_startup(server): - # ... - - await launch_external_services( - server, - "python ./tests/example_service_script.py --server-url={server_url} --service-id=external-test-service --workspace={workspace} --token={token}", - name="example_service_script", - check_services=["external-test-service"], - ) -``` - -In this example, `launch_external_services` starts an external service defined in `./tests/example_service_script.py`. The command string uses placeholders like `{server_url}`, `{workspace}`, and `{token}`, which are automatically replaced with their actual values during execution. - -By using `launch_external_services`, you can seamlessly integrate external services into your Hypha server, regardless of the programming language or Python environment they utilize. diff --git a/docs/launch-service.md b/docs/launch-service.md new file mode 100644 index 00000000..23a3f2bc --- /dev/null +++ b/docs/launch-service.md @@ -0,0 +1,71 @@ +# Launch Services from the Command Line + +## Introduction + +In certain scenarios, you may need to launch external services written in other programming languages or requiring a different Python environment than your Hypha server. For example, you might want to run a Python script that creates a Hypha service or start a non-Python-based service. + +Hypha offers a utility function, `launch_external_services`, that allows you to run external commands or scripts while keeping the associated Hypha service alive. This utility makes it easy to integrate and manage external services alongside your Hypha server. + +## The `launch_external_services` Utility + +The `launch_external_services` utility is available in the `hypha_rpc.utils.launch` module. It allows you to start external services (e.g., Python scripts or other command-line tools) and ensure that they stay running as long as the Hypha server is active. + +### Use Cases + +- **Launching services written in different languages**: You can use this utility to start services written in JavaScript, Go, Rust, etc. +- **Managing services in separate Python environments**: If you need a different environment than the one used by your Hypha server (e.g., a different Python version or environment), you can use this utility to launch the service in the appropriate environment. +- **Seamless integration**: External services are managed and monitored like regular Hypha services. + +## Example Usage + +Below is an example of how to use the `launch_external_services` utility in a startup function. This example demonstrates how to run a Python script that creates a Hypha service: + +```python +from hypha_rpc.utils.launch import launch_external_services + +async def hypha_startup(server): + # Example of launching an external service + await launch_external_services( + server, + "python ./tests/example_service_script.py --server-url={server_url} --service-id=external-test-service --workspace={workspace} --token={token}", + name="example_service_script", + check_services=["external-test-service"], + ) +``` + +### Explanation + +- **`server`**: The Hypha server instance. +- **Command string**: In this example, a Python script (`example_service_script.py`) is executed, which registers an external service in Hypha. The command uses placeholders like `{server_url}`, `{workspace}`, and `{token}` that are dynamically replaced with their actual values. +- **`name`**: This is the friendly name given to the external service (`example_service_script` in this case). +- **`check_services`**: This parameter specifies which services to check and keep alive (e.g., `external-test-service`). + +### Command Placeholders + +The command string supports several placeholders that are automatically replaced with the appropriate values during execution: + +- **`{server_url}`**: The URL of the Hypha server. +- **`{workspace}`**: The workspace name associated with the Hypha server. +- **`{token}`**: The authentication token for accessing the Hypha server. + +These placeholders allow you to dynamically insert values, making it easier to write reusable commands. + +## Keeping Services Alive + +One of the key benefits of `launch_external_services` is that it ensures the external service remains alive as long as the Hypha server is running. If the service stops unexpectedly, Hypha will attempt to restart it to maintain availability. + +## Example Command with Placeholders + +Here's an example of a typical command using placeholders: + +```bash +python ./my_script.py --server-url={server_url} --service-id=my-service --workspace={workspace} --token={token} +``` + +This command launches a Python script (`my_script.py`) that registers a service with the specified `server_url`, `workspace`, and `token`. The placeholders are automatically substituted with actual values when the command is executed. + +## Conclusion + +The `launch_external_services` utility provides a powerful way to manage external services alongside Hypha, whether those services are written in a different programming language or require a different Python environment. This utility simplifies the process of launching and maintaining external services, ensuring they stay alive and functional throughout the lifetime of your Hypha server. + +By integrating external services seamlessly into Hypha, you can extend the functionality of your system and manage diverse services with minimal overhead. Whether you’re dealing with Python scripts or other command-line tools, `launch_external_services` makes it easy to keep everything running smoothly. diff --git a/docs/migration-guide.md b/docs/migration-guide.md index eeca5c82..c78b4b15 100644 --- a/docs/migration-guide.md +++ b/docs/migration-guide.md @@ -15,7 +15,7 @@ To connect to the server, instead of installing the `imjoy-rpc` module, you will pip install -U hypha-rpc # new install ``` -We also changed our versioning strategy, we use the same version number for the server and client, so it's easier to match the client and server versions. For example, `hypha-rpc` version `0.20.32` is compatible with Hypha server version `0.20.32`. +We also changed our versioning strategy, we use the same version number for the server and client, so it's easier to match the client and server versions. For example, `hypha-rpc` version `0.20.33` is compatible with Hypha server version `0.20.33`. #### 2. Change the imports to use `hypha-rpc` @@ -128,10 +128,10 @@ loop.run_forever() To connect to the server, instead of using the `imjoy-rpc` module, you will need to use the `hypha-rpc` module. The `hypha-rpc` module is a standalone module that provides the RPC connection to the Hypha server. You can include it in your HTML using a script tag: ```html - + ``` -We also changed our versioning strategy, we use the same version number for the server and client, so it's easier to match the client and server versions. For example, `hypha-rpc` version `0.20.32` is compatible with Hypha server version `0.20.32`. +We also changed our versioning strategy, we use the same version number for the server and client, so it's easier to match the client and server versions. For example, `hypha-rpc` version `0.20.33` is compatible with Hypha server version `0.20.33`. #### 2. Change the connection method and use camelCase for service function names @@ -149,7 +149,7 @@ Here is a suggested list of search and replace operations to update your code: Here is an example of how the updated code might look: ```html - + + + - + @@ -71,7 +71,7 @@
- - - - + + + - + @@ -64,7 +64,6 @@ } .main-content { - margin-left: 250px; padding: 20px; transition: all 0.3s; } @@ -138,7 +137,7 @@
- diff --git a/hypha/utils/__init__.py b/hypha/utils/__init__.py index 247a92ef..899eff05 100644 --- a/hypha/utils/__init__.py +++ b/hypha/utils/__init__.py @@ -473,109 +473,6 @@ async def send(self, message, send, request_headers) -> None: await send(message) -async def _http_ready_func(url, p, logger=None): - """Check if the http service is ready.""" - async with httpx.AsyncClient(timeout=20.0) as client: - try: - resp = await client.get(url) - # We only care if we get back *any* response, not just 200 - # If there's an error response, that can be shown directly to the user - if logger: - logger.debug(f"Got code {resp.status} back from {url}") - return True - except httpx.RequestError as exc: - if logger: - logger.debug(f"Connection to {url} failed: {exc}") - return False - - -async def launch_external_services( - server: any, - command: str, - name=None, - env=None, - check_services=None, - check_url=None, - timeout=5, - logger=None, -): - """ - Launch external services asynchronously and monitors their status (requires simpervisor). - - Args: - server: The server instance for which the services are to be launched. - command (str): The command to be executed to start the service. Any placeholders such as {server_url}, - {workspace}, and {token} in the command will be replaced with actual values. - name (str, optional): The name of the service. If not provided, the first argument of the command is used as the name. - env (dict, optional): A dictionary of environment variables to be set for the service. - check_services (list, optional): A list of service IDs to be checked for readiness. The service is considered ready - if all services in the list are available. - check_url (str, optional): A URL to be checked for readiness. The service is considered ready if the URL is accessible. - timeout (int, optional): The maximum number of seconds to wait for the service to be ready. Defaults to 5. - logger (logging.Logger, optional): A logger instance to be used for logging messages. If not provided, no messages - are logged. - - Raises: - Exception: If the service fails to start or does not become ready within the specified timeout. - - Returns: - proc (SupervisedProcess): The process object for the service. You can call proc.kill() to kill the process. - """ - from simpervisor import SupervisedProcess - - token = await server.generate_token() - # format the cmd so we fill in the {server_url} placeholder - command = command.format( - server_url=server.config.local_base_url, - workspace=server.config["workspace"], - token=token, - ) - command = [c.strip() for c in command.split() if c.strip()] - assert len(command) > 0, f"Invalid command: {command}" - name = name or command[0] - # split command into list, strip off spaces and filter out empty strings - server_env = os.environ.copy() - server_env.update(env or {}) - - async def ready_function(p): - if check_services: - for service_id in check_services: - try: - await server.get_service( - server.config["workspace"] + "/" + service_id - ) - except Exception: - return False - return True - if check_url: - return await _http_ready_func(check_url, p, logger=logger) - return True - - proc = SupervisedProcess( - name, - *command, - env=server_env, - always_restart=False, - ready_func=ready_function, - ready_timeout=timeout, - log=logger, - ) - - try: - await proc.start() - - is_ready = await proc.ready() - - if not is_ready: - await proc.kill() - raise Exception(f"External services ({name}) failed to start") - except: - if logger: - logger.exception(f"External services ({name}) failed to start") - raise - return proc - - async def _example_hypha_startup(server): """An example hypha startup module.""" assert server.register_codec diff --git a/requirements.txt b/requirements.txt index dfb94f6a..1382855f 100644 --- a/requirements.txt +++ b/requirements.txt @@ -2,7 +2,7 @@ aioboto3==13.1.1 aiofiles==23.2.1 base58==2.1.1 fastapi==0.106.0 -hypha-rpc==0.20.32 +hypha-rpc==0.20.33 jinja2==3.1.4 lxml==4.9.3 msgpack==1.0.8 diff --git a/setup.py b/setup.py index 1691f241..d823b1be 100644 --- a/setup.py +++ b/setup.py @@ -12,7 +12,7 @@ REQUIREMENTS = [ "aiofiles", "fastapi>=0.70.0,<=0.106.0", - "hypha-rpc>=0.20.32", + "hypha-rpc>=0.20.33", "msgpack>=1.0.2", "numpy", "pydantic[email]>=2.6.1", diff --git a/tests/test_startup_function.py b/tests/test_startup_function.py index 2ce275c8..ad6512e8 100644 --- a/tests/test_startup_function.py +++ b/tests/test_startup_function.py @@ -11,7 +11,6 @@ from requests import RequestException from . import SIO_PORT, WS_SERVER_URL -from hypha.utils import launch_external_services @pytest.mark.asyncio @@ -59,37 +58,3 @@ def test_failed_startup_function(): proc.kill() proc.terminate() assert timeout < 0 # The server should fail - - -@pytest.mark.asyncio -async def test_launch_external_services(fastapi_server): - """Test the launch command utility fuction.""" - server = await connect_to_server( - { - "name": "my third app", - "server_url": WS_SERVER_URL, - } - ) - proc = await launch_external_services( - server, - "python ./tests/example_service_script.py --server-url={server_url} --service-id=external-test-service --workspace={workspace} --token={token}", - name="example_service_script", - check_services=["external-test-service"], - ) - external_service = await server.get_service("external-test-service") - assert external_service.id.endswith(":external-test-service") - assert await external_service.test(1) == 100 - await proc.kill() - await asyncio.sleep(0.1) - try: - await server.get_service("external-test-service") - except Exception as e: - assert "not found" in str(e) - proc = await launch_external_services( - server, - "python -m http.server 9391", - name="example_service_script", - check_url="http://127.0.0.1:9391", - ) - assert await proc.ready() - await proc.kill()