Setup everything for sotopia in one script (#122)

* git ignore * add nextra site * change path to docs * Add installation instruction (#76) * fix an accordion bug * Add tabs for choosing platform in Redis Instructions * add platform tab * updated open in colab instruction * add agents.md/environments.md rename all_the_issues to troubleshooting * add auto set script * add more dataset choices * adding download data menu and protection when the path exists * Unify benchmark and install into `sotopia` cli command * move install and benchmark in to folders * make flow more rigorious and add tests * update linux direction * add restart directions * remove _renderer * add use docker test * call -> run * add shell=True * mkdir when the dirctory doesn't exist * remove docker test due to memory limit * add prompt mocking * add install cli * add a sleep to make sure that the redis is launched before stopping * add matrix os in pytest * add poetry to path on macos * Use install poetry action * make redis directory on mac * avoid permission issue on mac * update macos test * test docker * install docker with brew * set up docker * give up testing docker on mac * check if docker daemon is up and brew update * improve prompts * update prompt tests --------- Co-authored-by: XuhuiZhou <[email protected]> Co-authored-by: Ruiyi Wang <[email protected]>
sotopia-lab · Jun 26, 2024 · 701f2a8 · 701f2a8
1 parent 5818a89
commit 701f2a8
Show file tree

Hide file tree

Showing 22 changed files with 1,017 additions and 286 deletions.
diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml
@@ -17,19 +17,27 @@ on:
 
 jobs:
   Pytest:
-    runs-on: ubuntu-latest
     strategy:
       max-parallel: 5
+      matrix:
+        os: [ubuntu-latest, macos-13]
+
+    runs-on: ${{ matrix.os }}
 
     steps:
-    - uses: actions/checkout@v3
+    - uses: actions/checkout@v4
     - name: Set up Python 3.11
       uses: actions/setup-python@v4
       with:
         python-version: 3.11.2
+    - name: Set up Docker
+      if: runner.os == 'ubuntu-latest'
+      uses: docker-practice/actions-setup-docker@master
+      timeout-minutes: 12
+    - name: Install Poetry
+      uses: abatilo/actions-poetry@v2
     - name: Install dependencies
       run: |
-        curl -sSL https://install.python-poetry.org | python3
         poetry lock
         poetry install --with test -E chat
     - name: Test with pytest

diff --git a/.gitignore b/.gitignore
@@ -161,3 +161,5 @@ backup/*
 node_modules/*
 docs/.next/*
 docs/node_modules/*
+
+redis-data/*
diff --git a/README.md b/README.md
@@ -82,7 +82,7 @@ For some experiments, TogetherAI key is required to run the code. Please set the
 conda env config vars set TOGETHER_API_KEY=your_key
 ```
 
-A redis-stack server is required to run the code. Please follow the [instruction](https://redis.io/docs/stack/get-started/install/docker/) to start a redis-stack server or use an existing server. You can also check [Q&A](/docs/all_the_issues.md) to initiate the redis server with the Sotopia data.
+A redis-stack server is required to run the code. Please follow the [instruction](https://redis.io/docs/stack/get-started/install/docker/) to start a redis-stack server or use an existing server. You can also check [Q&A](/docs/troubleshooting.md) to initiate the redis server with the Sotopia data.
 
 The `REDIS_OM_URL` need to be set before loading and saving agents:
 ```bash
@@ -156,7 +156,7 @@ To run a large batch of environments, you can change the `ENV_IDS` parameter in
 ## Getting access to your simulation
 After running experiments, you can go to the `examples/redis_stats.ipynb` notebook to check the existing episodes (Episode Log section), as well as calculate the performance.
 
-For the original Sotopia simulation in our paper's experiments, you can find how to get them in the [Q&A](/docs/all_the_issues.md) section in the `./docs` folder.
+For the original Sotopia simulation in our paper's experiments, you can find how to get them in the [Q&A](/docs/troubleshooting.md) section in the `./docs` folder.
 
 ## Adding new characters and environments
 You can use the following function with the `**kwargs` being the properties of the `AgentProfile` class. This is the same for the scenarios/environments.

diff --git a/docs/pages/_meta.json b/docs/pages/_meta.json
@@ -7,6 +7,14 @@
         "title": "Documentation",
         "type": "menu",
         "items": {
+            "agents": {
+                "title": "Agents",
+                "href": "/agents"
+            },
+            "environments": {
+                "title": "Environments",
+                "href": "/environments"
+            },
             "examples": {
                 "title": "Examples",
                 "href": "/examples"

diff --git a/docs/pages/agents.md b/docs/pages/agents.md
diff --git a/docs/pages/environments.md b/docs/pages/environments.md
diff --git a/docs/pages/index.mdx b/docs/pages/index.mdx
@@ -107,7 +107,7 @@ Redis stack is a required dependency for using Sotopia. There are two ways to se
       <AccordionItem value="item-1">
         <AccordionTrigger>Docker is my thing.</AccordionTrigger>
         <AccordionContent>
-        Please follow the [instruction](https://redis.io/docs/stack/get-started/install/docker/) to start a redis-stack server or use an existing server. You can also check [Q&A](/docs/all_the_issues.md) to initiate the redis server with the Sotopia data.
+        Please follow the [instruction](https://redis.io/docs/stack/get-started/install/docker/) to start a redis-stack server or use an existing server. You can also check [Q&A](/docs/troubleshooting.md) to initiate the redis server with the Sotopia data.
 
         The `REDIS_OM_URL` need to be set before loading and saving agents:
         ```bash

diff --git a/docs/pages/all_the_issues.md → docs/pages/troubleshooting.md b/docs/pages/all_the_issues.md → docs/pages/troubleshooting.md
@@ -1,4 +1,4 @@
-# Q&A
+# Troubleshooting
 ## Missing episodes
 
 Large batch size may cause some episodes to be skipped. This is due to the fact that the server may not be able to handle the load. Try reducing the batch size. But you can also use the script in `examples/fix_missing_episodes.py` to fix the missing episodes.

diff --git a/poetry.lock b/poetry.lock
diff --git a/pyproject.toml b/pyproject.toml
@@ -15,7 +15,6 @@ langchain = "~0.2.5"
 rich = "^13.6.0"
 PettingZoo = "1.24.3"
 redis-om = "^0.2.1"
-types-tqdm = "*"
 gin-config = "^0.5.0"
 absl-py = "^2.0.0"
 together = "^0.2.4"
@@ -42,6 +41,8 @@ pandas = { version = "^2.1.1", optional = true }
 pre-commit = "*"
 nbmake = "*"
 types-setuptools = "*"
+types-requests = "*"
+types-tqdm = "*"
 ruff = "*"
 
 [tool.poetry.group.test.dependencies]
@@ -79,4 +80,4 @@ testpaths = ["tests"]
 python_files = "test_*.py"
 
 [tool.poetry.scripts]
-sotopia_benchmark = "sotopia.benchmark:app"
+sotopia = "sotopia.cli:app"
diff --git a/sotopia/benchmark/__init__.py b/sotopia/benchmark/__init__.py
diff --git a/sotopia/cli/__init__.py b/sotopia/cli/__init__.py
@@ -0,0 +1,5 @@
+from .app import app
+from .install import install
+from .benchmark import benchmark
+
+__all__ = ["app", "install", "benchmark"]
diff --git a/sotopia/cli/app.py b/sotopia/cli/app.py
@@ -0,0 +1,3 @@
+from typer import Typer
+
+app = Typer()
diff --git a/sotopia/cli/benchmark/__init__.py b/sotopia/cli/benchmark/__init__.py
@@ -0,0 +1,3 @@
+from .benchmark import benchmark
+
+__all__ = ["benchmark"]
diff --git a/sotopia/benchmark/cli.py → sotopia/cli/benchmark/benchmark.py b/sotopia/benchmark/cli.py → sotopia/cli/benchmark/benchmark.py
@@ -34,8 +34,7 @@
 
 import typer
 from pathlib import Path
-
-app = typer.Typer()
+from ..app import app
 
 
 def check_existing_episodes(
@@ -264,7 +263,7 @@ def _set_up_logs(
 
 
 @app.command()
-def cli(
+def benchmark(
     model: str = typer.Option(..., help="The language model you want to benchmark."),
     partner_model: str = typer.Option(
         "together_ai/meta-llama/Llama-3-70b-chat-hf",
@@ -297,7 +296,3 @@ def cli(
         verbose=False,
         push_to_db=True,
     )
-
-
-if __name__ == "__main__":
-    app()
diff --git a/sotopia/cli/install/__init__.py b/sotopia/cli/install/__init__.py
@@ -0,0 +1,3 @@
+from .install import install
+
+__all__ = ["install"]