Skip to content

Commit

Permalink
docs: a slightly longer starter example
Browse files Browse the repository at this point in the history
  • Loading branch information
zhudotexe committed Aug 3, 2024
1 parent 6ef41c9 commit 75cafe6
Show file tree
Hide file tree
Showing 2 changed files with 81 additions and 14 deletions.
91 changes: 79 additions & 12 deletions docs/redel.rst
Original file line number Diff line number Diff line change
Expand Up @@ -177,23 +177,28 @@ There are two primary ways to interact with a system now that you've configured
interface, or programmatically. The former is particularly useful to debug your system's behaviour, iterate on prompts,
or otherwise provide an interactive experience. The latter is useful for running experiments and batch queries.

Serving for WebViz
^^^^^^^^^^^^^^^^^^
Serving for Web Interface
^^^^^^^^^^^^^^^^^^^^^^^^^
To serve a ReDel configuration using the included web interface, the library includes the :class:`.VizServer` class.

This class expects the arguments used to configure a ReDel session -- if you've used the ReDel constructor, the easiest
way to provide this is to change ``ReDel`` to ``dict``.
This class takes a :class:`.ReDel` instance and will make copies of it to serve interactive chats.

.. code-block:: python
:caption: An example of serving a ReDel configuration with web browsing over the web interface.
from kani.engines.openai import OpenAIEngine
from redel import Redel
from redel.server import VizServer
from redel.tools.browsing import Browsing
# configure your LLMs - see the Kani engine documentation for more info
root_engine = OpenAIEngine(model="gpt-4o", temperature=0.8, top_p=0.95)
delegate_engine = OpenAIEngine(model="gpt-3.5-turbo", temperature=0.8, top_p=0.95)
# a ReDel session configuration, with ReDel replaced by dict
redel_config = dict(
# a ReDel session configuration, acting as a prototype for interactive sessions
redel_proto = ReDel(
root_engine=root_engine,
delegate_engine=delegate_engine,
title=redel.AUTOGENERATE_TITLE,
tool_configs={
Browsing: {
"always_include": True,
Expand All @@ -202,8 +207,8 @@ way to provide this is to change ``ReDel`` to ``dict``.
},
)
# pass that dict to VizServer
server = VizServer(redel_kwargs=redel_config)
# pass the prototype to VizServer
server = VizServer(redel_proto)
# VizServer.serve() makes the web interface available at 127.0.0.1:8000 by default
server.serve()
Expand All @@ -221,7 +226,7 @@ Otherwise, to use a ReDel configuration in a larger program, you should use :met
to the root node. This method will return an iterator of events that the system emits -- you can use this to listen
for specific events, or filter the stream for :class:`.events.RootMessage` to emulate a chat with a single agent.

A full script might look something like this:
The minimal script for programmatically querying a ReDel system looks like this:

.. code-block:: python
:emphasize-lines: 8
Expand All @@ -230,12 +235,74 @@ A full script might look something like this:
from kani import ChatRole
from redel import ReDel, events
ai = ReDel() # configure your system here, or leave blank for a default system
async def main():
ai = ReDel() # configure your system here, or leave blank for a default system
async for event in ai.query("What is the airspeed velocity of an unladen swallow?"):
# your event logic here - this example prints all the messages from the root
if isinstance(event, events.RootMessage) and event.msg.role == ChatRole.ASSISTANT:
if event.msg.text:
print(event.msg.text)
asyncio.run(main())
In most cases, however, your programmatic entrypoint will be significantly longer, involving any data loading and
pre-/post-processing steps you need for your own application.

For example, if you're an academic wanting to run the ReDel system over each task in a dataset, you might have a
script that looks like this:

.. code-block:: python
:emphasize-lines: 32
import asyncio
from kani import ChatRole
from kani.engines.openai import OpenAIEngine
from redel import ReDel, events
from redel.delegation import DelegateOne
from redel.tools.browsing import Browsing
# configure your LLMs - see the Kani engine documentation for more info
root_engine = OpenAIEngine(model="gpt-4o", temperature=0.8, top_p=0.95)
delegate_engine = OpenAIEngine(model="gpt-3.5-turbo", temperature=0.8, top_p=0.95)
# helper: run one task through the ReDel system
async def query_redel(query):
# create a new instance for each query
# and configure your system here
ai = ReDel(
# LLMs
root_engine=root_engine,
delegate_engine=delegate_engine,
# delegation
delegation_scheme=DelegateOne,
# tools
tool_configs={
Browsing: {
"always_include": True,
"kwargs": {"long_engine": long_engine},
},
},
)
# send a message to the root node and listen for events from the system
async for event in ai.query(query):
# your event logic here - this example prints all the messages from the root
if isinstance(event, events.RootMessage) and event.msg.role == ChatRole.ASSISTANT:
if event.msg.text:
print(event.msg.text)
# any additional logic (e.g. evaluation, saving
# main entrypoint: load in your dataset and iterate over it
async def main():
# your logic to load the dataset
dataset = load_dataset(...)
for task in dataset:
await query_redel(task)
# run the script when invoked from the command line
if __name__ == "__main__":
asyncio.run(main())
4 changes: 2 additions & 2 deletions redel/server/server.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,10 +41,10 @@ def __init__(
:param save_dirs: A list of paths to scan for ReDel saves to make available to load. Defaults to
``~/.redel/instances/``.
:param redel_factory: An asynchronous function that creates a new :class:`.ReDel` instance when called.
If this is set, ``redel_kwargs`` and ``redel_proto`` must not be set.
If this is set, ``redel_proto`` must not be set.
"""
if redel_proto and redel_factory:
raise ValueError("At most one of ('redel_kwargs', 'redel_factory') may be supplied.")
raise ValueError("At most one of ('redel_proto', 'redel_factory') may be supplied.")
elif not (redel_proto or redel_factory):
redel_proto = ReDel()
self.redel_proto = redel_proto
Expand Down

0 comments on commit 75cafe6

Please sign in to comment.