Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add Parallel Tool mode for Vertex AI #1217

Merged
merged 8 commits into from
Dec 18, 2024

Conversation

devjn
Copy link
Contributor

@devjn devjn commented Nov 24, 2024

This PR intends to add parallel tool mode for Vertex AI.
That is done by adding new mode VERTEXAI_PARALLEL_TOOLS.


Important

Adds VERTEXAI_PARALLEL_TOOLS mode for parallel tool processing in Vertex AI, with updates to model handling and response processing.

  • Behavior:
    • Adds VERTEXAI_PARALLEL_TOOLS mode in mode.py.
    • Updates from_vertexai() in client_vertexai.py to support the new mode.
    • Implements handle_vertexai_parallel_tools() in process_response.py to process responses in parallel.
  • Models:
    • Introduces VertexAIParallelBase in parallel.py for handling parallel responses.
    • Adds VertexAIParallelModel() in parallel.py to create instances of VertexAIParallelBase.
  • Functions:
    • Modifies _create_vertexai_tool() in client_vertexai.py to handle multiple models.
    • Updates vertexai_process_response() in client_vertexai.py to support parallel tools.
  • Misc:
    • Adds type checks in _create_gemini_json_schema() in client_vertexai.py.

This description was created by Ellipsis for c582618. It will automatically update as commits are pushed.

Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me! Reviewed everything up to c582618 in 1 minute and 12 seconds

More details
  • Looked at 233 lines of code in 4 files
  • Skipped 0 files when reviewing.
  • Skipped posting 8 drafted comments based on config settings.
1. instructor/client_vertexai.py:39
  • Draft comment:
    Remove the debug print statement or replace it with proper logging.
# print(f"Debug - Model list: {[model.__name__ for model in model_list]}")
  • Reason this comment was not posted:
    Confidence changes required: 80%
    The debug print statement is not suitable for production code and should be removed or replaced with proper logging.
2. instructor/client_vertexai.py:30
  • Draft comment:
    The type hint for models should be updated to only include BaseModel or list[BaseModel].
def _create_vertexai_tool(models: Union[BaseModel, list[BaseModel]]) -> gm.Tool:
  • Reason this comment was not posted:
    Confidence changes required: 80%
    The function _create_vertexai_tool is designed to handle both single models and lists of models. However, the type hint for models includes Type, which is not necessary and could lead to confusion. The function should only accept BaseModel or list[BaseModel].
3. instructor/client_vertexai.py:104
  • Draft comment:
    The type hint for model should be updated to only include BaseModel or list[BaseModel].
def vertexai_process_response(_kwargs: dict[str, Any], model: Union[BaseModel, list[BaseModel]]):
  • Reason this comment was not posted:
    Confidence changes required: 80%
    The function vertexai_process_response is designed to handle both single models and lists of models. However, the type hint for model includes Type, which is not necessary and could lead to confusion. The function should only accept BaseModel or list[BaseModel].
4. instructor/client_vertexai.py:39
  • Draft comment:
    Debug print statements should be removed or replaced with proper logging.
# print(f"Debug - Model list: {[model.__name__ for model in model_list]}")
  • Reason this comment was not posted:
    Confidence changes required: 80%
    The function _create_vertexai_tool in instructor/client_vertexai.py has a debug print statement that should be removed or replaced with proper logging.
5. instructor/client_vertexai.py:13
  • Draft comment:
    Function _create_gemini_json_schema should have a return type annotation for clarity and consistency.
def _create_gemini_json_schema(model: BaseModel) -> dict[str, Any]:
  • Reason this comment was not posted:
    Confidence changes required: 80%
    The function _create_gemini_json_schema in instructor/client_vertexai.py lacks a return type annotation, which is important for clarity and consistency.
6. instructor/client_vertexai.py:30
  • Draft comment:
    Function _create_vertexai_tool should have a return type annotation for clarity and consistency.
def _create_vertexai_tool(models: Union[BaseModel, list[BaseModel], Type]) -> gm.Tool:
  • Reason this comment was not posted:
    Confidence changes required: 80%
    The function _create_vertexai_tool in instructor/client_vertexai.py lacks a return type annotation, which is important for clarity and consistency.
7. instructor/client_vertexai.py:104
  • Draft comment:
    Function vertexai_process_response should have a return type annotation for clarity and consistency.
def vertexai_process_response(_kwargs: dict[str, Any], model: Union[BaseModel, list[BaseModel], Type]) -> tuple[list[gm.Content], list[gm.Tool], ToolConfig]:
  • Reason this comment was not posted:
    Confidence changes required: 80%
    The function vertexai_process_response in instructor/client_vertexai.py lacks a return type annotation, which is important for clarity and consistency.
8. instructor/dsl/parallel.py:50
  • Draft comment:
    Function from_response should have a return type annotation for clarity and consistency.
def from_response(
  • Reason this comment was not posted:
    Confidence changes required: 80%
    The function from_response in instructor/dsl/parallel.py lacks a return type annotation, which is important for clarity and consistency.

Workflow ID: wflow_tx9ZvP1F3sMzVhxR


You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

@devjn devjn force-pushed the gemini-parallel-tools branch from c582618 to 41c1a78 Compare November 26, 2024 15:45
@jxnl jxnl requested a review from ivanleomk December 3, 2024 06:23
@ivanleomk
Copy link
Collaborator

Verified that all vertex tests are passing locally and updated the docs

tests/llm/test_vertexai/test_format.py ....                                                                                              [ 14%]
tests/llm/test_vertexai/test_message_parser.py ....                                                                                      [ 28%]
tests/llm/test_vertexai/test_modes.py ......                                                                                             [ 50%]
tests/llm/test_vertexai/test_retries.py ....                                                                                             [ 64%]
tests/llm/test_vertexai/test_simple_types.py ......                                                                                      [ 85%]
tests/llm/test_vertexai/test_stream.py ....                                                                                              [100%]

@ivanleomk ivanleomk merged commit caa300b into instructor-ai:main Dec 18, 2024
22 of 27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants