Is there a way to support OpenAI's File Search, Assistants, and Runs?

[CiANSfi]

File Search, Assistants, and Runs: https://platform.openai.com/docs/assistants/tools/file-search?context=without-streaming

OpenAI has a vector db and allows for Assistants to search and retrieve from them. Their API allows you to use this OpenAI tool along with your own custom tools via function calling. I'm wondering if Magentic can already wrap around the File Search and Assistants APIs, if not explicitly out-the-box then perhaps in a hacky way for now (eg. is the only option to write the wrapper myself in a custom function and then use FunctionCall() on it?).

[jackmpcollins]

@CiANSfi I haven't used the OpenAI Assistants API yet or put much thought into how magentic might use it. If there's a specific place you think magentic could help I'd be interested to hear.

For converting functions to JSON schema and parsing the LLM output back into a FunctionCall check out FunctionCallFunctionSchema in magentic. The other FunctionSchema classes are for python classes and might also be of interest. Here's how OpenaiChatModel uses these to convert types and functions for the chat completion call.

magentic/src/magentic/chat_model/openai_chat_model.py

Lines 599 to 608 in 6747d99

	if output_types is None:
	output_types = [] if functions else cast(list[type[R]], [str])
	# TODO: Check that Function calls types match functions
	function_schemas = [FunctionCallFunctionSchema(f) for f in functions or []] + [
	function_schema_for_type(type_)
	for type_ in output_types
	if not is_origin_subclass(type_, STR_OR_FUNCTIONCALL_TYPE)
	]
	tool_schemas = [FunctionToolSchema(schema) for schema in function_schemas]

[CiANSfi]

@jackmpcollins The Assistants API (and accompanying Vector Database API) are not great, or maybe it's just that their documentation is lacking, or a little of both. The users over at the OpenAI community forum seem to agree.

I'll look into this class a bit more when I have the time and then I'll see if I can whip up something on my own. Actually I just realized I failed to mention that both APIs are under beta so I'm not sure if that changes how you would approach things (from a feature prioritization point of view).