RAG with Azure AI Search and Azure OpenAI

[pranav700]

Hi there, is there a plan to support for RAG chain with Azure AI Search and OpenAI(Azure SDK). I see the Phoenix support for OpenAI python library but not for AI Search and while testing this, the whole RAG chain doesn't come on the tracing UI

[Jgilhuly]

Hi @pranav700 - we should support much of this already! Here's a demo notebook using Azure AI Search + Azure OpenAI for RAG.

https://colab.research.google.com/drive/1F-VAdg2ZSEi93bVbCwMxWKcib6KRkxYo?usp=sharing

Let me know if you're seeing something different there in the UI

[pranav700]

Thanks for responding back to me. I don't use Llamaindex :) but just Azure SDK

I have tried with two type of config below. I use a Quart app to create the Chat completion RAG (async).

I expect the whole RAG to be in a drilldown where you can see the sequence but they are separate rows. Am I missing something here?

[Jgilhuly]

Ah I see. It looks like there are two things going on here:

1. If you're not using some orchestration framework like LlamaIndex or Langchain, then we won't know to group the spans into one request, which is why you're seeing two separate rows.
2. We're missing some span attributes on plain Azure AI Search calls, which is why you're seeing the "unknown" values in your second screenshot. I've opened a ticket to add support for plain Azure AI Search: [ENHANCEMENT] Azure AI Search Instrumentation #4118

To fix your issues:

1. You can do a bit of manual instrumentation to group the two calls into one trace: https://docs.arize.com/phoenix/tracing/how-to-tracing/manual-instrumentation/custom-spans#creating-spans . Unfortunately we can't do this automatically if you're not using an orchestration framework, because we have no way to know the two operations are related. Luckily adding the with clause in those docs can solve it.
2. You could manually add in the missing span attributes if you wanted using this code: https://docs.arize.com/phoenix/tracing/how-to-tracing/manual-instrumentation/custom-spans#add-semantic-attributes

[pranav700]

Thank you.

1. I am able to group all calls together using the decorator option. Pretty cool!

2. For the missing span, I guess I'll hit and trial with different attributes.

[Narayanan12345]

@pranav700 - Could you please help how to group all calls together using the decorator option. Kindly provide the sample code.

[pranav700]

Hi, code samples on decorators are on below arize-ai documentation and can be used as is.

https://docs.arize.com/phoenix/tracing/how-to-tracing/manual-instrumentation/custom-spans#creating-spans