Citations in custom RAG pipeline

[ClancyD-SHC]

Can we pass through RAG metadata to show as chunks like with the local RAG interface from a pipeline?

Any help appreciated!

[csvance]

The biggest problem with OpenWebUI / Pipelines for custom RAG applications is not supporting citations. There are cases even with 400B+ parameter models where the LLM does not correctly synthesize the information from the retrieved documents or understand that it does not have sufficient context to answer. In these cases, the user has no easy way to verify the information since pipelines can only return a string (or character generator for streaming output).

RAG without citations is dangerous. I'm open to trying to contribute something here, but the issue is with current pipeline abstractions there is no way to handle this.

The issue with including all of the context in the response string is this will quickly saturate the context length for the chat history. Ideally the history shouldn't contain all of the context information for every previous response, it should only contain the users messages and the agents response. Even if the context was included in a way that was filtered out of the history, its still not a great way to present the information to the user.

A potential workaround creating an HTTP endpoint on the custom RAG service to display all of the context, and including a link to it in a way that would be parsed out for the chat history, like [<citations>](https://custom-rag-service/citations/123) - This is still kind of ugly and requires extra work, but that would also add potential for user provided feedback for the reranking model which is something that even OpenWebUI native RAG support lacks.

Another potential option is defining a standard XML/JSON citation schema for use at the end of the response which OpenWebUI could parse into citations. The disadvantage of this is it only works with simple pipeline structures. Both sides of the pipeline would have to be aware of the format, which would make any kind of composition or scaffolding difficult. To me most of the power of pipelines to begin with is the ability to easily connect an entire application into OpenWebUI's interface. Only a single pipeline is needed for such cases realistically because all of the heavy lifting will happen in the applications code rather than OpenWebUI's pipeline.

[ClancyD-SHC]

We sidestepped this issue by modifying the openwebUI html sanitizer to let the /detail/> and </summary/> tags through so the citations appear as drop down boxes. We still have to manage what goes into the context in the pipeline but it was a quick and easy way to have the citations viewable. I think someone should open a ticket to ask for the tags to be extracted and displayed like internal tag citations.

[nkostop]

This sounds good for a temporary solution!

Is there an example for this?
Is it very difficult for example to use event emitters or something like that?

[etiennebonnafoux]

Has someone open the ticket for this issue ?

[nkostop]

Is this issue related? can we add citation there maybe?
#225

[etiennebonnafoux]

There is also #229