Skip to content

Commit

Permalink
update the screenshot display method
Browse files Browse the repository at this point in the history
  • Loading branch information
openvino-dev-samples committed Nov 20, 2024
1 parent 044ec6b commit 2a20586
Show file tree
Hide file tree
Showing 3 changed files with 122 additions and 105 deletions.
2 changes: 1 addition & 1 deletion notebooks/multimodal-rag/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Constructing a RAG pipeline for text is relatively straightforward, thanks to th

To build a truly multimodal search for videos, you need to work with different modalities of a video like spoken content, visual. In this notebook, we showcase a Multimodal RAG pipeline designed for video analytics. It utilizes Whisper model to convert spoken content to text, CLIP model to generate multimodal embeddings, and Vision Language model (VLM) to process retrieved images and text messages. The following picture illustrates how this pipeline is working.

![image](https://github.com/user-attachments/assets/25ea940b-5415-4f5b-a271-58007ed01daa)
![image](https://github.com/user-attachments/assets/a8ebf3fc-7a34-416b-b744-609965792744)

## Notebook contents
The tutorial consists from following steps:
Expand Down
13 changes: 0 additions & 13 deletions notebooks/multimodal-rag/gradio_helper.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,6 @@ def make_demo(
build_index: Callable,
search: Callable,
run_fn: Callable,
load_img: Callable,
stop_fn: Callable,
):

Expand Down Expand Up @@ -103,12 +102,6 @@ def make_demo(
chatbot,
queue=True,
)
.then(
load_img,
[chatbot, image_list],
chatbot,
queue=True,
)
)
submit_click_event = (
submit.click(handle_user_message, [msg, chatbot], [msg, chatbot], queue=False)
Expand All @@ -124,12 +117,6 @@ def make_demo(
chatbot,
queue=True,
)
.then(
load_img,
[chatbot, image_list],
chatbot,
queue=True,
)
)
stop.click(
fn=stop_fn,
Expand Down
Loading

0 comments on commit 2a20586

Please sign in to comment.