Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LLM Templates - Created vision_gpt_explain_image llm application #964

Merged
merged 9 commits into from
Dec 3, 2023

Conversation

aybruhm
Copy link
Member

@aybruhm aybruhm commented Nov 29, 2023

Description

This PR introduces a new LLM application that utilizes Vision GPT to provide explanations for image(s).

Related Issue

Closes #953

@aybruhm aybruhm requested a review from mmabrouk November 29, 2023 12:42
Copy link
Member

@mmabrouk mmabrouk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for the PR @aybruhm !

I think there is one small confusion I see in the last examples and this one. The difference between configuration and input. For instance, in this case, the user is trying to create an llm application that compares different images. The configuration are the parameters they would iterate on in the playground to get a reliable app (let's say for comparing logo), these would include the temperature, max_tokens.. (https://platform.openai.com/docs/api-reference/chat/create), and models in case there are multiple options (in this case, I think there is only one model, so no need to include), and most importantly the prompt, since that is the main thing that they would play around with. On the other hand, the inputs for the application is the images (and not the prompt).

So first change, the system and user prompt are configuration. The images only are the inputs. Second change, we agreed yesterday that for now we will have the images in simple application and not in chat application, so no need for messages as an input.

Hope it's clear./

@aybruhm aybruhm marked this pull request as ready for review December 1, 2023 07:58
@aybruhm aybruhm requested a review from mmabrouk December 1, 2023 07:58
@aybruhm aybruhm changed the base branch from main to feat/file-input-support-playground December 1, 2023 08:03
Copy link
Member

@mmabrouk mmabrouk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR @aybruhm
However I think the PR is tainted, lots of files that have nothing to do with it. Can you please clean it up?

examples/vision_gpt_explain_image/app.py Outdated Show resolved Hide resolved
examples/vision_gpt_explain_image/app.py Outdated Show resolved Hide resolved
examples/vision_gpt_explain_image/app.py Outdated Show resolved Hide resolved
examples/vision_gpt_explain_image/app.py Outdated Show resolved Hide resolved
examples/vision_gpt_explain_image/app.py Outdated Show resolved Hide resolved
examples/vision_gpt_explain_image/app.py Outdated Show resolved Hide resolved
examples/vision_gpt_explain_image/app.py Outdated Show resolved Hide resolved
@aybruhm
Copy link
Member Author

aybruhm commented Dec 1, 2023

Thanks for the PR @aybruhm However I think the PR is tainted, lots of files that have nothing to do with it. Can you please clean it up?

I believe it's because the PR's base is feat/file-input-support-playground and not main.

@aybruhm aybruhm changed the base branch from feat/file-input-support-playground to main December 1, 2023 08:22
@aybruhm
Copy link
Member Author

aybruhm commented Dec 1, 2023

Thanks for the PR @aybruhm However I think the PR is tainted, lots of files that have nothing to do with it. Can you please clean it up?

I think it's because the PR's base is feat/file-input-support-playground and not 'main'.

Check again. I changed the PR's base to main and it's all good now.

cc: @mmabrouk

Copy link
Member

@mmabrouk mmabrouk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR @aybruhm
Just a small comment about the ag.init()

examples/vision_gpt_explain_image/app.py Outdated Show resolved Hide resolved
@mmabrouk
Copy link
Member

mmabrouk commented Dec 1, 2023

@aybruhm prettier is failing. Please make sure to use the same version we use in the workflow, I think that is the reason for the error

@aybruhm
Copy link
Member Author

aybruhm commented Dec 1, 2023

Weird, this is what I got:
image

@aybruhm aybruhm requested a review from mmabrouk December 1, 2023 17:28
@aybruhm
Copy link
Member Author

aybruhm commented Dec 1, 2023

I ran the exact command the "check web code" action workflow uses on my local machine and I got this:

image

@aybruhm
Copy link
Member Author

aybruhm commented Dec 1, 2023

@bekossy here also, thank you!

@mmabrouk
Copy link
Member

mmabrouk commented Dec 3, 2023

@MohammedMaaz -> template for #944

@mmabrouk mmabrouk merged commit 3fa26dc into main Dec 3, 2023
5 of 6 checks passed
@aakrem aakrem deleted the gh/vision-gpt-llm-app-temp branch January 19, 2024 08:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Create an example LLM application with vision gpt
3 participants