-
Notifications
You must be signed in to change notification settings - Fork 789
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: few shot example optimzier #1739
feat: few shot example optimzier #1739
Conversation
def add_example(self, input: BaseModel, output: BaseModel): | ||
# get json string for input | ||
input_json = input.model_dump_json() | ||
self._embeddings_of_examples.append(self.embeddings.embed_query(input_json)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In my experiments embedding user_input only and whole input (containing user_input, response,etc) gave almost identical results. But I think logically it makes more sense to only embed user_input. Although this has to be revisited again later. Let us leave it like this for now.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
but what if there is no user_input?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's something we have to figure out, ie what's the most accurate way to embed few shot examples. Let's leave it like this now.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall looks good. Logic and everything is perfect. Suggested few changes.
optimize with few short examples