Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

10 more post-tuning OPs, regarding dialog data analysis from multiple aspects #513

Merged
merged 145 commits into from
Dec 26, 2024

Conversation

BeachWang
Copy link
Collaborator

@BeachWang BeachWang commented Dec 13, 2024

Add 10 more post-tuning OPs, for fine-gained analysis of dialog data:

  • 7 mapper ops:
    • LLM base:
      • dialog_intent_detection_mapper: Mapper to generate user's intent labels in feed back dialog data.
      • dialog_sentiment_detection_mapper: Mapper to generate user's sentiment labels in feed back dialog data.
      • dialog_sentiment_intensity_mapper: Mapper to predict user's sentiment intensity (from -5 to 5 in default
        prompt) in feed back dialog data.
      • dialog_topic_detection_mapper: Mapper to generate user's topic labels in feed back dialog data.
    • BERT base:
      • query_intent_detection_mapper: Mapper to predict user's Intent label in a query.
      • query_sentiment_detection_mapper: Mapper to predict user's sentiment label ('negative', 'neutral' and
        'positive') in a query.
      • query_topic_detection_mapper: Mapper to predict user's topic label in a query.
  • 1 aggregator op:
    • Merge similar meta tags to one tag.
  • 1 selector op:
    • tags_specified_field_selector: Select samples based on the tags of specified field.
  • 1 grouper op:
    • naive_reverse_grouper: Split bathed sample to samples.

Others:

  • Add MetaKeys for mapper op.
  • Remove unneccesary lazy loader.
  • Refine nested_set function.

@BeachWang BeachWang marked this pull request as ready for review December 20, 2024 04:04
@BeachWang BeachWang requested review from HYLcool and yxdyc December 20, 2024 04:05
@yxdyc yxdyc changed the title Dev/feed back analysis 10 more post-tuning OPs, regarding dialog data analysis from multiple aspects Dec 26, 2024
@yxdyc yxdyc added the dj:post-tuning issues/PRs about post-tuning scenarios label Dec 26, 2024
Copy link
Collaborator

@yxdyc yxdyc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yxdyc yxdyc merged commit 9466c73 into main Dec 26, 2024
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dj:op issues/PRs about some specific OPs dj:post-tuning issues/PRs about post-tuning scenarios documentation Improvements or additions to documentation enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants