Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CPU] Optimize PagedAttention performance #27718

Draft
wants to merge 4 commits into
base: master
Choose a base branch
from
Draft

[CPU] Optimize PagedAttention performance #27718

wants to merge 4 commits into from
+2,208 −90

Conversation

luo-cheng2021
Copy link
Contributor

@luo-cheng2021 luo-cheng2021 commented Nov 25, 2024 •
edited
Loading

Details:

  • Common optimization for PagedAttention
    • Generate more work items to avoid thread imbalance
    • Simplify scale subgraph
  • ChatGLM
    • Fix ROPE pattern
    • Fuse (FC)->Split->Reshape/(ROPE)->Reshape/(PageAttention)->Reshape->(MLP) into PagedAttention
  • **
  • Other: extend gen_pattern debug logs

Tickets:

@github-actions github-actions bot added category: CPU OpenVINO CPU plugin category: transformations OpenVINO Runtime library - Transformations labels Nov 25, 2024
@luo-cheng2021 luo-cheng2021 added WIP work in progress do_not_review and removed category: CPU OpenVINO CPU plugin category: transformations OpenVINO Runtime library - Transformations labels Nov 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
do_not_review WIP work in progress
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@luo-cheng2021