Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cudnn frontend sdpa wrong assertion in attention module with d_bias #115

Open
Skylion007 opened this issue Oct 8, 2024 · 2 comments
Open

Comments

@Skylion007
Copy link

Skylion007 commented Oct 8, 2024

Describe the bug
A clear and concise description of what the bug is.

Cudnn_frontend asserts on d_bias, even when the d_bias isn't used. See this PyTorch issue: pytorch/pytorch#137347 When upgrading the CUDNN_Frontend, code that used to work fine no longer works, and the assertion does not seem appropiate:

RuntimeError: cuDNN Frontend error: For cuDNN version below 9.5.0, dBias not support s_q/s_kv which aren't multiple of 64

pytorch/pytorch#137347

Expected behavior
A clear and concise description of what you expected to happen.
Previously working code would remain working. The assertion appears to be incorrect as the dbias isn't used:

System Environment (please complete the following information):

  • cudnn_frontend 1.7.0

To Reproduce
Steps to reproduce the behavior:

  1. Update PyTorch and update the submodule to 1.7.0
  2. Run the example in cuDNN dBias error starting on 10/2 nightly pytorch/pytorch#137347
  3. Watch the cudnn error propogate

Additional context
Add any other context about the problem here.

@Skylion007
Copy link
Author

Skylion007 commented Oct 9, 2024

Ping @eqy since you were working on a fix

@eqy
Copy link

eqy commented Oct 9, 2024

IIUC cuDNN is targeting a fix in a few days-week in an upcoming release

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants